Predictive micro-service call chain anomaly detection method based on graph isomorphic network

Abstract

The invention discloses a predictive micro-service call chain anomaly detection method based on a graph isomorphic network. The method comprises the following steps: 1) converting structured csv format data into graph data; 2) determining a target network formed by a graph isomorphic network by using a designed target network determination method; 3) inputting the graph data into the target network to obtain a graph-level representation, and then inputting the graph data into the prediction network to enable the prediction network and the target network to generate the same graph-level representation; and 4) the input graph data respectively uses the prediction network and the target network to generate graph-level representations, and the difference degree of the graph-level representations generated by the two networks is calculated to serve as the abnormal degree of the graph data. According to the method, the graph isomorphic network technology and the micro-service call chain abnormal data are combined, so that the predictive micro-service call chain abnormal detection method based on the graph isomorphic network is formed, and the accuracy of micro-service call chain abnormal detection is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of graph isomorphic network anomaly detection, and in particular to a predictive microservice call chain anomaly detection method based on graph isomorphic networks. Background Art

[0002] With the continuous development of anomaly detection technology, various anomaly detection algorithms have emerged, providing feasible solutions for rapid anomaly location in massive data. Traditional anomaly detection methods are mainly used to detect anomalies in structured data, while graph data anomaly detection lacks effective means. Therefore, graph-level anomaly detection for graph data such as microservice call chains has become an urgent need.

[0003] At present, the graph-level anomaly detection framework based on graph neural networks can be mainly divided into two categories: deep single classification based methods and two-stage methods. Deep single classification methods require that the embedding vectors of normal data are as clustered as possible and the embedding vectors of anomaly data are scattered around the periphery of normal data. However, in some cases, the characteristics of normal data are diverse, and the embedding vectors of normal data are usually clustered into multiple clusters. Since it is difficult for graph neural networks to generate a vector representation close to the center of graph embedding for normal data, the detection effect of the model is difficult to further improve. The existing two-stage methods have achieved good graph-level anomaly detection performance by applying existing anomaly detection methods such as isolation forest (iForest), local anomaly factor (LOF), and one-class support vector machine (OCSVM) combined with advanced graph kernels such as WL kernel and propagation kernel (PK). However, the graph representation of this type of method is independent of the anomaly detector learning, which may lead to poor representation effect. In addition, although there are studies dedicated to extracting graph-level patterns of graph data, their performance is still limited by the complexity of graph-level patterns. Overall, a major challenge in the field of graph-level anomaly detection is to design an anomaly detection method to seek better anomaly detection performance. Summary of the invention

[0004] The purpose of the present invention is to solve the problem that distributed call chain data is difficult to detect using existing anomaly detection methods as graph data. In order to solve the above problem, a predictive microservice call chain anomaly detection method based on graph isomorphic network is proposed, which can effectively detect anomalies in distributed call chain data, and a new predictive graph-level anomaly detection framework is proposed for this purpose, so that the existing anomaly detection method based on graph neural network is no longer limited to detection using deep single classification method.

[0005] To achieve the above purpose, the technical solution provided by the present invention is: a predictive microservice call chain anomaly detection method based on a graph isomorphic network, comprising the following steps:

[0006] 1) Use the public Train_ticket microservice call chain dataset, which is formatted as structured CSV data, and convert the structured CSV data into graph data;

[0007] 2) Taking the graph data obtained in step 1) as input, a target network based on a graph isomorphic network is obtained using a target network determination method;

[0008] 3) Using the graph data obtained in step 1) and the target network obtained in step 2) to train a prediction network based on a graph isomorphism network, the goal of the training is to minimize the difference between the graph representation generated by the prediction network and the graph representation generated by the target network. After training, a set of target networks and prediction networks that can generate similar graph-level representations are obtained;

[0009] 4) Apply a set of target networks and prediction networks obtained in step 3), input the graph data used for testing into the trained target network and prediction network at the same time, generate a target network graph-level representation and a prediction network graph-level representation respectively; the difference between the target network graph-level representation and the prediction network graph-level representation is used to measure the abnormality of the graph data.

[0010] Further, in step 1), the publicly available Train_ticket microservice call chain data set is read from the local and formatted into structured csv format data, wherein the structured csv format data includes server node number, next hop server node number, request type, response status code, response time, and network transmission time service call information;

[0011] The steps to convert structured CSV format data into graph data are as follows: First, the structured CSV format data is classified according to the microservice call chain number. The data with the same number belongs to a graph. Then, the node structure and edge structure of a single microservice call chain data are determined according to the server node number and the next-hop server node number. Finally, other parameters in the structured CSV format data are integrated into a vector to represent node features or edge features.

[0012] Further, in step 2), the target network determination method for determining the target network includes the following steps:

[0013] First, several target networks based on graph isomorphic networks are randomly generated. The target network based on graph isomorphic networks consists of multiple layers of graph isomorphic networks. A single-layer graph isomorphic network is represented as:

[0014]

[0015] In the formula, represents the input embedding representation of node i in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, represents the input embedding representation of node i in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, j∈N(i) represents the first-order neighbor of node i, represents the input embedding representation of node j in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, ε l is the hyperparameter of the l-th layer graph isomorphism network, ρ is the activation function ReLU, and MLP is the multi-layer perceptron;

[0016] Then, the graph data is used as the input of the randomly generated target network based on the graph isomorphic network, and multiple graph-level feature vector representations are obtained. The graph representation vector dispersion dis is calculated based on the feature vector representation. The calculation method is as follows:

[0017]

[0018] In the formula, n is the number of vectors, represents the tth component of the kth eigenvector, represents the average value of the tth component of all eigenvectors, It represents the variance of the tth component of all vectors, T represents the number of vector components, and the dis value is obtained by averaging the components of T vectors. The larger the dis value, the smaller the similarity of multiple vectors, and the corresponding target network is better.

[0019] Further, in step 3), the graph data and the target network based on the graph isomorphic network are used to train the prediction network based on the graph isomorphic network, wherein the prediction network based on the graph isomorphic network is composed of multiple layers of graph isomorphic networks, and a single layer of graph isomorphic network is represented as follows:

[0020]

[0021] In the formula, represents the input embedding representation of node i in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, represents the input embedding representation of node i in the lth hidden layer of the prediction network based on graph isomorphism network, j∈N(i) represents the first-order neighbor of node i, represents the input embedding representation of node j in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, ε l is the hyperparameter of the l-th layer graph isomorphism network, ρ is the activation function ReLU, and MLP is the multi-layer perceptron;

[0022] Both the target network based on the graph isomorphic network and the prediction network based on the graph isomorphic network use the maximum pooling readout function to generate graph-level representations. h G :

[0023]

[0024] In the formula, represents the vector representation of node i in the embedding representation generated by the target network based on the graph isomorphism network, h i represents the vector representation of node i in the embedding representation generated by the prediction network based on the graph isomorphism network, The i in the graph refers to any node in the graph G, and max{·} is an aggregation function used to find the maximum value;

[0025] The training goal of the prediction network based on the graph isomorphism network is to minimize the following loss function L:

[0026]

[0027] Furthermore, in step 4), the degree of abnormality of the graph data is determined by calculating the difference between the target network graph-level representation and the predicted network graph-level representation. Specifically, the abnormality score of the graph is calculated by the following formula:

[0028]

[0029] Among them, the larger the anomaly score, the more likely the graph data is abnormal.

[0030] Compared with the prior art, the present invention has the following advantages and beneficial effects:

[0031] 1. The present invention converts the microservice call chain data into graph data. Compared with unstructured data, graph data can facilitate seeing more associations between data and thus contain richer information.

[0032] 2. The present invention designs a target network determination method to ensure that the target network has better representation capabilities for graph data.

[0033] 3. The present invention designs a predictive graph-level anomaly detection method, which provides a new anomaly detection model structure for the existing graph-level anomaly detection field. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] Figure 1 Schematic diagram of the method of the present invention.

[0035] Figure 2 This is a sample diagram of microservice call chain data.

[0036] Figure 3 Flow chart of the method of the present invention. DETAILED DESCRIPTION

[0037] The present invention will be further described below in conjunction with specific examples, but the embodiments of the present invention are not limited thereto.

[0038] This embodiment discloses a predictive microservice call chain anomaly detection method based on a graph isomorphic network, the architecture is as follows Figure 1 As shown. Graph-level anomaly detection is performed on microservice call chain data. The graph data used in this method is as follows: Figure 2 The relationship between the various steps in the method is shown in Figure 3 As shown, it includes the following steps:

[0039] 1) Read the public Train_ticket microservice call chain data set from the local machine and format it into structured csv format data, where the structured csv format data includes server node number, next hop server node number, request type, response status code, response time, and network transmission time service call information;

[0040] The steps to convert structured CSV format data into graph data are as follows: First, the structured CSV format data is classified according to the microservice call chain number. The data with the same number belongs to a graph. Then, the node structure and edge structure of a single microservice call chain data are determined according to the server node number and the next-hop server node number. Finally, other parameters in the structured CSV format data are integrated into a vector to represent node features or edge features.

[0041] 2) The target network determination method for determining the target network in step 1) includes the following steps:

[0042] First, several target networks based on graph isomorphic networks are randomly generated. The target network based on graph isomorphic networks consists of multiple layers of graph isomorphic networks. A single-layer graph isomorphic network is represented as:

[0043]

[0044] In the formula, represents the input embedding representation of node i in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, represents the input embedding representation of node i in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, j∈N(i) represents the first-order neighbor of node i, represents the input embedding representation of node j in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, ε l is the hyperparameter of the l-th layer graph isomorphism network, ρ is the activation function ReLU, and MLP is the multi-layer perceptron;

[0045] Then, the graph data is used as the input of the randomly generated target network based on the graph isomorphic network, and multiple graph-level feature vector representations are obtained. The graph representation vector dispersion dis is calculated based on the feature vector representation. The calculation method is as follows:

[0046]

[0047] In the formula, n is the number of vectors, represents the tth component of the kth eigenvector, represents the average value of the tth component of all eigenvectors, It represents the variance of the tth component of all vectors, T represents the number of vector components, and the dis value is obtained by averaging the components of T vectors. The larger the dis value, the smaller the similarity of multiple vectors, and the corresponding target network is better.

[0048] 3) Use graph data and a target network based on a graph isomorphic network to train a prediction network based on a graph isomorphic network, where the prediction network based on a graph isomorphic network consists of multiple layers of graph isomorphic networks, and a single-layer graph isomorphic network is represented as follows:

[0049]

[0050] In the formula, represents the input embedding representation of node i in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, represents the input embedding representation of node i in the lth hidden layer of the prediction network based on graph isomorphism network, j∈N(i) represents the first-order neighbor of node i, represents the input embedding representation of node j in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, ε l is the hyperparameter of the l-th layer graph isomorphism network, ρ is the activation function ReLU, and MLP is the multi-layer perceptron;

[0051] Both the target network based on the graph isomorphic network and the prediction network based on the graph isomorphic network use the maximum pooling readout function to generate graph-level representations. h G :

[0052]

[0053] In the formula, represents the vector representation of node i in the embedding representation generated by the target network based on the graph isomorphism network, h i represents the vector representation of node i in the embedding representation generated by the prediction network based on the graph isomorphism network, The i in the graph refers to any node in the graph G, and max{·} is an aggregation function used to find the maximum value;

[0054] The training goal of the prediction network based on the graph isomorphism network is to minimize the following loss function L:

[0055]

[0056] 4) The degree of abnormality of the graph data is determined by calculating the difference between the target network graph-level representation and the predicted network graph-level representation. Specifically, the abnormality score of the graph is calculated using the following formula:

[0057]

[0058] Among them, the larger the anomaly score, the more likely the graph data is abnormal.

[0059] The above embodiments are preferred implementation modes of the present invention, but the implementation modes of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations made without departing from the spirit and principle of the present invention should be equivalent replacement methods and are included in the protection scope of the present invention.

Claims

1. A predictive microservice call chain anomaly detection method based on graph isomorphic networks, characterized in that: The following steps are involved: 1) Use the public Train_ticket microservice call chain dataset, which is formatted as structured CSV data, and convert the structured CSV data into graph data; 2) Taking the graph data obtained in step 1) as input, a target network based on a graph isomorphic network is obtained using a target network determination method; 3) Using the graph data obtained in step 1) and the target network obtained in step 2) to train a prediction network based on a graph isomorphism network, the goal of the training is to minimize the difference between the graph representation generated by the prediction network and the graph representation generated by the target network. After training, a set of target networks and prediction networks that can generate similar graph-level representations are obtained; 4) Apply a set of target networks and prediction networks obtained in step 3), input the graph data used for testing into the trained target network and prediction network at the same time, generate a target network graph-level representation and a prediction network graph-level representation respectively; the difference between the target network graph-level representation and the prediction network graph-level representation is used to measure the abnormality of the graph data.

2. The predictive microservice call chain anomaly detection method based on graph isomorphic network according to claim 1 is characterized in that: In step 1), the publicly available Train_ticket microservice call chain data set is read locally and formatted into structured csv format data, wherein the structured csv format data includes server node number, next hop server node number, request type, response status code, response time, and network transmission time service call information; The steps to convert structured CSV format data into graph data are as follows: First, the structured CSV format data is classified according to the microservice call chain number. The data with the same number belongs to a graph. Then, the node structure and edge structure of a single microservice call chain data are determined according to the server node number and the next-hop server node number. Finally, other parameters in the structured CSV format data are integrated into a vector to represent node features or edge features.

3. The predictive microservice call chain anomaly detection method based on graph isomorphic network according to claim 1 is characterized in that: In step 2), the target network determination method for determining the target network includes the following steps: First, several target networks based on graph isomorphic networks are randomly generated. The target network based on graph isomorphic networks consists of multiple layers of graph isomorphic networks. A single-layer graph isomorphic network is represented as: In the formula, represents the input embedding representation of node i in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, represents the input embedding representation of node i in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, j∈N(i) represents the first-order neighbor of node i, represents the input embedding representation of node j in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, ε l is the hyperparameter of the l-th layer graph isomorphism network, ρ is the activation function ReLU, and MLP is the multi-layer perceptron; Then, the graph data is used as the input of the randomly generated target network based on the graph isomorphic network, and multiple graph-level feature vector representations are obtained. The graph representation vector dispersion dis is calculated based on the feature vector representation. The calculation method is as follows: In the formula, n is the number of vectors, represents the tth component of the kth eigenvector, represents the average value of the tth component of all eigenvectors, It represents the variance of the tth component of all vectors, T represents the number of vector components, and the dis value is obtained by averaging the components of T vectors. The larger the dis value, the smaller the similarity of multiple vectors, and the better the corresponding target network.

4. The predictive microservice call chain anomaly detection method based on graph isomorphic network according to claim 1 is characterized in that: In step 3), the graph data and the target network based on the graph isomorphic network are used to train the prediction network based on the graph isomorphic network, wherein the prediction network based on the graph isomorphic network is composed of multiple layers of graph isomorphic networks, and a single-layer graph isomorphic network is represented as follows: In the formula, represents the input embedding representation of node i in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, represents the input embedding representation of node i in the lth hidden layer of the prediction network based on graph isomorphism network, j∈N(i) represents the first-order neighbor of node i, represents the input embedding representation of node j in the (l-1)th hidden layer of the prediction network based on the graph isomorphism network, ε l is the hyperparameter of the l-th layer graph isomorphism network, ρ is the activation function ReLU, and MLP is the multi-layer perceptron; Both the target network based on the graph isomorphic network and the prediction network based on the graph isomorphic network use the maximum pooling readout function to generate graph-level representations. h G : In the formula, represents the vector representation of node i in the embedding representation generated by the target network based on the graph isomorphism network, h i represents the vector representation of node i in the embedding representation generated by the prediction network based on the graph isomorphism network, The i in the graph refers to any node in the graph G, and max{·} is an aggregation function used to find the maximum value; The training goal of the prediction network based on the graph isomorphism network is to minimize the following loss function L:

5. The predictive microservice call chain anomaly detection method based on graph isomorphic network according to claim 1 is characterized in that: In step 4), the degree of abnormality of the graph data is determined by calculating the difference between the target network graph-level representation and the predicted network graph-level representation. Specifically, the abnormality score of the graph is calculated by the following formula: Among them, the larger the anomaly score, the more likely the graph data is abnormal.

Images