Microservice system fault diagnosis method, device and equipment

Abstract

The present application provides a method, apparatus, and device for diagnosing faults in a microservice system. The method includes: constructing a service tracking graph; based on a graph attention network, determining a first embedded code according to the network structure and node attributes of the service tracking graph, and reconstructing the network structure according to the first embedded code; based on the graph attention network, determining a first embedded code of a node according to the network structure and node attributes of the service tracking graph, and determining the reconstructed network structure according to the first embedded code of the node; based on a long short-term memory network, determining a second embedded code of the node according to the node attributes of the service tracking graph, and determining the reconstructed node attributes according to the first embedded code and the second embedded code of the node; determining the loss value of each node according to the network structure and node attributes of the abnormal service tracking graph before and after reconstruction, and marking the node with the largest loss value as the abnormal root cause node. Through this application, the accuracy and efficiency of microservice system fault diagnosis are improved.

Description

Technical Field

[0001] The present application relates to the field of system operation and maintenance technology, and specifically to a microservice system fault diagnosis method, device and equipment. Background Art

[0002] Microservice Architecture (MSA) is a software architecture design that breaks down applications into small, independently deployable service units. Each service unit can be independently developed, deployed, and scaled. In a microservice system, the interactions and dependencies between services are complex, and the system's operating state is constantly changing, which poses challenges for system troubleshooting.

[0003] Fault diagnosis involves two aspects: anomaly detection and root cause location. Anomaly detection, as a crucial component of software operations and maintenance, significantly impacts the operational quality and reliability of software systems. Root cause location, based on a correlation model, utilizes algorithms such as graph theory, probability statistics, and game theory to accurately locate the fault. Current microservice system fault diagnosis methods suffer from low accuracy and efficiency. Summary of the Invention

[0004] The present application provides a microservice system fault diagnosis method, device and equipment, which can solve the technical problems of low accuracy and efficiency existing in the prior art.

[0005] In a first aspect, an embodiment of the present application provides a microservice system fault diagnosis method, the microservice system fault diagnosis method comprising:

[0006] Construct a service tracking graph, where each user request corresponds to a service tracking graph. The service tracking graphs corresponding to multiple user requests for the same operation form a detection sequence. In the service tracking graph, each node represents a service, and each edge represents the call relationship between the source service and the target service. Node attributes include performance data of the service call.

[0007] Based on the graph attention network, the first embedding code of the node is determined according to the network structure and node attributes of the service tracking graph, and the reconstructed network structure is determined according to the first embedding code of the node;

[0008] Based on the long short-term memory network, a second embedding code of the node is determined according to the node attributes of the service trace graph, and a reconstructed node attribute is determined according to the first embedding code and the second embedding code of the node. The service trace graphs of the same detection sequence are sequentially input into the corresponding long short-term memory network according to the order of user requests from earliest to latest.

[0009] Determine the total loss value of all nodes based on the network structure and node attributes before and after the reconstruction of the service trace graph, and mark the service trace graph with a total loss value greater than the loss threshold as an abnormal service trace graph;

[0010] The loss value of each node is determined based on the network structure and node attributes before and after reconstruction of the abnormal service tracking graph, and the node with the largest loss value is marked as the abnormal root cause node.

[0011] Furthermore, in one embodiment, the step of constructing a service tracking graph includes:

[0012] Collect trace data and time series performance metrics data generated during the service call process of the microservice system. Each user request has a unique trace ID, and each service call has a unique span ID.

[0013] For each trace ID, trace data and timing performance metrics data are aggregated, nodes and edges are constructed based on the trace data, and node attributes are constructed based on the timing performance metrics data. Each span ID corresponds to a node.

[0014] Furthermore, in one embodiment, the step of determining the first embedding code of the node based on the network structure and node attributes of the service tracking graph based on the graph attention network includes:

[0015] Convert node attributes of the service tracking graph into low-dimensional latent representation;

[0016] Determine the attention weight of each node to its neighboring nodes;

[0017] Each node is regarded as the central node. According to the attention weight of the central node to its adjacent nodes, the low-dimensional potential representations of the adjacent nodes are normalized and weighted to obtain the first embedded code of the central node.

[0018] Furthermore, in one embodiment, the total loss value of all nodes is calculated as follows:

[0019]

[0020] node The loss value is calculated as follows:

[0021]

[0022] in, and Represent the adjacency matrix of the network structure before and after reconstruction, and The attribute matrices representing the node attributes before and after reconstruction, Represents the weight of the difference in network structure before and after reconstruction, represents the Hadamard product, represents the Frobenius norm, and The penalty coefficient matrices representing the differences in network structure and node attributes before and after reconstruction are:

[0023]

[0024]

[0025] in, Representation node With node The connection relationship, 0 means no connection, 1 means connection, Representation node The kth attribute value of , 0 indicates that the attribute is missing, a>1, b>1.

[0026] Furthermore, in one embodiment, before the step of marking the service trace graph with a total loss value greater than the loss threshold as an abnormal service trace graph, the method further includes:

[0027] The loss threshold of the current service tracing graph is determined according to the total loss values ​​of a plurality of related service tracing graphs, wherein the related service tracing graphs are earlier service tracing graphs in the detection sequence where the current service tracing graph is located.

[0028] Furthermore, in one embodiment, before the step of determining the loss threshold of the current service trace graph according to the total loss values ​​of the multiple related service trace graphs, the method further includes:

[0029] If the current service tracking graph is the earliest preset number of service tracking graphs in the detection sequence, the preset value is determined as the loss threshold of the current service tracking graph; otherwise, the preset number of service tracking graphs before the current service tracking graph are determined as the relevant service tracking graphs.

[0030] Furthermore, in one embodiment, before the step of determining the loss threshold of the current service trace graph according to the total loss values ​​of the multiple related service trace graphs, the method further includes:

[0031] If the current service tracking graph is a service tracking graph within the preset time length after the detection sequence is established, the preset value is determined as the loss threshold of the current service tracking graph; otherwise, multiple service tracking graphs within the preset time length before the current service tracking graph are determined as related service tracking graphs.

[0032] Furthermore, in one embodiment, in the step of determining the loss threshold of the current service tracking graph based on the total loss values ​​of multiple related service tracking graphs, the loss threshold of the current service tracking graph is equal to the sum of the average value and three times the standard deviation of the total loss values ​​of the multiple related service tracking graphs.

[0033] In a second aspect, an embodiment of the present application further provides a microservice system fault diagnosis device, the microservice system fault diagnosis device comprising:

[0034] A graph construction module is used to construct a service tracking graph. Each user request corresponds to a service tracking graph. The service tracking graphs corresponding to multiple user requests for the same operation constitute a detection sequence. In the service tracking graph, each node represents a service, and each edge represents the call relationship from the source service to the target service. Node attributes include performance data of the service call.

[0035] A structure reconstruction module is used to determine a first embedding code of a node based on the network structure and node attributes of the service tracking graph based on the graph attention network, and to determine a reconstructed network structure based on the first embedding code of the node;

[0036] An attribute reconstruction module is configured to determine a second embedded code of a node based on the node attributes of a service tracking graph based on a long short-term memory network, and to determine a reconstructed node attribute based on the first embedded code and the second embedded code of the node, wherein the service tracking graphs of the same detection sequence are sequentially input into the corresponding long short-term memory network in the order of user requests, from earliest to latest;

[0037] The anomaly detection module determines the total loss value of all nodes based on the network structure and node attributes of the service trace graph before and after reconstruction, and marks the service trace graph with a total loss value greater than the loss threshold as an abnormal service trace graph;

[0038] The root cause location module is used to determine the loss value of each node based on the network structure and node attributes before and after the reconstruction of the abnormal service tracking graph, and mark the node with the largest loss value as the abnormal root cause node.

[0039] In a third aspect, an embodiment of the present application further provides a microservice system fault diagnosis device, which includes a processor, a memory, and a microservice system fault diagnosis program stored on the memory and executable by the processor, wherein when the microservice system fault diagnosis program is executed by the processor, the steps of the above-mentioned microservice system fault diagnosis method are implemented.

[0040] In this application, a service tracking graph is constructed by analyzing the service call relationship and performance data of the microservice system, and the network structure and node attributes are analyzed by the graph attention network to capture the call dependency relationship between services, thereby reconstructing the network structure, and the node attributes are analyzed by the long short-term memory network to identify abnormal changes in performance indicators over time, thereby reconstructing the node attributes, and calculating the loss value by the network structure and node attributes before and after reconstruction. The service tracking graph with a total loss value greater than the loss threshold is marked as an abnormal service tracking graph, thereby realizing fault anomaly detection, and the node with the largest loss value in the abnormal service tracking graph is marked as the abnormal root cause node, thereby realizing fault root cause location. Through this application, the accuracy and efficiency of microservice system fault diagnosis are improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 This is a flowchart of a method for diagnosing a fault in a microservice system according to an embodiment of the present application;

[0042] Figure 2 Schematic diagram of an adjacency matrix and an attribute matrix in one embodiment of the present application;

[0043] Figure 3 This is a schematic diagram of the principle of a microservice system fault diagnosis method in one embodiment of the present application;

[0044] Figure 4 This is a schematic diagram of the functional modules of a microservice system fault diagnosis device in one embodiment of the present application;

[0045] Figure 5 This is a schematic diagram of the hardware structure of the microservice system fault diagnosis device involved in the embodiment of the present application. DETAILED DESCRIPTION

[0046] In order to enable those skilled in the art to better understand the present invention, the following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0047] In order to make the objectives, technical solutions and advantages of this application clearer, the implementation methods of this application will be further described in detail below with reference to the accompanying drawings.

[0048] In a first aspect, an embodiment of the present application provides a method for diagnosing faults in a microservice system.

[0049] Figure 1 A flow chart of a microservice system fault diagnosis method in one embodiment of the present application is shown.

[0050] Reference Figure 1 In one embodiment, a microservice system fault diagnosis method includes the following steps:

[0051] S11. Construct a service tracking graph, where each user request corresponds to a service tracking graph. The service tracking graphs corresponding to multiple user requests for the same operation constitute a detection sequence. In the service tracking graph, each node represents a service, and each edge represents a call relationship from a source service to a target service. Node attributes include performance data of the service call.

[0052] S12. Based on the graph attention network, determine the first embedding code of the node according to the network structure and node attributes of the service tracking graph, and determine the reconstructed network structure according to the first embedding code of the node;

[0053] S13. Based on the long short-term memory network, determine a second embedded code of the node according to the node attributes of the service tracking graph, and determine a reconstructed node attribute according to the first embedded code and the second embedded code of the node, wherein the service tracking graphs of the same detection sequence are sequentially input into the corresponding long short-term memory network in the order of user requests from earliest to latest;

[0054] S14. Determine the total loss value of all nodes based on the network structure and node attributes before and after the reconstruction of the service tracking graph, and mark the service tracking graph with a total loss value greater than the loss threshold as an abnormal service tracking graph;

[0055] S15. Determine the loss value of each node based on the network structure and node attributes before and after the reconstruction of the abnormal service tracking graph, and mark the node with the largest loss value as the abnormal root cause node.

[0056] The service tracking graph uses a property graph data model to record the services involved in each user request, the call relationships between services, and the performance data of the service calls, facilitating subsequent troubleshooting. For example, performance data includes average response time, average CPU usage, and memory usage.

[0057] Graph Attention Networks (GATs) aim to improve the way graph neural networks assign weights when processing node features by introducing an attention mechanism, thereby more flexibly learning the dependencies between nodes in the graph. In traditional graph neural networks, the information of neighboring nodes is often merged with equal weights, while graph attention networks allow the network to dynamically learn the importance of different neighboring nodes to the central node. The core idea of ​​graph attention networks is to assign different attention weights to each neighboring node during the information aggregation process. These weights reflect the relative importance of neighboring nodes to the central node. Graph attention networks can be used to predict possible edges or missing links in a graph. This embodiment uses graph attention networks to reconstruct the network structure of the service tracking graph.

[0058] The Long Short-Term Memory Network (LSTM) is a special type of Recurrent Neural Network (RNN). The LSTM was originally designed to address the vanishing or exploding gradient problems of traditional RNNs when dealing with long-term dependencies, enabling more effective learning of long-term dependent sequence data. At each time step, the LSTM first calculates the opening levels of the forget gate, input gate, and output gate based on the input data and the previous hidden state. The cell state is then updated based on the input gate, the current input data, and the forget gate's adjustment to the previous cell state. Finally, the output gate determines which information in the cell state will be used to generate the hidden state for the current time step. This hidden state can be used for further calculations or as the final output. LSTMs can be used for time series prediction. In this embodiment, the outputs of the LSTM and graph attention networks are combined to reconstruct node attributes in a service tracking graph.

[0059] In terms of fault anomaly detection, this embodiment calculates the total loss value of all nodes based on the network structure and node attributes before and after reconstruction. The larger the total loss value, the greater the difference in network structure and node attributes before and after reconstruction. The service tracking graph with a total loss value greater than the loss threshold is marked as an abnormal service tracking graph. Since the service tracking graph corresponds to the user request one-to-one, the abnormal service tracking graph can be used to know that the corresponding user request has an abnormal response in the microservice system.

[0060] In terms of locating the root cause of the fault, this embodiment further calculates the loss value of each node. The larger the loss value of a single node, the greater the difference in the connection relationship and node attributes of the node before and after reconstruction. The node with the largest loss value is marked as the abnormal root cause node. Since the node corresponds one-to-one to the service, the abnormal root cause node can be used to know that the corresponding service is the root cause of the abnormality.

[0061] In this embodiment, a service trace graph is constructed by analyzing the service call relationships and performance data of a microservice system. A graph attention network is used to analyze the network structure and node attributes to capture the call dependencies between services, thereby reconstructing the network structure. A long-short-term memory network is used to analyze node attributes to identify abnormal changes in performance indicators over time, thereby reconstructing the node attributes. Loss values ​​are calculated based on the network structure and node attributes before and after reconstruction. Service trace graphs with a total loss value greater than a loss threshold are marked as abnormal service trace graphs, thereby achieving fault anomaly detection. The node with the largest loss value in the abnormal service trace graph is marked as the abnormal root cause node, thereby achieving fault root cause location. This embodiment improves the accuracy and efficiency of microservice system fault diagnosis.

[0062] Furthermore, in one embodiment, the step of constructing a service tracking graph includes:

[0063] Collect trace data and time series performance metrics data generated during the service call process of the microservice system. Each user request has a unique trace ID, and each service call has a unique span ID.

[0064] For each trace ID, trace data and timing performance metrics data are aggregated, nodes and edges are constructed based on the trace data, and node attributes are constructed based on the timing performance metrics data. Each span ID corresponds to a node.

[0065] Trace data primarily refers to every call in the service call chain. It details the complete path of a request from its entry into the system to its return response, including all services passed through, the order in which services are called, and the processing time of each service. This data is crucial for understanding the internal workings of the system and locating the root cause of problems.

[0066] Time series performance metrics focus on the performance of services and systems over time, such as service response time, throughput, and error rate. These metrics change over time. This data is used to monitor the real-time status of the system and identify performance bottlenecks or anomalies.

[0067] Furthermore, in one embodiment, the step of determining the first embedding code of the node based on the network structure and node attributes of the service tracking graph based on the graph attention network includes:

[0068] Convert node attributes of the service tracking graph into low-dimensional latent representation;

[0069] Determine the attention weight of each node to its neighboring nodes;

[0070] Each node is regarded as the central node. According to the attention weight of the central node to its adjacent nodes, the low-dimensional potential representations of the adjacent nodes are normalized and weighted to obtain the first embedded code of the central node.

[0071] Specifically, the node attributes of the service tracking graph are converted into a low-dimensional latent representation:

[0072]

[0073] in, is the activation function, such as ReLU or Tanh, and are the weights and biases learned by the encoder.

[0074] After conversion, the graph attention layer of node attributes is represented as , N is the number of nodes. A shared attention mechanism is performed on the graph attention layer to aggregate representations from neighboring nodes and determine the attention weight of each node to its neighboring nodes:

[0075]

[0076] in, Representation node For Node The attention weight of the function attn(•) is the weight shared by all nodes, and the LeakyReLU function can be used, so It can be written as:

[0077]

[0078] in, and Represents the neural network weights that parameterize the attention mechanism, shared among all nodes, D is the feature dimension, and the operator || represents concatenation.

[0079] The node When serving as a central node, Normalize the attention weights of its adjacent nodes to obtain the normalized attention weights:

[0080]

[0081] The low-dimensional potential representation of the adjacent nodes is weighted according to the normalized attention weight to obtain the node The first embedding code:

[0082]

[0083] in, Representation node The set of adjacent nodes.

[0084] Take each node as the central node and execute the above steps to get the set of the first embedded codes of all nodes .

[0085] Optionally, the network structure of the service tracing graph is represented by an adjacency matrix, and the node attributes of the service tracing graph are represented by an attribute matrix. For example, assuming that the service tracing graph has N nodes and each node has M attributes, the network structure of the service tracing graph is represented by an N×N adjacency matrix, and the node attributes of the service tracing graph are represented by an N×M attribute matrix.

[0086] Figure 2 A schematic diagram of an adjacency matrix and an attribute matrix in an embodiment of the present application is shown.

[0087] Reference Figure 2 The network structure consists of four nodes, A, B, C, and D, and three directed edges, A->B, B->C, and B->D. The information contained in the network is that user requests involve four services corresponding to A, B, C, and D. The service corresponding to A calls the service corresponding to B, and the service corresponding to B calls the services corresponding to C and D, which are represented by a 4×4 adjacency matrix. Each node has three attributes: average response time, average CPU usage, and memory usage, which are represented by a 4×3 attribute matrix.

[0088] In particular, although the service call relationship is a directed edge, it is represented by an undirected graph adjacency matrix in this embodiment, thereby ensuring that each node in the graph attention network can find adjacent nodes.

[0089] Figure 3 A schematic diagram illustrating the principles of a microservice system fault diagnosis method in one embodiment of the present application is shown.

[0090] Reference Figure 3 , the adjacency matrix Represents the network structure before reconstruction, attribute matrix Represents the node attributes before reconstruction, and the adjacency matrix and the attribute matrix Input graph attention network encoder (GAT encoder) to get the first embedding code , the attribute matrix Input the long short-term memory network encoder (LSTM encoder) to get the second embedding code , the first embedded code Input the graph attention network decoder (GAT decoder) to obtain the adjacency matrix , represents the reconstructed network structure, the first embedding code and the second embedded code Input the long short-term memory network decoder (LSTM decoder) to obtain the attribute matrix , represents the reconstructed node attributes. According to the adjacency matrix and Calculate the structure reconstruction loss according to the attribute matrix and Calculate the attribute reconstruction loss.

[0091] It should be noted that the first embedded code and the second embedded code The same decoder needs to be input for fusion decoding, and the feature dimensions need to be consistent. Figure 3 Both of them are N×D.

[0092] Furthermore, in one embodiment, the total loss value of all nodes is calculated as follows:

[0093]

[0094] node The loss value is calculated as follows:

[0095]

[0096] in, and Represent the adjacency matrix of the network structure before and after reconstruction, and The attribute matrices representing the node attributes before and after reconstruction, Represents the weight of the difference in network structure before and after reconstruction, represents the Hadamard product, represents the Frobenius norm, and The penalty coefficient matrices representing the differences in network structure and node attributes before and after reconstruction are:

[0097]

[0098]

[0099] in, Representation node With node The connection relationship, 0 means no connection, 1 means connection, Representation node The kth attribute value of , 0 indicates that the attribute is missing, a>1, b>1.

[0100] In this embodiment, Used to balance the importance between network structure differences and node attribute differences. and It is used to reduce the reconstruction error of non-zero elements and compensate for the existence of missing edges or attributes in the network structure and node attributes before reconstruction. By setting a>1 and b>1, an additional penalty is imposed on the reconstruction error of non-zero elements.

[0101] Furthermore, in one embodiment, before the step of marking the service trace graph with a total loss value greater than the loss threshold as an abnormal service trace graph, the method further includes:

[0102] The loss threshold of the current service tracing graph is determined according to the total loss values ​​of a plurality of related service tracing graphs, wherein the related service tracing graphs are earlier service tracing graphs in the detection sequence where the current service tracing graph is located.

[0103] In this embodiment, for the service tracking graph of the same detection sequence, the loss threshold of the latest loss value is calculated based on the historical total loss value. Since the detection sequence is constantly updated, the historical total loss value will also change, thereby obtaining a dynamic loss threshold, which helps to improve the accuracy of fault detection.

[0104] As an optional implementation manner, before the step of determining the loss threshold of the current service trace graph according to the total loss value of multiple related service trace graphs, the method further includes:

[0105] If the current service tracking graph is the earliest preset number of service tracking graphs in the detection sequence, the preset value is determined as the loss threshold of the current service tracking graph; otherwise, the preset number of service tracking graphs before the current service tracking graph are determined as the relevant service tracking graphs.

[0106] In this embodiment, within a period of time after the detection begins, the number of service tracking graphs in the detection sequence is insufficient, and a preset value is used as a temporary loss threshold. After the number of service tracking graphs in the detection sequence is greater than the preset number, a sliding window is set based on the preset number to ensure that the number of relevant service tracking graphs remains stable.

[0107] As another optional implementation, before the step of determining the loss threshold of the current service trace graph according to the total loss values ​​of the multiple related service trace graphs, the method further includes:

[0108] If the current service tracking graph is a service tracking graph within the preset time length after the detection sequence is established, the preset value is determined as the loss threshold of the current service tracking graph; otherwise, multiple service tracking graphs within the preset time length before the current service tracking graph are determined as related service tracking graphs.

[0109] In this embodiment, within a period of time after the detection is started, the establishment time of the detection sequence is insufficient, and a preset value is used as a temporary loss threshold. After the establishment time of the detection sequence is greater than the preset time, a sliding window is set based on the preset time to ensure that the maximum time difference between the relevant service tracking graph and the current service tracking graph remains stable.

[0110] Furthermore, in one embodiment, in the step of determining the loss threshold of the current service tracking graph based on the total loss values ​​of multiple related service tracking graphs, the loss threshold of the current service tracking graph is equal to the sum of the average value and three times the standard deviation of the total loss values ​​of the multiple related service tracking graphs.

[0111] In this embodiment, the loss threshold is determined based on the 3 sigma principle of normal distribution, thereby improving the reliability of the loss threshold.

[0112] The experimental process of this application is as follows:

[0113] 1. Experimental setup

[0114] This experiment was conducted on Windows 10, using Python 3.6 and PyTorch version 1.9.0. The Adam optimizer was used, with a learning rate of 0.001, a batch size of 32, and 100 epochs. Early stopping was used if the validation loss did not improve after 10 consecutive epochs. This experiment was conducted on a high-performance computer equipped with an NVIDIA GeForce RTX4050 GPU.

[0115] 2. Comparison method

[0116] In order to evaluate the performance of this method in fault anomaly detection in microservice systems, this method is compared with five advanced fault anomaly detection methods, namely: FSA, MultimodalTrace, TraceAnomaly, DeepTraLog and TraceCRL.

[0117] In order to evaluate the accuracy of this method in locating the root cause of faults in microservice systems, this method is compared with four advanced root cause localization methods: MEPFL, TraceAnomaly, MicroRank, and TraceRCA.

[0118] 3. Evaluation indicators

[0119] To evaluate the performance of fault anomaly detection, this experiment uses three common anomaly detection evaluation metrics: precision, recall, and F1 score. Trajectory anomaly detection is treated as a binary classification task for each individual trajectory, with the F1 score serving as the key metric for evaluating performance. The F1 score is derived from the counts of true positive samples (TP), false positive samples (FP), and false negative samples (FN) generated by the model responsible for identifying faults.

[0120] Accuracy: The proportion of true positive predictions among all positive predictions made by the model, which can be calculated by the following formula: .

[0121] Recall: The proportion of all actual positive examples that are predicted to be positive. It can be calculated by the following formula: .

[0122] F1 score: The harmonic mean of precision and recall, providing a balance between the two, can be calculated using the following formula: .

[0123] In order to evaluate the efficiency of this method, this experiment compares the time consumed by different methods for training.

[0124] In order to evaluate the accuracy of fault root cause location, this experiment uses the general evaluation indicator accuracy A@k.

[0125] Top-k Accuracy (A@k): This metric measures the likelihood that the actual root cause is among the top-k identified candidates. In this experiment, k values ​​of 1, 2, and 3 are chosen to cover a range of scenarios.

[0126] 4. Results Analysis

[0127] This experiment compared the performance of our method in fault anomaly detection with five existing methods on the Train-Ticket dataset. As shown in Table 1, the experimental results demonstrate that our method achieves superior performance in microservice system fault detection, as evidenced by its high precision (0.971), recall (0.935), and F1 score (0.953). Compared to the best baseline method, our method achieved a score exceeding 17.9%, further demonstrating its effectiveness in microservice system fault detection.

[0128] In addition, this experiment also evaluated the training efficiency of each method. MulitmodaleTrace and TraceCRL showed relatively short training times because they effectively processed sequence data. TraceAnomaly converted trajectories into structured trace vectors for detecting response time and call path anomalies, resulting in slightly longer training times. DeepTraLog processed a large amount of simulated log information and showed the longest training time. This experimental method introduced a service trace graph to represent service traces. This method significantly reduced the time required, making it the method with the shortest training time among all methods.

[0129] Table 1

[0130]

[0131] This experiment compared our method with four existing methods on the Train-Ticket dataset to determine its accuracy in root cause localization. As shown in Table 2, the experimental results demonstrate that our method outperforms the comparison methods in root cause localization accuracy. In particular, our method achieves excellent performance on the A@1 metric, demonstrating its robust ability to locate root causes in microservice systems.

[0132] Table 2

[0133]

[0134] Experimental results show that this method can improve the accuracy and efficiency of fault diagnosis in microservice systems.

[0135] In a second aspect, an embodiment of the present application also provides a microservice system fault diagnosis device.

[0136] Figure 4 A schematic diagram of the functional modules of a microservice system fault diagnosis device in one embodiment of the present application is shown.

[0137] Reference Figure 4 In one embodiment, the microservice system fault diagnosis device includes:

[0138] A graph construction module 10 is used to construct a service tracking graph, wherein each user request corresponds to a service tracking graph, and the service tracking graphs corresponding to multiple user requests for the same operation constitute a detection sequence. In the service tracking graph, each node represents a service, and each edge represents the call relationship from the source service to the target service. The node attributes include the performance data of the service call;

[0139] A structure reconstruction module 20 is configured to determine a first embedding code of a node based on the network structure and node attributes of the service tracking graph based on the graph attention network, and determine a reconstructed network structure based on the first embedding code of the node;

[0140] An attribute reconstruction module 30 is configured to determine a second embedded code of a node based on the node attributes of the service trace graph based on a long short-term memory network, and to determine a reconstructed node attribute based on the first embedded code and the second embedded code of the node, wherein the service trace graphs of the same detection sequence are sequentially input into the corresponding long short-term memory network in the order of user requests, from earliest to latest;

[0141] Anomaly detection module 40 determines the total loss value of all nodes based on the network structure and node attributes before and after the reconstruction of the service tracking graph, and marks the service tracking graph with a total loss value greater than a loss threshold as an abnormal service tracking graph;

[0142] The root cause location module 50 is used to determine the loss value of each node based on the network structure and node attributes before and after the reconstruction of the abnormal service tracking graph, and mark the node with the largest loss value as the abnormal root cause node.

[0143] Furthermore, in one embodiment, the graph construction module 10 is configured to:

[0144] Collect trace data and time series performance metrics data generated during the service call process of the microservice system. Each user request has a unique trace ID, and each service call has a unique span ID.

[0145] For each trace ID, trace data and timing performance metrics data are aggregated, nodes and edges are constructed based on the trace data, and node attributes are constructed based on the timing performance metrics data. Each span ID corresponds to a node.

[0146] Furthermore, in one embodiment, the structure reconstruction module 20 is configured to:

[0147] Convert node attributes of the service tracking graph into low-dimensional latent representation;

[0148] Determine the attention weight of each node to its neighboring nodes;

[0149] Each node is regarded as the central node. According to the attention weight of the central node to its adjacent nodes, the low-dimensional potential representations of the adjacent nodes are normalized and weighted to obtain the first embedded code of the central node.

[0150] Furthermore, in one embodiment, the total loss value of all nodes is calculated as follows:

[0151]

[0152] node The loss value is calculated as follows:

[0153]

[0154] in, and Represent the adjacency matrix of the network structure before and after reconstruction, and The attribute matrices representing the node attributes before and after reconstruction, Represents the weight of the difference in network structure before and after reconstruction, represents the Hadamard product, represents the Frobenius norm, and The penalty coefficient matrices representing the differences in network structure and node attributes before and after reconstruction are:

[0155]

[0156]

[0157] in, Representation node With node The connection relationship, 0 means no connection, 1 means connection, Representation node The kth attribute value of , 0 indicates that the attribute is missing, a>1, b>1.

[0158] Furthermore, in one embodiment, the anomaly detection module 40 is further configured to:

[0159] The loss threshold of the current service tracing graph is determined according to the total loss values ​​of a plurality of related service tracing graphs, wherein the related service tracing graphs are earlier service tracing graphs in the detection sequence where the current service tracing graph is located.

[0160] Furthermore, in one embodiment, the anomaly detection module 40 is further configured to:

[0161] If the current service tracking graph is the earliest preset number of service tracking graphs in the detection sequence, the preset value is determined as the loss threshold of the current service tracking graph; otherwise, the preset number of service tracking graphs before the current service tracking graph are determined as the relevant service tracking graphs.

[0162] Furthermore, in one embodiment, the anomaly detection module 40 is further configured to:

[0163] If the current service tracking graph is a service tracking graph within the preset time length after the detection sequence is established, the preset value is determined as the loss threshold of the current service tracking graph; otherwise, multiple service tracking graphs within the preset time length before the current service tracking graph are determined as related service tracking graphs.

[0164] Furthermore, in one embodiment, in the step of determining the loss threshold of the current service tracking graph based on the total loss values ​​of multiple related service tracking graphs, the loss threshold of the current service tracking graph is equal to the sum of the average value and three times the standard deviation of the total loss values ​​of the multiple related service tracking graphs.

[0165] Among them, the functional implementation of each module in the above-mentioned microservice system fault diagnosis device corresponds to the various steps in the above-mentioned microservice system fault diagnosis method embodiment, and its functions and implementation processes are no longer repeated here.

[0166] In a third aspect, an embodiment of the present application provides a microservice system fault diagnosis device, which may be a personal computer (PC), a laptop, a server, or other device with data processing capabilities.

[0167] Figure 5 A schematic diagram of the hardware structure of the microservice system fault diagnosis device involved in the embodiment of the present application is shown.

[0168] Reference Figure 5 In an embodiment of the present application, a microservice system fault diagnosis device may include a processor, a memory, a communication interface, and a communication bus.

[0169] The communication bus may be of any type and is used to interconnect the processor, memory, and communication interface.

[0170] Communication interfaces include input / output (I / O) interfaces, physical interfaces, and logical interfaces. These interfaces interconnect components within the microservice system fault diagnosis device, as well as interfaces that connect the microservice system fault diagnosis device to other devices (such as other computing devices or user devices). Physical interfaces can include Ethernet, fiber, and ATM interfaces; user devices can include displays and keyboards.

[0171] The memory can be various types of storage media, such as random access memory (RAM), read-only memory (ROM), non-volatile RAM (NVRAM), flash memory, optical storage, hard disk, programmable ROM (PROM), erasable PROM (EPROM), electrically erasable PROM (EEPROM), etc.

[0172] The processor can be a general-purpose processor that can call a microservice system fault diagnosis program stored in a memory and execute the microservice system fault diagnosis method provided in the embodiments of the present application. For example, the general-purpose processor can be a central processing unit (CPU). The method executed when the microservice system fault diagnosis program is called can be referred to in the various embodiments of the microservice system fault diagnosis method of the present application and will not be further described here.

[0173] Those skilled in the art will understand that Figure 5 The hardware structure shown in the figure does not constitute a limitation to the present application and may include more or fewer components than shown in the figure, or a combination of certain components, or a different arrangement of components.

[0174] Among them, the method implemented when the microservice system fault diagnosis program is executed can refer to the various embodiments of the microservice system fault diagnosis method of this application, and will not be repeated here.

[0175] It should be noted that the serial numbers of the above-mentioned embodiments of the present application are for description only and do not represent the advantages or disadvantages of the embodiments.

[0176] The terms "including" and "having" and any variations thereof in the specification and claims of this application and the above-mentioned drawings are intended to cover non-exclusive inclusions. For example, a process, method, system, product or device that includes a series of steps or units is not limited to the listed steps or units, but optionally includes steps or units that are not listed, or optionally includes other steps or units inherent to these processes, methods, products or devices. The terms "first", "second" and "third" are used to distinguish different objects, etc., and do not represent a sequence, nor do they limit the "first", "second" and "third" to different types.

[0177] In the description of the embodiments of this application, the words "exemplary," "for example," or "for example" are used to indicate examples, illustrations, or descriptions. Any embodiment or design described as "exemplary," "for example," or "for example" in the embodiments of this application should not be construed as being preferred or advantageous over other embodiments or designs. Rather, the use of words such as "exemplary," "for example," or "for example" is intended to present the relevant concepts in a concrete manner.

[0178] In the description of the embodiments of the present application, unless otherwise specified, “ / ” means or, for example, A / B can mean A or B; “and / or” in the text is merely a description of the association relationship of associated objects, indicating that three relationships may exist, for example, A and / or B can mean: A exists alone, A and B exist at the same time, and B exists alone. In addition, in the description of the embodiments of the present application, “multiple” refers to two or more than two.

[0179] In some processes described in the embodiments of the present application, multiple operations or steps are included that appear in a specific order. However, it should be understood that these operations or steps may not be performed in the order in which they appear in the embodiments of the present application or may be performed in parallel. The sequence numbers of the operations are only used to distinguish between different operations, and the sequence numbers themselves do not represent any order of execution. In addition, these processes may include more or fewer operations, and these operations or steps may be performed in sequence or in parallel, and these operations or steps may be combined.

[0180] Through the description of the above embodiments, those skilled in the art can clearly understand that the above-mentioned embodiment methods can be implemented by means of software plus the necessary general hardware platform. Of course, they can also be implemented by hardware, but in many cases the former is a better implementation method. Based on this understanding, the technical solution of this application, or the part that contributes to the existing technology, can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) as described above and includes a number of instructions for enabling a terminal device to execute the methods described in each embodiment of this application.

[0181] The above are only preferred embodiments of the present application and do not limit the patent scope of the present application. Any equivalent structure or equivalent process transformation made using the contents of the present application specification and drawings, or directly or indirectly applied in other related technical fields, are also included in the patent protection scope of the present application.

Claims

1. A microservice system fault diagnosis method, characterized in that: The microservice system fault diagnosis method includes: Construct a service tracking graph, where each user request corresponds to a service tracking graph. The service tracking graphs corresponding to multiple user requests for the same operation form a detection sequence. In the service tracking graph, each node represents a service, and each edge represents the call relationship between the source service and the target service. Node attributes include performance data of the service call. Based on the graph attention network, the first embedding code of the node is determined according to the network structure and node attributes of the service tracking graph, and the reconstructed network structure is determined according to the first embedding code of the node; Based on the long short-term memory network, a second embedding code of the node is determined according to the node attributes of the service trace graph, and a reconstructed node attribute is determined according to the first embedding code and the second embedding code of the node. The service trace graphs of the same detection sequence are sequentially input into the corresponding long short-term memory network according to the order of user requests from earliest to latest. Determine the total loss value of all nodes based on the network structure and node attributes before and after the reconstruction of the service trace graph, and mark the service trace graph with a total loss value greater than the loss threshold as an abnormal service trace graph; The loss value of each node is determined based on the network structure and node attributes before and after reconstruction of the abnormal service tracking graph, and the node with the largest loss value is marked as the abnormal root cause node.

2. The microservice system fault diagnosis method according to claim 1, characterized in that: The steps of constructing the service tracking graph include: Collect trace data and time series performance metrics data generated during the service call process of the microservice system. Each user request has a unique trace ID, and each service call has a unique span ID. For each trace ID, trace data and timing performance metrics data are aggregated, nodes and edges are constructed based on the trace data, and node attributes are constructed based on the timing performance metrics data. Each span ID corresponds to a node.

3. The microservice system fault diagnosis method according to claim 1, characterized in that: The step of determining the first embedding code of the node based on the network structure and node attributes of the service tracking graph based on the graph attention network includes: Convert node attributes of the service tracking graph into low-dimensional latent representation; Determine the attention weight of each node to its neighboring nodes; Each node is regarded as the central node. According to the attention weight of the central node to its adjacent nodes, the low-dimensional potential representations of the adjacent nodes are normalized and weighted to obtain the first embedded code of the central node.

4. The microservice system fault diagnosis method according to claim 1, wherein: The total loss value of all nodes is calculated as: node The loss value is calculated as follows: in, and Represent the adjacency matrix of the network structure before and after reconstruction, and The attribute matrices representing the node attributes before and after reconstruction, Represents the weight of the difference in network structure before and after reconstruction, represents the Hadamard product, represents the Frobenius norm, and The penalty coefficient matrices representing the differences in network structure and node attributes before and after reconstruction are: in, Representation node With node The connection relationship, 0 means no connection, 1 means connection, Representation node The kth attribute value of , 0 indicates that the attribute is missing, a>1, b>1.

5. The microservice system fault diagnosis method according to claim 1, wherein: Before the step of marking a service tracing graph whose total loss value is greater than a loss threshold as an abnormal service tracing graph, the method further includes: The loss threshold of the current service tracing graph is determined according to the total loss values ​​of a plurality of related service tracing graphs, wherein the related service tracing graphs are earlier service tracing graphs in the detection sequence where the current service tracing graph is located.

6. The microservice system fault diagnosis method according to claim 5, characterized in that: Before the step of determining the loss threshold of the current service tracking graph according to the total loss values ​​of the multiple related service tracking graphs, the method further includes: If the current service tracking graph is the earliest preset number of service tracking graphs in the detection sequence, the preset value is determined as the loss threshold of the current service tracking graph; otherwise, the preset number of service tracking graphs before the current service tracking graph are determined as the relevant service tracking graphs.

7. The microservice system fault diagnosis method according to claim 5, characterized in that: Before the step of determining the loss threshold of the current service tracking graph according to the total loss values ​​of the multiple related service tracking graphs, the method further includes: If the current service tracking graph is a service tracking graph within the preset time length after the detection sequence is established, the preset value is determined as the loss threshold of the current service tracking graph; otherwise, multiple service tracking graphs within the preset time length before the current service tracking graph are determined as related service tracking graphs.

8. The microservice system fault diagnosis method according to claim 5, wherein: In the step of determining the loss threshold of the current service tracing graph according to the total loss values ​​of the multiple related service tracing graphs, the loss threshold of the current service tracing graph is equal to the sum of the average value and three standard deviations of the total loss values ​​of the multiple related service tracing graphs.

9. A microservice system fault diagnosis device, characterized in that: The microservice system fault diagnosis device includes: A graph construction module is used to construct a service tracking graph. Each user request corresponds to a service tracking graph. The service tracking graphs corresponding to multiple user requests for the same operation constitute a detection sequence. In the service tracking graph, each node represents a service, and each edge represents the call relationship from the source service to the target service. Node attributes include performance data of the service call. A structure reconstruction module is used to determine a first embedding code of a node based on the network structure and node attributes of the service tracking graph based on the graph attention network, and to determine a reconstructed network structure based on the first embedding code of the node; An attribute reconstruction module is configured to determine a second embedded code of a node based on the node attributes of a service tracking graph based on a long short-term memory network, and to determine a reconstructed node attribute based on the first embedded code and the second embedded code of the node, wherein the service tracking graphs of the same detection sequence are sequentially input into the corresponding long short-term memory network in the order of user requests, from earliest to latest; The anomaly detection module determines the total loss value of all nodes based on the network structure and node attributes of the service trace graph before and after reconstruction, and marks the service trace graph with a total loss value greater than the loss threshold as an abnormal service trace graph; The root cause location module is used to determine the loss value of each node based on the network structure and node attributes before and after the reconstruction of the abnormal service tracking graph, and mark the node with the largest loss value as the abnormal root cause node.

10. A microservice system fault diagnosis device, characterized in that: The microservice system fault diagnosis device includes a processor, a memory, and a microservice system fault diagnosis program stored in the memory and executable by the processor, wherein when the microservice system fault diagnosis program is executed by the processor, the steps of the microservice system fault diagnosis method according to any one of claims 1 to 8 are implemented.

Images