A fault diagnosis method based on graph convolutional adversarial neural network and tracking data
By using a fault diagnosis method based on graph convolutional adversarial neural networks, the problems of low information density and poor scalability of tracking data in distributed software systems are solved, and a more efficient fault diagnosis effect is achieved.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- BEIJING YUNJI ZHIZAO TECH CO LTD
- Filing Date
- 2022-11-07
- Publication Date
- 2026-06-02
AI Technical Summary
Existing technologies for fault diagnosis in distributed software systems based on tracking data suffer from problems such as low information density, complex data associations, high labor costs, poor scalability, and poor algorithm performance. In particular, methods based on LSTM and Tree-CNN perform poorly on tracking datasets.
A fault diagnosis method based on graph convolutional adversarial neural networks (GCN and GAN) is adopted. By preprocessing and structuring the tracking data into a tree, structural features are extracted using GCN, and a discriminator is trained by generating fake data through GAN to identify abnormal tracking data.
It improves the accuracy and robustness of fault diagnosis in distributed systems, and can more effectively extract features from tracking data, thereby enhancing the precision and efficiency of fault diagnosis.
Smart Images
Figure CN115756917B_ABST
Abstract
Description
Technical Field
[0001] This invention relates to the field of fault diagnosis methods, and in particular to a fault diagnosis method based on graph convolutional adversarial neural networks and tracking data. Background Technology
[0002] Distributed software systems are large-scale and complex, making them highly susceptible to system failures, which are often difficult to detect and diagnose. System tracing technology is a key technique for analyzing and understanding the request execution paths and program logic of distributed systems, and is one of the key data sources for system failure diagnosis. However, due to the massive scale, low information density, and complex relationships between tracing data, failure diagnosis based on tracing data is extremely difficult. Existing failure diagnosis methods based on tracing data often employ manually defined anomaly rules, diagnosing fault requests and their locations by online detection of whether the tracing links meet these rules. This method requires significant manpower to build a rule base, and the rule definition method is poorly adaptable to complex tracing data, difficult to extend, and results in poor accuracy in failure diagnosis.
[0003] Existing tracking data fault diagnosis solutions can be mainly divided into two categories: solutions based on manually defined anomaly rules and solutions based on deep learning. The former filters out statistically extreme values based on statistically significant characteristics of the tracking data, such as the type of function called, the duration of the function call, and the server on which it resides, along with other manually defined rules, to identify anomalies. The latter, on the other hand, inputs the tracking data into a neural network and uses deep learning techniques to extract relevant temporal features and filter anomalies. While both existing tracking data fault diagnosis solutions exist, they each have their own drawbacks.
[0004] While the first type of solution is simple to implement and easy to understand, it often fails to effectively extract deep information from the tracking data and cannot handle the complex tracking data of large systems. Furthermore, the manual cost is too high, requiring significant manpower to construct the rule base, which is then unusable on other systems, exhibiting poor scalability. Nevertheless, we generally use this type of solution as a baseline to examine the performance of other proposed solutions.
[0005] For the second type of approach, existing algorithms are mainly based on LSTM (Long Short-Term Memory Network) and Tree-CNN (Tree Convolutional Neural Network). However, LSTM is primarily used for temporal features such as natural language processing, while Tree-CNN is designed for Abstract Syntax Trees (ASTs). Neither type of network performs well on tracking datasets. Although Tree-CNN also extracts tree-structure features, it requires a relatively balanced input tree structure (approximately a complete binary tree structure), while the tree structure of the tracking dataset is highly imbalanced, resulting in unsatisfactory algorithm performance.
[0006] Based on the above, a fault diagnosis method based on graph convolutional adversarial neural networks and tracking data has become a technical problem that the whole society urgently needs to solve. Summary of the Invention
[0007] In view of this, the present invention proposes a fault diagnosis method based on graph convolutional adversarial neural networks and tracking data, comprising the following fault diagnosis steps:
[0008] (1) First, the tracking dataset is read and preprocessed to remove useless features and convert useful features into standard data types;
[0009] (2) Read and restore the tree structure of the tracking data, use a multi-branch tree as the data structure to store the tracking data, and calculate the node features and adjacency matrix of the multi-branch tree corresponding to each tracking data;
[0010] (3) Then input it into GCN and use GCN to fully extract the structural features of the tracking data;
[0011] (4) Use GAN to generate fake tracking data with the help of real tracking data, and iteratively optimize the network parameters of the generator and discriminator;
[0012] (5) Use the discriminator of GAN to judge the original tracking dataset. If a tracking data is judged as "false data", it is considered that the tracking data has some degree of abnormality. The tracking data judged as abnormal will be collected to meet the needs of fault diagnosis.
[0013] Furthermore, the tracking dataset includes the following characteristics:
[0014] (1) Span numbering information: the number of the Trace where the Span is located, the number of the Span itself, and the number of its parent process. If this Span is the root process, then the number of its parent process is NaN.
[0015] (2) Content characteristics of Span: the name of the function called by Span, the request type of the function called by Span, and the total time of Span from start to end;
[0016] (3) The time and location of the Span event, the timestamp of the Span call, and the server number where the Span call was performed.
[0017] Furthermore, among the features mentioned above, by using the first type of "Span numbering information", we can reconstruct the tree structure of the Trace from the original dataset; by using the second type of "Span content features", we can evaluate the anomalies of the Span and the anomalies of the Trace to which it belongs; and by using the third type of "time and location of Span occurrence", we can locate the anomaly and determine the time and location of the anomaly.
[0018] Furthermore, the training process of the GAN network is as follows:
[0019] (1) Initialize the network parameters of the discriminator D and the generator G;
[0020] (2) Sample m samples from the real dataset and m noise samples from the prior distribution noise, and obtain m generated samples through the generator. Fix the generator G, train the discriminator D to distinguish between real samples and generated samples as accurately as possible, and update the network parameters of the discriminator D.
[0021] (3) Fix the discriminator D and train only the generator G to generate more realistic samples so that the discriminator D will make a mistake.
[0022] (4) Repeat steps 2 and 3 to alternately train the discriminator D and the generator G to improve their performance.
[0023] Furthermore, the discriminator consists of a convolutional layer (referring to graph convolution), a downsampling layer, and a fully connected layer, while the generator consists of a deconvolutional layer, an upsampling layer, and a fully connected layer. The purpose is to enable the discriminator to generate realistic "generated trace data" after training, given "real trace data" as input.
[0024] The above technical solution has the following beneficial effects:
[0025] This technical solution can fully utilize the information from tracking data to enhance the accuracy and robustness of fault diagnosis in distributed systems. By utilizing graph convolutional adversarial neural network structures, this technical solution has a more powerful ability to extract features from tracking data, and therefore will achieve better results in tasks including but not limited to fault diagnosis. Attached Figure Description
[0026] Figure 1 is a flowchart of the training process of the present invention;
[0027] Figure 2 is a flowchart of the process of using this invention;
[0028] Figure 3 is a structural diagram of GCN;
[0029] Figure 4 is a structural diagram of a GAN network. Detailed Implementation
[0030] The present invention will now be further described.
[0031] A fault diagnosis method for tracking datasets is proposed. This method is designed specifically for the unique data structure of tracking datasets, fully utilizing the distributed system state information contained within them, thereby improving the accuracy of system fault diagnosis algorithms.
[0032] The technical solution of this invention is as follows:
[0033] First, the tracking dataset is read and preprocessed, removing useless features and transforming useful features into standard data types. Second, the tree structure of the tracking data is read and reconstructed, using a multi-branch tree as the data structure to store the tracking data. The node features and adjacency matrix of the multi-branch tree corresponding to each tracking data point are calculated and then input into a GCN (Graph Convolutional Network) to fully extract the structural features of the tracking data. Next, a GAN (Generative Adversarial Network) is used to generate fake tracking data using real tracking data, and the network parameters of the generator and discriminator are iteratively optimized. Finally, the GAN's discriminator judges the original tracking dataset; if a tracking data point is judged as "fake data," it is considered to have some degree of anomaly. Tracking data judged as abnormal is collected to meet the needs of fault diagnosis.
[0034] 1. Tracking Data Preprocessing
[0035] A software system's trace dataset typically occupies a large amount of space, comparable to text log data. Generally, trace data is generated at a rate of 10,000 to 1,000,000 records per day, occupying 100MB to 10GB of space per day. Each trace data record is also called a Trace (in the following text, "Trace" refers to each specific sample, while "trace data" refers to the entire dataset), and each function call is called a Span. A Trace consists of approximately 4 to 40 Spans, and the Trace records information such as the call relationships between Spans and the duration of the calls.
[0036] In the Trace shown in the diagram below, we can see that Span A first calls Span B, and then Span B calls Span D; after Span D and Span B finish executing in sequence, Span A calls Span C. Finally, the calling process between different Spans forms a tree structure, which also constitutes a Trace.
[0037] However, the raw trace data does not provide a tree structure of traces, but rather information such as the trace to which each span belongs and its parent process. A typical trace dataset contains the following characteristics:
[0038] (1) Span numbering information: the number of the Trace where the Span is located, the number of the Span itself, and the number of its parent process. If this Span is the root process, then the number of its parent process is NaN.
[0039] (2) The content characteristics of the Span, the name of the function called by the Span, the request type of the function called by the Span, and the total time of the Span from start to end;
[0040] (3) The time and location of the Span event, the timestamp of the Span call, and the server number where the Span call was performed.
[0041] Among these features, using the first type, "Span numbering information," we can reconstruct the tree structure of the trace from the original dataset; using the second type, "Span content features," we can evaluate anomalies in the span (and the anomalies in the trace to which it belongs); and using the third type, "Span occurrence time and location," we can locate the anomaly and determine its occurrence time and location. Therefore, we first traverse the "Span numbering information" portion of the dataset, storing each trace in a multi-branch tree data structure; each node of the tree corresponds to a span, and all the features of the span are stored in the node to avoid any information loss during processing.
[0042] 2. GCN module
[0043] GCN (Graph Convolutional Network) is a neural network structure designed to extract features from graph structures. Its main idea is to use the feature information of each node in a graph to extract the features of its neighboring nodes and use them as the features of the next layer to complete a "convolution".
[0044] Expressed as a formula, the "convolution" operation of GCN is:
[0045] ;
[0046] Here, H represents the features of each node, A is the adjacency matrix, D is the balancing factor matrix that normalizes A, and W represents the network parameters to be trained. The authors of GCN point out that even without training W, graph convolution can still extract graph information, unlike classic CNNs. After two such "convolution" operations combined with an activation function, the output is the high-level features of each node extracted by GCN.
[0047] After preprocessing the tracking dataset and structuring it into a tree, we can input the tree corresponding to the tracking data (which we call the Trace Tree) into the GCN. We treat each span in the Trace as a node in the graph, then treat the features of the span as the features of the node to construct a matrix H, and treat the parent-child relationships between spans as edges to construct an adjacency matrix A. We further calculate the corresponding balance factor matrix D using A. In this way, we obtain all the input variables required by the GCN. In the output part, since the root node of the tree (that is, the root process of the Trace) best represents the state of the entire Trace, and the root node has absorbed more information from other nodes in the GCN, directly using the high-level features of the root node as the output features of the entire tree is a good choice; of course, we can also directly use the high-level features of all nodes concatenated together as the network output.
[0048] 3. GAN Module and Dataset Partitioning
[0049] Generative Adversarial Networks (GANs) are network structures invented to generate realistic data that closely resembles real data. The main idea is to train a discriminator and a generator alternately, allowing both to achieve high algorithmic performance. GANs have already found mature applications in deep learning, demonstrating good performance in tasks such as style transfer, virtual image generation, and text-to-image conversion.
[0050] The image below shows the structure of a simple GAN network. Its discriminator and generator consist of convolutional layers, downsampling layers, fully connected layers, deconvolutional layers, upsampling layers, and fully connected layers, respectively. The goal is to enable the discriminator to generate realistic handwritten digits 0-9 after training, given a "handwritten digits 0-9 dataset" as input.
[0051] The training process for this network is as follows:
[0052] 1. Initialize the network parameters of the discriminator (D) and generator (G);
[0053] 2. Sample m samples from the real dataset and m noise samples from the prior distribution noise, and obtain m generated samples through a generator. Fix the generator G, train the discriminator D to distinguish between real samples and generated samples as accurately as possible, and update the network parameters of the discriminator D;
[0054] 3. Fix the discriminator D, and only train the generator G to generate more realistic samples so that the discriminator D will make a wrong judgment.
[0055] 4. Repeat steps 2 and 3, alternating between training the discriminator D and the generator G to improve their performance.
[0056] Once we obtain the trace trees, we can use these samples to train a high-accuracy discriminator D using a GAN. We design the discriminator D in the GAN directly with a GCN structure, which allows it to extract the features of the trace tree to the greatest extent. The generator G in the GAN is designed as a network with the structure inverted from D to generate fake trace trees. The generated features need to include the tree structure of the trace, the content features of the spans, and information such as the occurrence time and location of the spans.
[0057] To obtain a dataset free of impurities, we need to use trace data that has not experienced any failures as "real samples" input into the GAN. If the distributed system provides the time of server failure, we need to discard all traces within five minutes before and after the failure based on the timestamp information of the traces, as they may all have a causal relationship with the server failure; the remaining traces can be regarded as data excluding anomalies and input into the GAN as "real samples".
[0058] Once the GAN's discriminator D is trained, we can directly treat it as an "anomaly detector" for the tracking data, as it can effectively distinguish between real samples without anomalies and generated samples (fake samples). When it identifies certain traces in the original dataset as generated samples, it indicates that the distribution of these traces differs from that of real samples, and there is a high probability that some anomalous traces exist. In this way, we detect and locate some potential server faults from the tracking dataset. As a further explanation of this invention, a flowchart of the training process is provided.
[0059] As shown in the figure, after processing each trace in the tracking dataset into a tree structure, we first exclude all traces that may contain anomalies based on the failure time given in the dataset, and input the remaining samples as real samples into the discriminator D (structured as GCN) in the GAN. Then, we randomly generate noise samples, input them into the generator G (structured as inverse GCN) in the GAN, and input the results as fake samples into the discriminator D. This process constructs the GAN dataset and trains the discriminator's discrimination ability.
[0060] Using the flowchart shown in the figure, after we finish the training process and save the discriminator model, we use the discriminator to classify the entire tracking dataset, and regard the data classified as "false samples" as "traces that may contain abnormalities". Then, we perform fault diagnosis on the system based on the server and timestamp of this part of the trace.
[0061] The basic principles and main features of the present invention have been described above. Those skilled in the art should understand that the present invention is not limited to the above embodiments. The embodiments and descriptions in the specification are only illustrative of the principles of the present invention. Various changes and modifications can be made to the present invention without departing from the spirit and scope of the present invention. All such changes and modifications fall within the scope of the present invention as claimed. The scope of protection of the invention is defined by the appended claims and their equivalents.
Claims
1. A fault diagnosis method based on graph convolutional adversarial neural networks and tracking data, characterized in that, The following fault diagnosis steps are included. (1) First, the tracking dataset is read and preprocessed to remove useless features and convert useful features into standard data types; (2) Read and restore the tree structure of the tracking data, use a multi-branch tree as the data structure to store the tracking data, and calculate the node features and adjacency matrix of the multi-branch tree corresponding to each tracking data; (3) Then input it into GCN and use GCN to fully extract the structural features of the tracking data; (4) Use GAN to generate fake tracking data with the help of real tracking data, and iteratively optimize the network parameters of generator G and discriminator D; (5) Use the discriminator of GAN to judge the original tracking dataset. If a tracking data is judged as "false data", it is considered that the tracking data has some degree of abnormality. The tracking data judged as abnormal will be collected to meet the needs of fault diagnosis.
2. The fault diagnosis method based on graph convolutional adversarial neural networks and tracking data according to claim 1, characterized in that, The tracking dataset includes the following characteristics: (1) Span number information: the number of the Trace to which the Span is located, the number of the Span itself, and the number of its parent process. If this Span is the root process, then the number of its parent process is NaN. (2) The content characteristics of the Span, the name of the function called by the Span, the request type of the function called by the Span, and the total time of the Span from start to end; (3) The time and location of the Spam event, the timestamp of the Spam call, and the server number where the Spam call took place.
3. The fault diagnosis method based on graph convolutional adversarial neural networks and tracking data according to claim 2, characterized in that, Among the features included in the aforementioned tracking dataset, the first type of "Span numbering information" is used to reconstruct the tree structure of the Trace from the original dataset; the second type of "Span content features" is used to evaluate the anomalies of the Span and the Trace to which it belongs; and the third type of "time and location of Span occurrence" is used to locate the anomaly and determine the time and location of the anomaly.
4. The fault diagnosis method based on graph convolutional adversarial neural networks and tracking data according to claim 1, characterized in that, The training process of the GAN is as follows: (1) Initialize the network parameters of the discriminator D and the generator G; (2) Sample m samples from the real dataset and m noise samples from the prior distribution noise, and obtain m generated samples through the generator. Fix the generator G, train the discriminator D to distinguish between real samples and generated samples as accurately as possible, and update the network parameters of the discriminator D. (3) Fix the discriminator D and only train the generator G to generate more realistic generated samples so that the discriminator D will make a mistake. (4) Repeat (2) and (3) to alternately train the discriminator D and the generator G to improve their performance.
5. The fault diagnosis method based on graph convolutional adversarial neural networks and tracking data according to claim 4, characterized in that, The discriminator D consists of a convolutional layer, a downsampling layer, and a fully connected layer, while the generator G consists of a deconvolutional layer, an upsampling layer, and a fully connected layer. The purpose is to enable the discriminator D to autonomously generate realistic "Trace data" after training, given "real Trace data" as input.