Dynamic graph anomaly detection method based on time-structure attention and variational graph auto-encoder

Abstract

The invention provides a dynamic graph anomaly detection method based on time-structure attention and a variational graph auto-encoder, and belongs to the field of graph data mining. As dynamic graph data are more and more widely applied to the fields of social networks, financial services, network security and the like, dynamic graph anomaly detection attracts more and more attention. Due to dynamic nature and complexity brought by continuous change of a topological structure of a dynamic graph, an existing dynamic graph anomaly detection method has certain limitation in the aspects of time feature modeling and spatial-temporal feature dependency relation extraction, so that a model is difficult to fully capture interaction information of a structure and time in the dynamic graph. Therefore, the invention provides a dynamic graph anomaly detection method TSAVGA (Time-Structural Attention Graphics Array) combining time-structure attention graph embedding and a variational graph auto-encoder. According to the method, a time-structure attention mechanism is used for hierarchically extracting short-term space-time characteristics of a dynamic graph, and a variational graph auto-encoder based on GCN-GRU enhancement is used for modeling a long-term space-time dependency relationship of the dynamic graph. Meanwhile, a strategy based on spectral clustering anomaly injection is designed, so that the diversity of training data anomaly is enhanced. Experimental results on six real data sets show that the AUC value of the method provided by the invention is averagely improved by 1%-5% compared with that of the most advanced method in the prior art; and the stable anomaly detection performance can still be kept when the abnormal proportion changes, and the stability and accuracy of the method are verified.

Description

(I) TECHNICAL FIELD

[0001] The present application belongs to the field of graph data mining, and is a dynamic graph anomaly detection method based on time-structure attention and variational graph autoencoder. (II) BACKGROUND

[0002] Graph data, as a data structure capable of describing entities and their relationships in the real world, has been widely used in many fields such as social networks, financial risk control, and bioinformatics. Dynamic graphs are graphs whose structure changes over time. They not only accurately depict the complex relationships between entities, but also capture the dynamic evolution of these relationships over time, thereby revealing the underlying patterns and rules behind entities and their relationships. With the increasing demand for dynamic relationship analysis in various fields, dynamic graphs have become increasingly popular and in-depth. For example, in social networks, dynamic graphs are used to analyze the evolution of user relationships and information dissemination paths; in the financial services sector, dynamic graphs are used to model the dynamic changes in user transaction relationships; in the field of network security, dynamic graphs are used to analyze real-time changes in network traffic and device communication relationships to maintain network security and stability.

[0003] However, the dynamic nature of dynamic graphs also presents new challenges, as their evolution often involves abnormal phenomena such as sudden behavior, abnormal interactions, or structural mutations. These abnormal phenomena can have a significant impact on the accuracy of data analysis, and therefore dynamic graph anomaly detection has emerged to identify and detect abnormal patterns in dynamic graph data, thereby improving the reliability of data analysis. In recent years, dynamic graph anomaly detection technology has been widely used in various fields. For example, in social networks, dynamic graph anomaly detection is used to identify fake accounts and malicious behavior; in the financial sector, this technology helps to identify abnormal transaction patterns, thereby enhancing anti-fraud capabilities; in the field of network security, dynamic graph anomaly detection can be used to detect potential network attacks and data leakage risks. As research into dynamic graph anomaly detection deepens, scholars continue to develop new detection methods to improve the accuracy and applicability of anomaly detection, thereby advancing the technology of dynamic graph anomaly detection.

[0004] Existing dynamic graph anomaly detection methods typically start with dynamic graph modeling, extracting the latent representation of dynamic graph nodes to compare the differences in graph structure between different time snapshots for anomaly detection. According to the different graph representation learning methods used, existing dynamic graph anomaly detection methods can be broadly divided into two categories: discrete-time dynamic graph anomaly detection methods and continuous-time dynamic graph anomaly detection methods.

[0005] Anomaly detection methods for discrete-time dynamic graphs typically employ a GNN-RNN framework to extract spatiotemporal features from dynamic graphs, thereby enabling anomaly detection. These methods first divide the dynamic graph into a series of snapshots, then use GNN-based methods to extract graph structural information from each snapshot. Finally, an RNN model is used to capture dynamic changes between snapshots. For example, Dynamic-DGI first uses GCN and DGI to learn node embeddings for each snapshot, then uses LSTM to capture temporal features of the nodes, and finally combines this with the RRCF algorithm for anomaly detection. AddGraph first uses GCN to obtain node embeddings, then uses GRU to capture temporal changes in the dynamic graph, and uses reconstruction error for anomaly detection. StrGNN first constructs an h-hop closure subgraph of the nodes, then uses GCN and GRU to jointly learn spatiotemporal features of the subgraph nodes, and performs anomaly detection based on a binary cross-entropy loss. PIKACHU uses the Skip-gram model to learn node embeddings for the graph, then uses GRU to learn long-term temporal dependencies between nodes. Finally, a conditional probability distribution is used to estimate the probability of edge existence, thereby enabling anomaly detection.

[0006] To better capture the spatiotemporal features of dynamic graphs, dynamic graph anomaly detection methods have introduced attention mechanisms to optimize the GNN and RNN stages, thereby improving anomaly detection accuracy. For example, BEA uses an attention mechanism to optimize the node embedding process, assigning different weights to different edge types, and achieving more accurate node representation learning. THGNN introduces node-level and type-level attention mechanisms to fully consider the node's inherent characteristics and differences between node types when generating node embeddings. DGL-LS combines the attention mechanism with GRU to optimize the evolution of dynamic node features. GraphLSTA proposes the Long-Short-Term Temporal Attention Network (LSTAN), which first uses a GCN to capture the structural features of nodes and then employs a recurrent attention mechanism to extract long-term and short-term temporal features of dynamic graphs. TAAD uses a temporal graph attention network to extract node features and combines it with a generative adversarial network to predict whether a node is an anomaly. Although these methods can effectively extract spatiotemporal features of dynamic graphs for anomaly detection, their use of discrete temporal processing makes it difficult to fully utilize the continuity of time.

[0007] Continuous-time dynamic graph anomaly detection methods encode both structural and temporal features of dynamic graphs, generating node embedding representations that fuse spatio-temporal characteristics to improve dynamic graph anomaly detection performance. For example, NetWalk first samples nodes using a random walk method, then encodes the spatio-temporal features of dynamic graphs using a deep autoencoder, and uses a distance-based clustering method to identify anomalies. ESEAD selects nodes closely related to the target edge using random walk, then uses an encoder model combining multi-head attention mechanism to learn the spatio-temporal features of edge neighborhoods, and finally uses contrastive learning for anomaly detection. SAD uses a graph attention network (GAT) to pre-embed nodes, and further combines a temporal graph autoencoder to jointly encode the structural information of the graph and the timestamp information of the edge, effectively capturing the spatio-temporal features of dynamic graphs for anomaly detection. GADY uses a GAN framework as the generator, combines a PINT-based encoder, recursively aggregates neighbor node features to generate edge latent representations, and uses reconstruction error for anomaly determination.

[0008] RegraphGAN also introduces a generative adversarial network (GAN) to reconstruct dynamic graph structures, and uses a Transformer encoder to capture spatio-temporal characteristics of dynamic graphs for more accurate anomaly detection. AET proposes an attribute-encoding Transformer that fuses current structural information and historical interaction patterns of dynamic graphs to enhance the modeling ability of long-term temporal features of dynamic graphs, effectively improving the model's anomaly detection ability. Similarly, TADDY also uses a Transformer encoder and combines graph diffusion techniques to encode the structural and temporal features of dynamic graphs, respectively, and finally uses binary cross-entropy loss to train the model for anomaly detection. FALCON extends the encoding method of the Transformer encoder based on TADDY, combining neighbor-based encoding and non-neighbor-based encoding to capture more fine-grained node dynamic features. Although the above methods can better handle the continuity of time and capture the original temporal information of dynamic graphs, there is still room for improvement in modeling time-structure dependencies. (III) SUMMARY

[0009] Due to the existing dynamic graph anomaly detection method, in the graph embedding process, the time information of the edge of the dynamic graph cannot be fully mined, and it is difficult to capture the deep dependence relationship between the time and structure of the dynamic graph. This makes the graph representation learning unable to comprehensively represent the spatio-temporal characteristics of the dynamic graph, and limits its performance in anomaly detection and other tasks. Therefore, the present application proposes a dynamic graph anomaly detection method combining time-structure attention and GRU variational graph autoencoder. The method uses a time-structure attention mechanism to effectively extract the short-term spatio-temporal features of the dynamic graph; at the same time, a GRU-based variational graph autoencoder framework is used for time series modeling to capture the time series variation law of the nodes of the dynamic graph, thereby enhancing the anomaly detection ability of the model. When the training negative sample data is generated in the anomaly detection module, the present application designs an anomaly injection strategy based on spectral clustering, which injects anomalies between different clusters of the original graph data, more realistically simulates the complexity and uncertainty of the dynamic graph data in the actual application scenario.

[0010] The technical content of the present application is as follows:

[0011] Step one: extract short-term spatio-temporal features of dynamic graph

[0012] The original dynamic graph data contains graph structure information and edge timestamp information. In order to deeply mine the short-term spatio-temporal features of the dynamic graph, the present application constructs a time-structure attention mechanism from the structure information of the dynamic graph and the timestamp information of the edge. First, based on the timestamp of the edge, the time attention mechanism is used to calculate the attention score of the edge, and the edge weight is converted through normalization processing; then, combined with the generated time dimension edge weight, the structure attention mechanism is used to calculate the edge weight twice; finally, the edge weight is aggregated to aggregate the neighbor node features, effectively capturing the short-term spatio-temporal features of the dynamic graph.

[0013] Step two: extract long-term spatio-temporal features and time-structure dependence relationship of dynamic graph

[0014] The abnormal phenomenon in the dynamic graph is usually caused by the joint action of the evolution of the time dimension and the change of the structure dimension, and if the time-structure dependence relationship of the dynamic graph is not effectively modeled, it will significantly affect the accuracy of anomaly detection. In order to capture the long-term spatio-temporal features of the dynamic graph and the dependence relationship between time and structure, the present application uses a GRU-based variational graph autoencoder in the extraction of long-term spatio-temporal features. This model can accurately capture the long-term variation law of the dynamic graph in the time series and the global structure features by virtue of the memory ability of GRU for time series data and the learning ability of variational graph autoencoder for graph structure. In the modeling of time-structure dependence relationship, the GCN-GRU architecture is introduced, which combines the structure feature extraction ability of graph convolution network (GCN) with the long time series feature capture ability of GRU, to realize the deep mining of the interaction relationship between time-structure features in the dynamic graph.

[0015] Step three: generating negative sample data of dynamic graph edges

[0016] To enhance the richness of the abnormality injection and the integrity of the abnormality category in the training data set. When generating negative sample data for anomaly detection, the present application proposes a training data enhancement method based on spectral clustering to inject abnormalities in the training data. The method first clusters the nodes in the graph, and then assigns category labels to each node according to the clustering results. Then, according to the proportion of each category of nodes in the whole, different categories of nodes are selected to generate abnormal edges, so as to simulate the abnormality in the real world in a more systematic and diverse way.

[0017] Step four: dynamic graph abnormal edge detection

[0018] The abnormality detection of the edge in the dynamic graph can be formally represented as the abnormality scoring problem of the edge in the dynamic graph. The present application extracts the spatio-temporal features of the nodes in the dynamic graph, and takes the average value of the endpoint features of each edge in the training data after data enhancement as the feature representation of the edge. An abnormality scoring function is constructed using a fully connected layer to calculate the abnormality score of the edge, so as to determine whether the edge is abnormal.

[0019] The existing dynamic graph anomaly detection method based on graph representation learning has two technical problems: first, the existing graph embedding method often focuses on capturing the structural information of the dynamic graph, and ignores the time information of the edge in the dynamic graph, as shown in Figure 1 Second, there is a lack of modeling and analysis of the time-structure feature dependency relationship of the dynamic graph. This makes it difficult for the model to fully capture the spatio-temporal evolution law of the dynamic graph, significantly restricting the accuracy and reliability of the anomaly detection. The present application overcomes the defects of the prior art through the following creative design, which specifically describes as follows:

[0020] (1) A graph embedding method based on time-structure attention is proposed, which effectively extracts the short-term spatio-temporal features of the dynamic graph using time-structure attention, while taking into account the time and structure information, so as to more comprehensively represent the global characteristics of the dynamic graph.

[0021] (2) An encoding method of a variational graph autoencoder based on GRU is proposed, which uses GRU inside the encoder to capture the long-term change information of the time-structure of the dynamic graph; at the same time, a GCN-GRU model combining GCN and GRU is used to effectively capture the deep dependency relationship between the time-structure of the dynamic graph.

[0022] (3) An abnormality injection strategy based on spectral clustering is designed to construct the training data of the anomaly detection model, which generates diversified abnormal edges through clustering to simulate the abnormality in the real world, and enhances the diversity and rationality of the training data.

[0023] (4) The experimental results on six real datasets show that the method is superior to existing methods in anomaly detection performance, with an average performance improvement of 1% to 5%, and exhibits excellent accuracy and stability. (IV) DESCRIPTION OF DRAWINGS

[0024] Figure 1 A comparison chart of the graph embedding method of the present application and the conventional graph embedding method.

[0025] Figure 2 A whole framework chart of the present application.

[0026] Figure 3 A GRU variational graph auto-encoder structure chart of the present application.

[0027] Figure 4 A comparison chart of the AUC values of different anomaly detection methods of the present application with the change of training rounds.

[0028] Figure 5 A sensitivity analysis result chart of the embedding dimension of the present application.

[0029] Figure 6 A sensitivity analysis result chart of the training proportion of the data set of the present application.

[0030] Figure 7 An analysis result chart of the weight of each component of the loss function of the present application.

[0031] Figure 8 A sensitivity analysis result chart of the snapshot size of the present application. (V) PREFERRED EMBODIMENT

[0032] In order to make the purpose, technical scheme and advantages of the present application clearer, the present application will be further described in detail below in combination with specific examples and with reference to the drawings.

[0033] In view of the problems existing in the existing dynamic graph anomaly detection method, the present application proposes a dynamic graph anomaly detection method (Dynamic Graph Anomaly Detection via Temporal-Structural Attention and Variational Graph Auto-Encoder, TSAVGA) combining temporal-structural attention and GRU variational graph auto-encoder. The overall framework of the present application is shown in Figure 2 .

[0034] Graph embedding module based on temporal-structural attention

[0035] In order to learn the graph embedding representation containing both time features and structure features at the same time, the application proposes a graph embedding module based on time-structure attention. This method unifies the time characteristics and structure characteristics of dynamic graphs into an embedding framework by combining time attention and structure attention, realizing more comprehensive graph data feature representation learning. Firstly, the time information of each edge in the dynamic graph is encoded by using the sine-cosine encoding method, so as to obtain the fixed-dimensional time feature representation As shown in formula (1), wherein d represents the feature dimension. Then, the attention score of the edge is calculated by using the attention mechanism, and the attention score is normalized, and then the final edge weight is obtained, as shown in formula (2). t i,j =encode(t i,j ) (1) Wherein, encode(·) is a sine-cosine encoding function. MLP(·) is a multi-layer perceptron for extracting high-order features of the edge. a is a learnable weight vector for transforming the attention score. The neighbor set of node i is represented as k represents the kth neighbor of node i. After obtaining the edge weight of the graph, the structure information of the dynamic graph is extracted by using the structure attention combined with the edge weight. Specifically, first, a randomly initialized learnable vector is assigned to each node of the graph as its initial feature. Then, the node embedding is iteratively updated by using the stacked attention layer. For the lth layer of the node embedding, the attention score of the edge is calculated by using the structure attention, as shown in formula (3). Wherein, a and W are learnable vectors of the attention mechanism, w i,j is the edge weight, and represent the embedding of nodes i and j at the lth layer respectively. Then, each node aggregates information from its neighbor nodes and updates its feature representation as: Wherein σ is the activation function ReLu. Finally, the node embedding representation of the whole dynamic graph is obtained by aggregating all node features: X=[x1,x2,…,x n ] T .

[0036] Variational graph autoencoder module based on GRU

[0037] To capture the time-varying information of dynamic graphs, the present application uses a variational graph autoencoder to realize the synchronous encoding of time and structure information, in which the GCN is used as the core of the encoder and the MLP is used as the decoder. Specifically, the encoder uses the node embedding representation obtained by time-structure attention as input, and introduces a GRU internally to capture dynamic time features in dynamic graphs. To further enhance the modeling ability of the model on the time and structure dependence relationship, the present application replaces the traditional fully connected layer in the GRU with a GCN encoder layer to capture deeper dependence between time and structure of dynamic graphs. The specific structure of the encoder is as shown in Figure 3 .

[0038] (1) GCN encoder-MLP decoder

[0039] For the original dynamic graph, first, the edges are sorted according to the time sequence, and then divided into multiple snapshots according to the preset number of edges. For a single snapshot, a feature mapping function is used to map the node embedding X t to a hidden space X' t as the input of the encoder. X' t = ReLU(linear(X t )) (5) Initialize a GCN layer containing current node information as the main body of the encoder. In order to consider the influence of node historical information on the current state of the node, a hidden state h t-1 is introduced in the encoder. h t-1 represents the historical information of the previous timestamp. Then, the conditional probability distribution in the latent space is generated by the encoder, and the reparameterization trick is used to sample Z t from the normal distribution of the encoder: wherein, respectively represent the mean and standard deviation matrix of the encoder in the latent space at timestamp t, W μ and W σ are learnable weight matrices used to convert the features output by the GCN into mean and variance parameters, ∈ is a random variable sampled in the standard normal distribution, and · represents the dot product operation. For the latent variable Z t , the present application uses an MLP as a decoder to reconstruct X t :

[0040] (2) GCN-GRU module

[0041] Since the anomaly of dynamic graphs is often driven by time variation and structure variation, the lack of modeling of time-structure dependency will affect the accuracy of anomaly detection. Therefore, the application selects GRU to update the current time hidden state h t to capture the long-term time variation information in dynamic graphs. And the graph convolution operation of GCN is used to update the hidden state h t of GRU to capture the deeper dependency between time-structure of dynamic graphs. Specifically, in order to better consider the dependency between time-structure, the application replaces the fully connected layer in GRU with GCN layer. GCN-GRU is used to update the hidden state h t , as shown in equations (11-14). z t =σ(GCN xz (X' t )+GCN hz (h t-1 )) (11) r t =σ(GCN xr (X' t )+GCN hr (h t-1 )) (12) Where sigma is the sigmoid activation function, z t , r t and h' represent the update gate, reset gate and hidden state of GRU respectively. represents the element-wise multiplication operation. Using GCN-GRU to get the hidden state h t , and combining equations (6-10) to update the latent variable Z t , the reconstructed feature of the node is calculated On this basis, the overall loss function of the variational graph autoencoder module is defined as: Where x i is the feature of each node of the encoder input, the node feature reconstructed by the decoder. The mean square error is used as the reconstruction loss, which is used to evaluate the difference between the node feature reconstructed by the decoder and the node feature x i embedded in the graph.

[0042] Spectral clustering based training data enhancement module

[0043] ​To enhance the richness of the injected anomalies and the completeness of the anomaly categories in the training dataset, the present application introduces a spectral clustering method to enhance the processing of the training data. This method first clusters the nodes in the graph, and then assigns a category label to each node according to the clustering results, and then selects nodes of different categories to generate abnormal edges according to the proportion of each category node in the whole, in order to simulate the anomalies in the real scene in a more systematic and diverse way. Specifically, first, the node embedding combined with the time-structure attention graph embedding is used to construct the similarity matrix of the graph using cosine similarity, as shown in equation (16), then combined with the similarity matrix, the spectral clustering algorithm is used to cluster the nodes, and according to the clustering results, each node is assigned a label, as shown in equations (17-18): SC=Clustering(k,S) (17) label=SC(S) (18) Where k is the expected number of clusters, SC represents the clustering result calculated by the spectral clustering algorithm, and label is the label corresponding to each node after spectral clustering. Then, according to the number of nodes of each category, the number of abnormal edges between categories is determined: Where, N i,j is the number of abnormal edges that need to be connected between category i and category j, n label(i) is the number of nodes with label i, n is the number of nodes in the graph, and m is the number of abnormal edges to be injected. On this basis, nodes between the corresponding categories are selected to be connected to generate abnormal edges, and it is ensured that the generated abnormal edges are not repeated to enhance the diversity and authenticity of the anomalies. e i,j,neg ={(v i ,v j )|label(i)≠label(j)},|e i,j,neg |=N i,j (20) Finally, the injected abnormal edges are combined with the normal edges in the original graph to generate a dynamic graph training data containing normal edges and abnormal edges. This provides a complete and diverse test scenario for subsequent anomaly detection tasks. ε train =ε neg ∪ε pos (21) Through the spectral clustering method for data enhancement, the anomalies in the real world can be more accurately simulated, the rationality and diversity of the injected anomalies can be improved, and thus the performance of the anomaly detection model can be better evaluated.

[0044] Anomaly detection module

[0045] The present invention uses time-structure attention graph embedding and variational graph autoencoder module to obtain the node v in the snapshot The final representation Z in t , where Z t Each row of represents the feature of a node. The average value of the endpoint features of each edge in the training set after data augmentation is selected as the feature of the edge, as shown in formula (22). Then, a fully connected neural network layer is used as the anomaly scoring function, and the edge features are used to calculate the anomaly score, as shown in formula (23): Among them, σ is the sigmoid activation function, and are the learnable weight matrices and bias terms. Since dynamic graph data only contains normal edges and abnormal edges, the anomaly detection problem of the graph can be regarded as a binary classification task. Therefore, the present invention uses binary cross entropy as the loss function to train the anomaly detector. For each snapshot The corresponding loss function is: in, is a positive sample, is the negative sample corresponding to the positive sample obtained by abnormal injection. The reconstruction loss obtained by combining the variational graph autoencoder of formula (15) The final loss function of the model is:

[0046] The embodiments of the present invention have achieved some positive effects during the development or use process and have certain advantages over the existing technology. The following describes them with reference to data, charts, etc. during the test process.

[0047] (1) Experimental dataset

[0048] The datasets used in this paper come from the fields of social networks, financial transactions, and network topology. The statistical information of the datasets is shown in Table 1. Table 1 Six public dynamic graph datasets

[0049] (2) Benchmark method

[0050] To verify the effectiveness of the proposed method, we compared it with seven state-of-the-art benchmark methods. The details are as follows:

[0051] NetWalk uses random walk to generate context information and uses an autoencoder model to learn node embeddings. AddGraph uses GCN to learn node embedding representations and extracts short-term and long-term dynamic changes through the GRU-Attention module. StrGNN focuses on target edges, extracts h-hop closure subgraphs, and combines GCN and GRU to learn edge structure information. TADDY constructs a spatio-temporal node encoding method and uses Graph-Transformer to capture the coupled spatio-temporal information in dynamic graphs. RegraphGAN uses a Transformer encoder to improve the traditional graph generative adversarial network and combines edge-based substructure sampling and spatio-temporal node encoding methods to generate latent representations of nodes for dynamic graph anomaly detection. GraphLSTA effectively extracts and fuses long and short-term temporal features of dynamic graphs using a long short-term temporal attention network, and then captures their evolution patterns to detect anomalies. FALCON uses fine-grained sampling and embedding modules to utilize complex temporal information, and introduces an attention alignment strategy to minimize the difference in context attention between source nodes and target nodes.

[0052] These benchmark methods cover from traditional random walk-based graph embedding methods to the latest GNN-based graph embedding methods, which can comprehensively evaluate the anomaly detection performance of the proposed method.

[0053] (3) Experimental design

[0054] For each dynamic graph dataset, the present application divides it into a training set and a test set according to the order of the edges of the dynamic graph, where the first 50% of the edges are used as the training set and the last 50% of the edges are used as the test set. The data augmentation method proposed in the present application is used to inject anomalies, and negative sample data is injected in the training set at a ratio of 1:1 for training the anomaly detection model. To ensure fairness, the same anomaly injection strategy as the comparison method [13, 14] is used in the test set, and 1%, 5%, and 10% of the test set are injected with abnormal data.

[0055] The present application uses AUC-ROC (abbreviated as AUC) as a performance indicator to evaluate the effect of anomaly detection. The ROC curve depicts the relationship between the true positive rate and the false positive rate at different thresholds to evaluate the performance of the classification model. Among them, the abnormal samples are considered as "positive" and the normal samples are considered as "negative". AUC is defined as the area under the ROC curve, which represents the probability that the abnormal samples are correctly identified as abnormal, and the value range is 0-1. The larger the AUC value, the better the performance of the anomaly detection model.

[0056] (4) Parameter settings

[0057] In the experiments of the present application, the corresponding parameters are configured according to the characteristics of each data set. Specifically, the node embedding dimension of BTC-Alpha, BTC-OTC, Digg and UCI Messages is set to 128, the node embedding dimension of AS-Topology is set to 64, and the node embedding dimension of Email-DNC is set to 512. The snapshot size of UCI Messages and BTC-OTC is set to 1000, the snapshot size of Email-DNC and BTC-Alpha is set to 2000, and the snapshot size of Digg and AS-Topology is set to 6000. For the number of training rounds, the present application sets the number of training rounds of BTC-OTC, UCI Messages, BTC-Alpha and Digg to 200 rounds, and sets the number of training rounds of Email-DNC and AS-Topology to 400 rounds. The present application uses Adam as the optimizer, and sets the learning rate according to the characteristics of the data set, wherein the learning rate of UCI Messages, Email-DNC and AS-Topology is 0.001, and the learning rate of Digg, BTC-Alpha and BTC-OTC is 0.0005.

[0058] (5) Experimental results

[0059] In order to verify the anomaly detection effect of the method of the present application, the proposed method is compared with seven benchmark methods in different anomaly injection ratios for anomaly detection accuracy, and the experimental results are shown in Table 2, wherein the optimal result of each column is marked in bold, and the suboptimal result is marked with an underscore.

[0060] As can be seen from Table 2, the method proposed in the present application is higher than the comparison methods on six different anomaly ratio dynamic graph data sets. Compared with the best result of these benchmark methods, the average performance of TSAVGA on AUC is increased by 2.47%. This is mainly due to the fact that TSAVGA can effectively extract the original time information and structure information of dynamic graph through time-structure attention; by combining GCN-GRU in variational graph autoencoder, the long-term dependence change relationship between time and structure is effectively captured, thereby further improving the detection ability of the anomaly.

[0061] In addition, when the anomaly ratio changes, the method of the present application shows strong stability. When the anomaly ratio is 1%, 5% and 10%, the AUC value of TSAVGA is increased by 2.28%, 2.61% and 2.58% respectively. Compared with other methods, the fluctuation of AUC value of TSAVGA under different anomaly ratios is smaller, which further verifies the stability of the model under various conditions.

[0062] Notably, the TSAVGA achieved a significant improvement in anomaly detection on the AS-Topology dataset, with an average of 4.6% higher AUC value than the optimal method FALCON. This is mainly due to two aspects: on the one hand, the method fully considers the influence of different types of nodes when injecting anomalies; on the other hand, the method not only effectively utilizes the time information of the edges, but also captures the dependence between time and structure. In contrast, FALCON, although it also utilizes the time and structural characteristics of dynamic graphs, fails to effectively exploit their relationship, thus limiting its performance improvement.

[0063] To more intuitively demonstrate the performance comparison of the method and other methods, the experiment was conducted on the UCIMessages and Digg datasets with 1% anomaly ratio, and the AUC value was tested with the change of training rounds (Epochs). The results are shown in FIG. 2. Figure 4 As can be seen from the figure, TSAVGA exhibits a high AUC value at the beginning of training; as the number of training rounds increases, the AUC value continues to rise and quickly reaches a peak. Compared with other methods, TSAVGA converges faster and maintains a high level in the later training period. Other methods show a slight performance decline in the later training period, while TSAVGA maintains stable performance. This also shows that TSAVGA has stronger learning ability and higher stability on different datasets. Table 2 AUC values of anomaly detection results

[0064] (6) Ablation experiment

[0065] To test the contribution of each component in the TSAVGA method to the overall performance, this section conducts an ablation experiment on the main modules. Specifically, w / o att_t: this variant removes the time attention part of the time-structure attention, w / o: this variant removes the variational graph autoencoder loss, w / o GCN-GRU: this variant removes the GCN layer in the GRU. Experiments were conducted on each dataset with an anomaly ratio of 1%, 5%, and 10%. To ensure fairness, the hyperparameter settings and training rounds in the experiment remain unchanged. The experimental results are shown in Table 3, where the optimal result of each column is marked in bold and the suboptimal result is marked with an underscore.

[0066] w / o att_t: According to the data in the table, it can be observed that the anomaly detection performance of the method of the application on each data set appears different degrees of decline. The reason for this phenomenon may be that only using structural attention makes the embedding result of the node only contain static node features, which cannot effectively capture the time information of the edge in the dynamic graph, thereby weakening the expression ability of the node embedding in the time dimension, resulting in the decline of the overall performance.

[0067] w / o It can be observed from the results in the table that the impact on the anomaly detection performance is smaller on the Email-DNC, Bitcoin-Alpha, Bitcoin-OTC and Digg four data sets. But on the UCI Messages, AS-Topology two data sets, the impact is more significant. This shows the importance of reconstruction loss in improving the performance of the model in a specific scenario, and embodies the effectiveness of the module.

[0068] w / o GCN-GRU: The experimental results show that on the UCI Messages, Bitcoin-Alpha, Bitcoin-OTC, Digg four data sets, the performance of the anomaly detection model has declined to a certain extent. The reason may be that the anomaly in the dynamic graph is caused by the joint action of time and structure change, which further verifies the effectiveness of the GCN-GRU module. Table 3 ablation experiment results

[0069] (7) Parameter sensitivity analysis

[0070] In order to test the influence of different hyperparameters on the anomaly detection results. On the UCI Messages, Bitcoin-Alpha, Bitcoin-OTC three data sets, the application respectively carries out related experiments on four hyperparameters of embedding dimension, training data set proportion, loss weight of each component and snapshot size.

[0071] Node embedding dimension

[0072] The embedding dimension of the node in this experiment is selected as {8, 16, 32, 64, 128, 256, 512}, and experiments are carried out on the test set with 1%, 5%, 10% anomaly proportion. The results are shown in Figure 5 Figure 5 ​It can be seen that under different anomaly proportions, with the increase of embedding dimension, the AUC value shows a gradual upward trend, and reaches a peak when the dimension is 128. However, when the embedding dimension continues to increase, the AUC value begins to decrease slightly. The reason is as follows: when the embedding dimension is too small, the graph embedding may not effectively capture the spatio-temporal feature information of the nodes, resulting in poor performance of the model anomaly detection. When the embedding dimension is too large, the complexity of the anomaly detection model increases, and the training process is difficult to converge, thereby affecting the final stability and performance.

[0073] Training proportion of data set

[0074] In this experiment, the training proportions of {20%, 30%, 40%, 50%, 60%, 70%} were selected in the above three data sets, and the experiment was carried out on the test set with 10% anomaly proportion. The results are shown in Figure 6 As can be seen from Figure 6 When the training proportion decreases, the AUC value shows a downward trend, but the decrease is not significant. When the training proportion exceeds 50%, the AUC value is basically stable. This shows that even with limited training data, the model of the present application can maintain high stability. In addition, as the training proportion increases, the variance of the AUC value decreases, further verifying that the method of the present application has good stability and robustness under different training scales.

[0075] Weight of each component of loss function

[0076] In this experiment, the weight of each component of the loss function was evaluated. The sum of the weights was set to 1, and the experiment was carried out on the test set with 1%, 5%, and 10% anomaly proportion. The results are shown in Figure 7 As can be seen from Figure 7 On three different data sets, the sum of the weights of formula (25) has a certain influence on the AUC value of anomaly detection. Overall, when the sum decreases, the AUC value decreases to different degrees, and when the sum is close to 1, the AUC value reaches the highest value, indicating that the model performs best at this time. Therefore, the weight sum of the loss in the method of the present application is set to the optimal state to obtain better anomaly detection results.

[0077] Snapshot size

[0078] In this experiment, snapshots of sizes {100, 500, 1000, 2000, 4000, 6000} were selected, and experiments were carried out under the conditions of 1%, 5%, and 10% anomaly proportion. The results are shown in Figure 8 As can be seen from Figure 8It can be seen that with the increase of snapshot size, the AUC value presents a trend of first rising and then falling. The reason is as follows: when the snapshot is small, the number of nodes in a single snapshot is small, resulting in insufficient structure feature extraction. When the snapshot is large, although the model can effectively extract the structure feature, the number of snapshots is reduced, resulting in a decline in the ability to capture the time feature. This shows that in dynamic graph data, the snapshot size plays a key role in the balance between structure features and time features.

[0079] The above merely describes a specific implementation of the present application, but the protection scope of the present application is not limited thereto, and any modification, equivalent replacement, and improvement made by any person skilled in the art within the technical range disclosed by the present application and within the spirit and principle of the present application shall be encompassed within the protection scope of the present application.

Claims

1. This paper proposes a dynamic graph anomaly detection method based on temporal-structural attention and variational graph autoencoder. It is characterized by: include: Step 1: Construct a temporal-structural attention mechanism to extract short-term spatiotemporal features of nodes. Based on the timestamps of the edges in the dynamic graph, the temporal attention mechanism is used to calculate the edge attention score, which is then normalized to obtain the edge weight. Combined with the edge weight, the structural attention mechanism is used to recalculate the edge weight, aggregating the features of neighboring nodes to obtain the spatiotemporal features of the node, thus capturing the short-term spatiotemporal features of the dynamic graph. Step 2: Use a GRU-based variational graph autoencoder to mine long-term spatiotemporal features and time-structure dependencies. Leveraging the GRU's ability to memorize time series data and the variational graph autoencoder's ability to learn graph structure, we can capture the long-term changes and global structural features of dynamic graph time series. The GCN-GRU architecture is introduced, combining the structural feature extraction capability of graph convolutional network (GCN) with the long-term feature capture capability of GRU to explore the interaction between time-structure features in dynamic graphs. Step 3: Use spectral clustering to enhance training data. Spectral clustering is used to cluster nodes in the dynamic graph and assign category labels to the nodes. Based on the proportion of nodes in each category in the overall dataset, nodes of different categories are selected to generate abnormal edges, simulating abnormal situations in real scenarios. Step 4: Perform anomaly detection. Using the node features extracted in steps 1 and 2, take the average of the endpoint features of each edge in the data-augmented training set as the feature representation of the edge. A fully connected layer is used to construct an anomaly scoring function, calculate the anomaly score of the edge, and determine whether the edge is abnormal.

2. The method for detecting anomalies in a dynamic graph according to claim 1, wherein: The specific implementation of the temporal-structural attention mechanism includes: To simultaneously learn graph embedding representations that incorporate both temporal and structural features, this paper proposes a graph embedding module based on temporal-structural attention. By combining temporal and structural attention, this method unifies the temporal and structural characteristics of dynamic graphs into a single embedding framework, enabling more comprehensive feature representation learning for graph data. The sine-cosine coding method is used to encode the temporal information of each edge in the dynamic graph to obtain a fixed-dimensional temporal feature representation, as shown in formula (1). The attention mechanism is used to calculate the attention score of the edge, and the normalized processing is used to obtain the final edge weight, as shown in formula (2). t i,j =encode(t i,j ) (1) Assign a randomly initialized learnable vector as the initial feature to each node of the graph, and use stacked attention layers to iteratively update the node embedding; for the lth layer of the node embedding, use structural attention to calculate the attention score of the edge, as shown in formula (3). Update the node feature representation by aggregating neighbor node information, as shown in formula (4). Finally, aggregate all node features to obtain the node embedding representation X = [x1, x2, ..., x n ] T .

3. The method for detecting anomalies in dynamic graphs according to claim 1, wherein: The specific implementation of the GRU-based variational graph autoencoder includes: Using the node embedding representation obtained by time-structure attention as input, the encoder introduces a GRU to capture the dynamic temporal characteristics of the dynamic graph. The traditional fully connected layer in the GRU is replaced with a GCN encoder layer to improve the model's ability to model temporal and structural dependencies. The original dynamic graph is sorted by the time sequence of the edges and divided into multiple snapshots. The node embedding is mapped to the latent space X' using the feature mapping function. t As the encoder input, the hidden state is introduced in the encoder, considering the node history information h t-1 Impact on the current state. The encoder generates the conditional probability distribution in the latent space, and the reparameterization technique samples Z from the normal distribution of the encoder. t : X' t =ReLU(linear(X t )) (5) For the latent variable Z t , the present invention uses MLP as a decoder to reconstruct X t : Use GRU to update the current hidden state h t , capturing the long-term temporal change information of dynamic graphs. The GCN graph convolution operation is used to update the hidden state h of GRU t , replace the fully connected layer in GRU with the GCN layer, use GCN-GRU to update the hidden state, and combine (6-10) to update the latent variables and calculate the node reconstruction features. z t =σ(GCN xz (X' t )+GCN hz (h t-1 )) (11) r t =σ(GCN xr (X' t )+GCN hr (h t-1 )) (12) On this basis, the overall loss function of the variational graph autoencoder module is defined as: Among them, x i is the feature of each node input to the encoder, The node features reconstructed by the decoder are evaluated using the mean square error as the reconstruction loss. and the node features x of the graph embedding i The difference between.

4. The method for detecting anomalies in dynamic graphs according to claim 1, wherein: The training data enhancement step of the spectral clustering includes: Combining the node embedding of the time-structure attention graph embedding, the cosine similarity is used to construct the graph similarity matrix, as shown in Equation (16). The spectral clustering algorithm is used to cluster the nodes and assign a label to each node, as shown in Equations (17-18): SC=Clustering(k,S) (17) label=SC(S) (18) According to the number of nodes in each category, the number of abnormal edges between categories is determined, as shown in formula (19): Select nodes between corresponding categories to connect and generate abnormal edges. Ensure that the abnormal edges are not repeated. Combine the injected abnormal edges with the normal edges in the original graph to generate dynamic graph training data containing normal edges and abnormal edges. yes i,j,neg ={(v i ,v j )|label(i)≠label(j)},|e i,j,neg |=N i,j (20) Finally, the injected anomalous edges are combined with the normal edges in the original graph to generate dynamic graph training data containing both normal and anomalous edges. This provides a complete and diverse testing scenario for subsequent anomaly detection tasks. e train =e neg ∪e pos (21).

5. The method for detecting anomalies in dynamic graphs according to claim 1, wherein: The anomaly detection step includes: The final representation of the node in the snapshot is obtained by using the time-structure attention graph embedding and the variational graph autoencoder module. The average value of the endpoint features of each edge in the training set after data augmentation is selected as the edge feature, as shown in Equation (22): A fully connected neural network layer is used as the anomaly scoring function, and the anomaly score is calculated using the edge features, as shown in Equation (23): The binary cross entropy is used as the loss function to train the anomaly detector; combined with the reconstruction loss obtained by the variational graph autoencoder, the loss function of the final model is determined:

6. A method for detecting anomalies in dynamic graphs based on temporal-structural attention and variational graph autoencoders according to any one of claims 1 to 5, characterized in that: The dynamic graph anomaly detection method based on temporal-structural attention and variational graph autoencoder consists of four modules: A time-structure attention graph embedding module, configured to execute the time-structure attention mechanism construction and node spatiotemporal feature extraction operations as described in step 1 of claim 1 and claim 2; A GRU variational graph autoencoder module, configured to perform the GRU-based variational graph autoencoder operation as described in step 2 of claim 1 and claim 3, and to capture the long-term temporal dependency features and time-structure dependency relationships of the dynamic graph; A training data enhancement module, configured to perform the spectral clustering training data enhancement operation as described in step 3 of claim 1 and claim 4, and construct diverse anomaly detection model training data; The anomaly detection module is used to perform the anomaly detection operation described in step 4 of claim 1 and claim 5, calculate the anomaly score of the dynamic graph edge and determine whether the edge is abnormal.

Images