Graph anomaly detection method based on knowledge modeling and double auto-encoders
By introducing knowledge modeling and dual autoencoder methods in attribute network anomaly detection, the problem of difficulty in capturing complex interactions and detecting mixed anomalies in the prior art is solved, and more efficient anomaly detection performance is achieved.
Patent Information
- Application Number
- CN202411729832.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-28
- Publication Date
- 2025-04-08
AI Technical Summary
Existing attribute network anomaly detection methods are difficult to effectively capture complex cross-modal interactions, and cannot effectively detect mixed-type anomalies.
Using the graph anomaly detection method based on knowledge modeling and dual autoencoder, node embedding and attribute embedding are learned respectively through structural autoencoder and attribute autoencoder, and human knowledge is integrated through comparison loss, and network topology and attributes are reconstructed to calculate exception scores.
Improve the performance of attribute network anomaly detection, can more effectively capture complex cross-modal interactions and detect mixed-type anomalies.
Smart Images
Figure CN119782973A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of deep learning and attribute graph anomaly detection, and specifically relates to a graph anomaly detection method based on knowledge modeling and dual autoencoders. Background Art
[0002] Attribute networks are ubiquitous in the real world. As a kind of graph structured data, in addition to having a complex topological structure, each node also contains rich attribute feature information. As attribute networks are widely used to model various complex systems, such as social networks, financial transaction networks, biological networks, etc., anomaly detection research on attribute networks has attracted more and more attention in the research community. In the context of attribute networks, anomaly detection aims to identify nodes whose patterns and behaviors deviate significantly from most normal nodes. Nowadays, attribute network anomaly detection has become a basic task and has a wide range of applications in real-world scenarios, especially in social spam detection and fraud detection.
[0003] In the past decade, anomaly detection methods based on attribute networks have made significant progress and can be divided into two categories: one is a shallow method without deep learning, and the other is a method based on deep learning. Shallow methods mainly rely on various types of heuristic anomaly measurements, but when applied to complex graph data, due to the nonlinearity of the data, the performance of anomaly detection may not be ideal. With the development of graph neural networks, graph anomaly detection technology based on GNN has been widely studied. Specifically, anomaly detection methods based on graph deep learning are usually based on encoder-decoder schemes. The encoder takes the given original attribute network as input and applies the graph neural network to learn the potential representation of nodes, while the decoder reconstructs the network topology and attribute features, and uses the reconstruction error between the reconstructed attribute network and the original network as the criterion for judging abnormal nodes. However, it is worth noting that the autoencoders designed by some existing methods are still insufficient in capturing the complex cross-modal interactions in the attribute graph, and there are often mixed types of anomalies in the graph structure and node attributes of the attribute network. Therefore, it is necessary to design an attribute network anomaly detection technology that can mine more abnormal patterns. Summary of the invention
[0004] Purpose of the invention: The present invention proposes a graph anomaly detection method based on knowledge modeling and dual autoencoders.
[0005] Technical solution: The graph anomaly detection method based on knowledge modeling and dual autoencoders proposed in the present invention comprises the following steps:
[0006] (1) Knowledge modeling is introduced to provide complementary insights for anomaly detection by leveraging human prior knowledge about different types of anomalies, and anomaly graphs are generated based on the original attribute graphs.
[0007] (2) Through the structured autoencoder, the node embeddings of the original attribute graph and the anomaly graph are learned in the latent space respectively, and then the modeled knowledge is integrated into the encoder through a carefully designed contrastive loss.
[0008] (3) Obtain the attribute embedding of the original attribute graph through the attribute autoencoder.
[0009] (4) Use the node embedding of the original attribute graph to reconstruct the network topology, and combine the node embedding and attribute embedding to reconstruct the network attributes. Calculate the anomaly scores of all nodes based on the reconstruction error to detect anomalies.
[0010] The step (1) of knowledge modeling comprises the following steps:
[0011] (11) Since the dataset used does not contain the ground truth, a certain proportion of anomalies are first injected into the original network. The dataset after the anomaly injection is called the original attribute graph.
[0012] (12) Given human prior knowledge of different anomaly types, four known common anomaly types on the structure side and attribute side are introduced into the original attribute graph and anomaly graphs are generated accordingly to model and formalize specific comparison samples.
[0013] The specific form of the structure autoencoder model in step (2) is:
[0014] (21) In order to obtain sufficiently representative high-level node features, the original node attributes are first converted into low-dimensional latent representations through MLP, as follows:
[0015]
[0016] Where X is the feature matrix of the original attribute graph, and are the weight matrix and bias of MLP respectively, and σ is the nonlinear activation function.
[0017] (22) After obtaining the transformed low-dimensional latent representation, the graph attention layer GAT is used to aggregate the representations from neighboring nodes by performing a shared attention mechanism on the nodes. The formula is as follows:
[0018]
[0019] where e ij is node v i For node v j The importance weight of attn represents the attention calculated by the parameterized neural network, a and are weight vectors and weight matrices shared by all nodes. ij is the weight coefficient after softmax normalization, N iRepresents node v i 's neighborhood. is the original attribute graph node v i The final node embedding is obtained by weighted summation of the learned attention weights.
[0020] After the graph attention network GAT, the node embedding of the original attribute graph can be obtained
[0021] (23) In order to make full use of the role of human knowledge in anomaly graphs, the anomaly graphs are compared with the original attribute graphs, and binary and ternary contrasts are introduced.
[0022] 1) Binary comparison
[0023] The above structured autoencoder obtains the final node embedding of the original attribute graph and the abnormal graph through the GAT layer. and Binary comparison is to compare the representation of each node in the original attribute graph and the abnormal graph. The loss function formula is as follows:
[0024]
[0025] in Indicates that node i is considered an anomaly in the anomaly graph, It means that node i is considered normal in the anomaly graph, and m is a boundary value.
[0026] 2) Ternary comparison
[0027] Specifically, consider each connected node pair (i, j) where j is identified as an anomaly in the anomaly graph and i remains unchanged. The triplet consists of the representation of node i in the original attribute graph and the representation of node j in the original attribute graph and the anomaly graph. The loss function formula is as follows:
[0028]
[0029] The specific form of the attribute autoencoder model in step (3) is:
[0030] (31) The attribute autoencoder only targets the original attribute graph.
[0031] (32) The attribute autoencoder uses two nonlinear feature transformation layers to map the observed attribute data to latent attribute embeddings as follows:
[0032]
[0033] Where X T represents the transpose of the original attribute graph feature matrix, and are the weight matrix and bias of the first nonlinear transformation layer, and are the weight matrix and bias of the second nonlinear transformation layer, For the final attribute embedding.
[0034] The reconstruction of attribute network and anomaly detection in step (4) comprises the following steps:
[0035] (41) Based on the original attribute graph, the node embedding learned from the structured autoencoder reconstructs the network topology, which is specifically expressed as reconstructing the adjacency matrix, as follows:
[0036]
[0037] Where sigmoid is the activation function, is the reconstructed adjacency matrix.
[0038] (42) The attribute feature matrix is reconstructed by combining the node embedding learned from the structure autoencoder and the attribute embedding learned from the attribute autoencoder of the original attribute graph. The formula is as follows:
[0039]
[0040] (43) The anomaly score of each node is calculated and sorted according to the reconstruction error, and the nodes with higher scores are regarded as abnormal nodes.
[0041] The reconstruction error loss function is as follows:
[0042]
[0043] Among them, α is a hyperparameter that weighs the structural reconstruction and attribute reconstruction.
[0044] Finally, the loss function of the entire model is expressed as follows:
[0045]
[0046] So each node v i The anomaly score of is obtained by the reconstruction error, and the formula is as follows:
[0047]
[0048] Among them, a i and are matrices A and The i-th row, x i and are matrices X and The i-th row of .
[0049] Beneficial effects: Compared with the prior art, the beneficial effects of the present invention are as follows: 1. Through the deep model, the network sparsity and data nonlinearity problems that cannot be solved by shallow learning are overcome. A graph neural network is introduced, with topological structure and node attributes as input, and node embedding is learned by superimposing multiple layers of linear units and nonlinear activation functions. 2. Human prior knowledge of different anomaly types is used to model and integrate into the encoder, and the performance of anomaly detection is further improved through comparison strategies. 3. Different from a single autoencoder, the structure and attribute autoencoders are separated to capture the complex cross-modal interactions of the attribute network. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1 It is a flow chart of the present invention;
[0051] Figure 2 Four types of anomalies for modeling knowledge; DETAILED DESCRIPTION
[0053] The present invention is further described in detail below with reference to the accompanying drawings and tables.
[0054] Three widely used benchmark datasets for graph anomaly detection are used here, including the Cora citation dataset and the Flickr and BlogCatalog social network datasets, as shown in Table 1. Table 1 Details of the benchmark datasets For citation datasets, nodes are papers and edges are citation relationships between papers. For social network datasets, nodes are users and edges are relationships between users. The graph anomaly detection method based on knowledge modeling and dual autoencoders proposed in this paper has an operation flow chart as shown below: Figure 1 The specific steps are as follows:
[0055] 1. Knowledge modeling, including injecting anomalies into the benchmark dataset to obtain the original attribute graph and generating anomaly graphs using human prior knowledge about different anomaly types.
[0056] 1) Since the benchmark dataset lacks ground truth, anomaly injection is performed manually, including structural anomaly and attribute anomaly injection. Structural anomaly injection is to select a fixed number of nodes to be fully connected, and these nodes are considered structural anomaly nodes. Repeat this process multiple times to generate multiple structural anomaly clusters. Attribute anomaly injection is to randomly select a part of nodes, and for each selected node, select the attribute of the node with the largest Euclidean distance to its attribute from the candidate nodes to replace it, and these nodes are considered attribute anomalies.
[0057] 2) Consider four different types of anomalies in the attribute network, namely, height and remote anomalies on the structure side and deviation and disproportionate anomalies on the attribute side. The schematic diagrams of various anomalies are shown in Figure 2 As shown. For the height anomaly on the structural side, in order to simulate the anomaly, a certain number of average degree nodes are selected, and then they are connected to other random nodes; for the remote anomaly on the structural side, a certain number of nodes are selected and most of their edges are deleted in the network. For the deviation anomaly on the attribute side, some central nodes are selected, and for each central node, some nodes are randomly selected from the entire network, and the similarity between the attribute vector of the central node and the attribute vector of other nodes is calculated, and then the most dissimilar attribute vector is assigned to the central node; for the disproportionate anomaly on the attribute side, the values of certain node attributes are scaled and reduced with a preset probability.
[0058] 2. Construction of structural autoencoder.
[0059] The structural autoencoder targets both the original attribute graph and the abnormal graph. The purpose is to perform a multi-dimensional comparison between the original attribute graph and the abnormal graph after obtaining the node embeddings of the two, thereby optimizing the potential representation so that the node embedding can increase the distance between normal nodes and abnormal nodes.
[0060] 1) In order to obtain sufficiently representative high-level node features, the original node attributes are first converted into low-dimensional potential representations through MLP. The formula is as follows:
[0061]
[0062] Where X is the feature matrix of the original attribute graph, and are the weight matrix and bias of MLP respectively,
[0063] σ is a nonlinear activation function such as ReLU or Tanh. Similarly, a low-dimensional potential representation of the abnormal graph can be obtained
[0064] 2) After obtaining the converted low-dimensional latent representation, the graph attention layer GAT is used to aggregate the representations from neighboring nodes by performing a shared attention mechanism on the nodes. The formula is as follows:
[0065]
[0066] where e ij is node v i For node v j The importance weight of attn represents the attention calculated by the parameterized neural network, a and are weight vectors and weight matrices shared by all nodes. ijis the weight coefficient after softmax normalization, N i Represents node v i 's neighborhood. is the original attribute graph node v i The final node embedding is obtained by weighted summation of the learned attention weights.
[0067] After the graph attention network GAT, the node embedding of the original attribute graph can be obtained Similarly, we can get the node embedding of the abnormal graph
[0068] 3) In order to make full use of the role of human knowledge in the anomaly graph, the anomaly graph is compared with the original attribute graph. The comparison can well characterize the anomaly pattern on the attribute network. Since the enhanced anomaly nodes are different from themselves and their adjacent nodes, the comparison strategies are introduced: binary comparison and ternary comparison. The former compares the node embeddings of the anomaly graph and the original attribute graph, and the latter compares between a pair of connected nodes.
[0069] (1) Binary Contrast
[0070] The above structured autoencoder obtains the final node embedding of the original attribute graph and the abnormal graph through the GAT layer. and Binary comparison is to compare the representation of each node in the original attribute graph and the abnormal graph. The loss function formula is as follows:
[0071]
[0072] in Indicates that node i is considered an anomaly in the anomaly graph, It means that node i is considered normal in the anomaly graph, and m is a boundary value.
[0073] (2) Ternary comparison
[0074] Specifically, consider each connected node pair (i, j) where j is identified as an anomaly in the anomaly graph and i remains unchanged. The triplet consists of the representation of node i in the original attribute graph and the representation of node j in the original attribute graph and the anomaly graph. The loss function formula is as follows:
[0075]
[0076] 3. Attribute autoencoder construction.
[0077] 1) The attribute autoencoder only targets the original attribute graph.
[0078] 2) The attribute autoencoder uses two nonlinear feature transformation layers to map the observed attribute data to potential attribute embeddings, as follows:
[0079]
[0080] Where X T represents the transpose of the original attribute graph feature matrix, and are the weight matrix and bias of the first nonlinear transformation layer, and are the weight matrix and bias of the second nonlinear transformation layer, For the final attribute embedding.
[0081] 4. Reconstruct attribute network and anomaly detection.
[0082] 1) Reconstruct the network topology based on the node embedding learned from the structured autoencoder based on the original attribute graph, which is specifically manifested as reconstructing the adjacency matrix. The formula is as follows:
[0083]
[0084] Where sigmoid is the activation function, is the reconstructed adjacency matrix.
[0085] 2) Combine the node embedding learned from the structure autoencoder and the attribute embedding learned from the attribute autoencoder of the original attribute graph to reconstruct the attribute feature matrix. The formula is as follows:
[0086]
[0087] 3) Calculate the anomaly score of each node based on the reconstruction error and sort them, and regard the nodes with higher scores as abnormal nodes.
[0088] The reconstruction error loss function is as follows:
[0089]
[0090] Among them, α is a hyperparameter that weighs the structural reconstruction and attribute reconstruction.
[0091] Finally, the loss function of the entire model is expressed as follows:
[0092]
[0093] So each node v i The anomaly score of is obtained by the reconstruction error, and the formula is as follows:
[0094]
[0095] Among them, ai and are matrices A and The i-th row, x i and are matrices X and The i-th row of .
Claims
1. A graph anomaly detection method based on knowledge modeling and dual autoencoders, characterized in that: The following steps are involved: (11) Knowledge modeling is introduced to provide complementary insights for anomaly detection by leveraging human prior knowledge about different types of anomalies, and anomaly graphs are generated based on the original attribute graphs. (12) Through the structured autoencoder, the node embeddings of the original attribute graph and the anomaly graph are learned in the latent space respectively, and then the modeled knowledge is integrated into the encoder through a carefully designed contrastive loss. (13) Obtain the attribute embedding of the original attribute graph through the attribute autoencoder. (14) The network topology is reconstructed using the node embedding of the original attribute graph, and the network attributes are reconstructed by combining the node embedding and the attribute embedding. The anomaly scores of all nodes are calculated based on the reconstruction error to detect anomalies.
2. The graph anomaly detection method based on knowledge modeling and dual autoencoders according to claim 1 is characterized in that: The knowledge modeling described in step (1) includes the following steps: (21) Since the dataset used does not contain the ground truth, a certain proportion of anomalies are first injected into the original network. The dataset after the anomaly injection is called the original attribute graph. (22) Given human prior knowledge of different anomaly types, four known common anomaly types on the structure side and attribute side are introduced into the original attribute graph and anomaly graphs are generated accordingly to model and formalize specific comparison samples.
3. The graph anomaly detection method based on knowledge modeling and dual autoencoders according to claim 1 is characterized in that: The structural autoencoder described in step (2) specifically includes: (31) The structural autoencoder targets both the original attribute graph and the anomaly graph. In order to obtain sufficiently representative high-level node features, the node attributes are first converted into low-dimensional latent representations through a multi-layer perceptron (MLP). (32) The graph attention network GAT is used to perform a shared attention mechanism on the nodes, aggregate the representations from neighboring nodes, and obtain node embedding. (33) In order to fully utilize the role of human knowledge in the anomaly graph, it is necessary to compare the anomaly graph with the original attribute graph. Since the enhanced anomaly nodes are different from themselves and their adjacent nodes, two comparison strategies are introduced: binary comparison and ternary comparison. The former compares the node embeddings of the anomaly graph and the original attribute graph, and the latter compares between a pair of connected nodes.
4. The graph anomaly detection method based on knowledge modeling and dual autoencoders according to claim 1 is characterized in that: The attribute autoencoder described in step (3) specifically includes: (41) The attribute autoencoder only targets the original attribute graph. (42) The attribute autoencoder uses two nonlinear feature transformation layers to map the observed attribute data to latent attribute embeddings.
5. The graph anomaly detection method based on knowledge modeling and dual autoencoders according to claim 1, characterized in that: The reconstruction of attribute network and anomaly detection described in step (4) includes the following steps: (51) The network topology is reconstructed based on the node embeddings learned from the structured autoencoder based on the original attribute graph, which is specifically manifested as reconstructing the adjacency matrix. (52) The attribute feature matrix is reconstructed by combining the node embeddings learned from the structure autoencoder and the attribute embeddings learned from the attribute autoencoder of the original attribute graph. (53) The anomaly score of each node is calculated and sorted according to the reconstruction error, and the nodes with higher scores are regarded as abnormal nodes.