Construction method and application of two-layer evolving dynamic graph convolutional neural network model
By introducing a dynamic graph convolutional neural network model with two-layer evolution in graph data representation learning, combining GCN and RNN to evolve GCN parameters and node embeddings, the universality of new node feature learning and embeddings in graph data dynamic representation learning is solved. The problem is to effectively capture the dynamics and stability of graph data structures and node embeddings, and is suitable for a variety of graph data applications.
Patent Information
- Application Number
- CN202111543402.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2021-12-16
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2041-12-16
AI Technical Summary
The dynamic representation learning model of existing graph data has shortcomings in the commonality of new node feature learning and node embedding results, especially inability to effectively capture the dynamic and stable features of graph data.
A dynamic graph convolution neural network model DEDGCN with two-layer evolution is proposed. The graph convolution neural network GCN and recurrent neural network RNN are used to double-layer evolution of GCN network parameters and node embeddings to build an unsupervised dynamic graph representation learning framework. The GCN network parameters are corrected through the loss function, and the changes in the graph data structure characteristics are captured and the stability of node embedding is ensured.
DEDGCN can effectively learn dynamic and stable features in graph data, and is suitable for a variety of graph data application tasks, especially in unsupervised tasks, which improves the commonality of new node feature learning and node embedding results. sex.
Smart Images

Figure CN114254738B_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of graph data processing, and in particular relates to a method for constructing a double-layer evolving dynamic graph convolutional neural network model and its application. Background Art
[0002] Many complex systems in real life are composed of graph data, including social networks, finance, biological molecules, e-commerce platforms, knowledge graphs, etc., which can be converted into graph data composed of nodes and node interactions. Scholars are paying more and more attention to the study of graph data. However, due to the diverse forms and complex connection relationships of graph data, graph data is more difficult to represent than other types of data. This makes it an important part of graph data research to present the elements such as nodes, edges, and attributes in a stable form. In particular, with the successful application of deep learning technology in many fields, converting graph data into vector representation is currently the most popular way to learn graph representation. The more mature technologies mainly include Deepwalk and Node2vec based on random walks, GraRep based on matrix decomposition, struc2vec and DyREP based on graph structure characteristics, and SDNE based on neural networks. In particular, the rise of graph neural networks (GNNs) has brought new directions to the research of graph representation learning technology and has brought it into a new stage of development.
[0003] However, in real life, graph data is not static, but evolves over time. For example, friends change in a friendship network, devices are updated in a computer topology network, users and products increase and decrease in a user product network, and so on. These graph data often have their own time attributes during the interaction process. For example, in an e-commerce platform, the transaction records of users purchasing products contain time attributes, calls between users in a telephone network have time attributes, and emails sent in an email network have time attributes. Currently popular deep learning models mainly target static graph data, ignoring the inherent time attributes of graph data, and are unable to learn the evolution of nodes and their relationships in graph data and update node embeddings, making the graph representation results lack dynamics and authenticity.
[0004] In recent years, the dynamic representation learning technology of graph data has been widely developed. Graph Convolution Networks (GCN) in graph neural networks have the characteristics of simple structure, low complexity, few training times, and good learning effect. As the basis of graph data representation, it has been applied by many models. GCN itself cannot capture the dynamic characteristics of graph data, while recursive neural network (RNN) can process sequence information containing time relations, which means that RNN can learn the dynamic characteristics of nodes in graph data evolving over time. Therefore, many methods combine GCN with RNN technology to construct dynamic graph representation learning models, such as GCRN, RgCNN, AddGraph, etc. These methods input the graph data structure features captured by GCN into RNN, learn the temporal relationship of nodes, and capture the dynamic characteristics of graph data. However, as the graph data evolves, the graph structure changes. A single GCN model can only capture the characteristics of the currently existing nodes, and the learning effect of the characteristics of new nodes is poor. EvolveGCN proposed by Pareja [Pareja A, Domeniconi G, Chen J, et al. EvolveGCN: Evolving Graph Convolutional Networks for Dynamic Graphs [C] / / AAAI. 2020: 5363-5370.] uses RNN to evolve the parameters of the GCN model, so that the graph structure features extracted by GCN at different times have temporal characteristics. EvolveGCN is a supervised learning method and cannot be applied to unlabeled data, especially tasks such as clustering and community discovery, which leads to poor universality of node embedding results. Summary of the invention
[0005] In view of the problems that the existing dynamic representation learning of graph data has poor feature learning effect for new nodes and poor universality of node embedding results, the present invention proposes a double-layer evolutionary dynamic graph convolutional neural network model construction method and application. The constructed model DEDGCN can capture stability characteristics while learning node dynamic characteristics.
[0006] In order to achieve the above object, the present invention adopts the following technical solutions:
[0007] On the one hand, the present invention proposes a method for constructing a double-layer evolving dynamic graph convolutional neural network model. Based on the graph convolutional neural network GCN, the recurrent neural network RNN is used to perform double-layer evolution on the network parameters and node embedding of GCN respectively, so as to capture the changes in the structural characteristics of the graph data and ensure the temporal stability of the node embedding. The loss function is constructed using the node embedding evolution results and the results generated by GCN, the GCN network parameters are corrected, and an unsupervised dynamic graph representation learning framework is formed to complete the construction of the double-layer evolving dynamic graph convolutional neural network model DEDGCN.
[0008] Furthermore, the GCN consists of two graph convolutional layers, which extract the structural features of nodes in the network based on the features of neighbor nodes; for the graph G t =(V t ,E t ), V t represents the set of nodes at the snapshot at time t, E t Represents the set below the snapshot at time t. The propagation rules between graph convolutional layers are as follows:
[0009]
[0010] A′ t =A t +I
[0011] Among them, H t (l) It represents the result obtained after the graph data at time t passes through the lth layer of graph convolution. Represents the adjacency matrix A of the graph data at time t t The regularized form of A′ t represents the self-connection matrix of the graph data at time t, D represents the degree matrix, and W t (l) represents the parameters of the lth graph convolutional layer at time t, and σ(·) represents the activation function;
[0012] The input of GCN is the vector of nodes in the graph data at time t. After L layers of graph convolution, the output of GCN is the learned node embedding.
[0013] Furthermore, the GCN parameters of the lth layer at time t-1 are Input into LSTM to get the GCN parameter W of the lth layer at time t t (l) , the calculation method of GCN parameter evolution is as follows:
[0014]
[0015]
[0016]
[0017]
[0018]
[0019]
[0020]
[0021] where i (t) Indicates the calculation result of the input gate at time t, U (i) represents the weight parameter of the input gate, B (i) represents the bias parameter of the input gate, f (t) Indicates the calculation result of the forget gate at time t, U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t) Indicates the calculation result of the output gate at time t, U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t, U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t) represents the cell update result at time t, c (t-1) It represents the cell update result at time t-1, and tanh() represents the activation function.
[0022] Furthermore, the nodes obtained by GCN at different times are embedded into {X t-w+1 ,X t-w+2 ,…,X t} is input into LSTM to predict the embedding of the node at the next moment. The calculation method of GCN node evolution is as follows:
[0023] P t+1 =LSTM(X t ,P t )
[0024] i (t+1) =sigmoid(W (i) x (t) +U (i) P t +B (i) )
[0025] f (t+1) =sigmoid(W (f) x (t) +U(f) P t +B (f) )
[0026] o (t+1) =sigmoid(W (o) x (t) +U (o) P t +B (o) )
[0027]
[0028]
[0029]
[0030] Where w represents the window size of LSTM, P t represents the predicted node embedding at time t, i (t+1) represents the calculation result of the input gate at time t+1, W (i) , U (i) represents the weight parameter of the input gate, B (i) represents the bias parameter of the input gate, f (t+1) Indicates the calculation result of the forget gate at time t+1, W (f) , U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t+1) Indicates the calculation result of the output gate at time t+1, W (o) , U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t+1, W (c) , U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t+1) represents the cell update result at time t+1, c (t) Represents the cell update result at time t, and tanh() represents the activation function.
[0031] Furthermore, the loss function is:
[0032]
[0033]
[0034]
[0035]
[0036] Where loss represents the loss function, α and β represent and , P represents the predicted node embedding, X represents the node embedding, and W represents the weight parameter in the LSTM model.
[0037] The second aspect of the present invention proposes a node classification method based on a two-layer evolving dynamic graph convolutional neural network model DEDGCN, comprising:
[0038] The labeled node features are learned through DEDGCN, and the node embedding probability is calculated through the feedforward neural network and softmax function to determine the type of node u at time t.
[0039] The third aspect of the present invention proposes an edge classification method based on a two-layer evolving dynamic graph convolutional neural network model DEDGCN, comprising:
[0040] The labeled edge features are learned through DEDGCN, and the probability of node embedding is calculated through feedforward neural network and softmax function to determine the type of edge (u, v) at time t.
[0041] The fourth aspect of the present invention proposes a link prediction method based on a two-layer evolving dynamic graph convolutional neural network model DEDGCN, comprising:
[0042] DEDGCN is used to calculate the embedding information of nodes u and v at time t and before, and the embedding information of nodes u and v at time t and before is used to predict whether the edge (u, v) exists at time t+1. The features of nodes u and v are aggregated, and then a feedforward neural network is used to obtain the existence probability of the edge.
[0043] Compared with the prior art, the present invention has the following beneficial effects:
[0044] The present invention proposes a dual-layer evolutionary dynamic graph convolutional neural network model DEDGCN. Based on the GCN model, RNN is used to perform dual-layer evolution on the GCN network parameters and node embedding, ensuring that GCN can capture the ever-changing structural features of graph data, and feed back the evolution results of node embedding to the GCN network, correct the GCN network parameters, and enable the model to learn the dynamic features of nodes while capturing stability features. On the other hand, the model uses the stability features of nodes and the structural features of graphs to construct a loss function, so that the model can be applied to unsupervised tasks and is suitable for application tasks of graph data in various scenarios. BRIEF DESCRIPTION OF THE DRAWINGS
[0045] Figure 1 A schematic diagram of the framework of a dual-layer evolving dynamic graph convolutional neural network model DEDGCN according to an embodiment of the present invention;
[0046] Figure 2 Schematic diagram of the GCN parameter evolution process according to an embodiment of the present invention;
[0047] Figure 3 Schematic diagram of the node evolution process of an embodiment of the present invention;
[0048] Figure 4 The F1 value of different models in the node classification task on the Elliptic dataset;
[0049] Figure 5 The F1 values of different models for edge classification tasks on the Reddit Hyperlink Network and Bitcoin Alpha datasets. DETAILED DESCRIPTION
[0050] The present invention will be further explained below with reference to the accompanying drawings and specific embodiments:
[0051] First, we formally define the basic concepts and related problems of dynamic graph representation learning.
[0052] G=(V,E) represents a graph, which is a set of nodes V={v1,....,v N} and the connection relationship (edge) between nodes E∈{(v i ,v j )|(v i ,v j )∈V×V}, where N is the number of nodes. A is the adjacency matrix of the graph, with a size of N×N. ij ∈E,A ij =1, otherwise A ij = 0. If the graph is undirected, A is a symmetric matrix. A dynamic graph consists of a series of continuously evolving graphs. Indicates that every graph G t =(V t ,E t ) is called a snapshot. Where t represents the time corresponding to the snapshot, V t represents the set of nodes in the snapshot at the tth moment, E t represents the set below the snapshot at time t, A t Represents the adjacency matrix of the graph below at time t snapshots. Generally speaking, we define the problem related to our work as follows.
[0053] Definition 1: (Graph Representation Learning)
[0054] For a given graph G = (V, E), graph representation learning is defined as Map the nodes in the graph to a d-dimensional vector space. Here d < < |V|. The function f can preserve the structural properties of the network, and the mapping result is expressed in the matrix In simple terms, in practice, the closer the relationship between nodes, the smaller the distance in the embedding space of the nodes.
[0055] Definition 2: (Dynamic Graph Representation Learning)
[0056] For a given dynamic graph Dynamic graph representation learning is defined as learning the set of functions F = {f1,f2,…,f T}, the dynamic graph The nodes in are mapped into the d vector space. The mapping function representing the snapshot of the network t can transform the node set V of the snapshot t into t All nodes in the d-vector space are mapped to ensure the similarity of nodes in both structure and time dimensions.
[0057] A method for constructing a dual-layer evolving dynamic graph convolutional neural network model. This method is based on GCN and uses recurrent neural networks to perform dual-layer evolution on GCN network parameters and node embeddings, respectively, to capture changes in the structural characteristics of graph data. At the same time, it ensures the temporal stability of node embeddings, that is, the state of nodes will not change dramatically over a period of time. We use the node embedding evolution results and the results generated by GCN to construct a loss function, correct the GCN network parameters, form an unsupervised dynamic graph representation learning framework, and complete the construction of the dual-layer evolving dynamic graph convolutional neural network model DEDGCN. The DEDGCN construction method is mainly divided into three parts: graph convolutional neural network GCN, dual-layer evolution, and GCN parameter correction. The dual-layer evolution includes GCN parameter evolution and node evolution. The DEDGCN framework diagram is shown in the figure. Figure 1 It is worth noting that, for the convenience of description, it is assumed that at each moment, the number of nodes contained in the network is N, but the number of nodes in the network at each moment changes with time and is not completely the same, and needs to be distinguished in the actual application process.
[0058] (1) Graph Convolutional Neural Network (GCN)
[0059] For graph data representation learning technology, we use the GCN model as the main body for the following two reasons:
[0060] 1. The GCN model does not require training to extract good graph data features. Our method belongs to an unsupervised network, and only learns the embedding of nodes through the network structure. Although GCN belongs to supervised learning, its good ability to extract graph data features makes it fully suitable for unsupervised learning.
[0061] 2. The GCN network structure is simple. A GCN network usually has 2 to 3 layers to achieve good results, which makes the model have fewer parameters and reduces the complexity of time evolution of parameters.
[0062] Generally, GCN consists of two graph convolutional layers, which extract the structural features of nodes in the network based on the features of neighboring nodes. t =(V t ,E t ), V t represents the set of nodes at the snapshot at time t, E t Represents the set below the snapshot at time t. The propagation rules between graph convolutional layers are as follows:
[0063]
[0064] A′ t =A t +I
[0065] Here, H t (l) It represents the result obtained after the graph data at time t passes through the lth layer of graph convolution. The adjacency matrix A of the graph data at time t is defined as t The regularized form of A′ t is the self-connection matrix of the graph data at time t, D is the degree matrix, W t (l) represents the parameters of the lth graph convolutional layer at time t. σ(·) refers to the activation function, such as the ReLU function.
[0066] The input H of GCN t (0) =X t is the vector of nodes in the graph data at time t, usually obtained through initialization rules. After L layers of graph convolution, the output of GCN is H t (L) , represents the learned node embedding. Figure 1 It can be seen that at time t, the graph data passes through two layers of graph convolutional layers to obtain node embedding.
[0067] (2) Two-layer evolution
[0068] As graph data evolves, the structural features of the graph are constantly changing. A single GCN model cannot capture this dynamic change, which requires the network to have a certain degree of adaptability and be able to extract dynamic structural features according to the evolution of the graph. In addition, the nodes of dynamic graph data are stable, and the features will not change dramatically over a period of time. This requires GCN to be able to adaptively extract dynamic graph structural features while ensuring node stability. Since recurrent neural networks can capture dynamic features in temporal relationships and have good prediction and evolution capabilities, we use recurrent neural networks to evolve the parameters of GCN so that GCN can adaptively change over time and extract dynamic graph structural features. At the same time, we use recurrent neural networks to evolve the node embedding results, capture the stability features of the nodes, feed them back to the GCN network, and correct the GCN parameters so that GCN can learn the stability features of the nodes.
[0069] a. GCN parameter evolution
[0070] At present, the commonly used RNN memory cell is the Long-Short Term Memory (LSTM), which consists of an input gate, a forget gate, and an output gate. It can selectively capture the information of the time sequence, save key information, and forget redundant information. For the parameter evolution and node evolution of GCN, we choose LSTM as the memory cell and evolve the GCN parameters. The process is as follows: Figure 2 As shown. The GCN parameters of the lth layer at time t-1 are Input into LSTM to get the GCN parameter W of the lth layer at time t t (l) .
[0071] The calculation method of GCN parameter evolution is shown in the following formula (1):
[0072]
[0073]
[0074]
[0075]
[0076]
[0077]
[0078]
[0079] where i (t) Indicates the calculation result of the input gate at time t, U (i) represents the weight parameter of the input gate, B(i) represents the bias parameter of the input gate, f (t) Indicates the calculation result of the forget gate at time t, U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t) Indicates the calculation result of the output gate at time t, U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t, U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t) represents the cell update result at time t, c (t-1) It represents the cell update result at time t-1, and tanh() represents the activation function.
[0080] In the process of GCN parameter evolution, the GCN parameters at time t-1 have the historical information of GCN in the past period of time. Through LSTM cells, new GCN parameters with dynamic characteristics are further maintained, learned and generated, thereby ensuring the dynamic characteristics between each snapshot and effectively capturing the structural characteristics of dynamically changing graph data.
[0081] b. Node evolution
[0082] The evolution of GCN parameters is only related to the dynamics of graph data from the perspective of the model, while the dynamics of the nodes themselves have a direct effect on the representation of graph data, that is, the changes of nodes over time directly affect the quality of node embedding. Therefore, we use LSTM to extract the dynamic features of the nodes themselves, further strengthen the stability features of the nodes, and lay the foundation for correcting network parameters.
[0083] By capturing the node behavior over a period of time, we can predict the behavior of the node at the next moment. When the predicted node embedding is similar to the real node embedding, it means that the node embedding learned by the GCN model can directly capture the dynamic characteristics of the node itself. Since LSTM has the functions of selective memory and selective amnesia, it can predict time series data. Therefore, we embed the nodes obtained by GCN at different moments into {X t-w+1 ,X t-w+2 ,…,X t} is input into LSTM to predict the embedding of the node at the next moment. The process is as follows Figure 3 As shown. Where w represents the window size of LSTM, x (t) represents the node embedding calculated at time t, P t It represents the predicted node embedding at time t, as shown in the following formula (2):
[0084] P t+1 =LSTM(X t ,P t )
[0085] i (t+1) =sigmoid(W (i) x (t) +U (i) P t +B (i) )
[0086] f (t+1) =sigmoid(W (f) x (t) +U (f) P t +B (f) )
[0087] o (t+1) =sigmoid(W (o) x (t) +U (o) P t +B (o) )
[0088]
[0089]
[0090]
[0091] i (t+1) represents the calculation result of the input gate at time t+1, W (i) , U (i) represents the weight parameter of the input gate, B (i) represents the bias parameter of the input gate, f (t+1) Indicates the calculation result of the forget gate at time t+1, W (f) , U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t+1) Indicates the calculation result of the output gate at time t+1, W (o) , U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t+1, W (c) , U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t+1) represents the cell update result at time t+1, c (t)Represents the cell update result at time t, and tanh() represents the activation function.
[0092] (III) GCN parameter correction
[0093] We use node stability, node evolution, and regularization to construct a loss function, feed back the GCN network, and correct the GCN parameters. The above three parts are explained below.
[0094] In a dynamic network, the nodes themselves are stable and their features will not change dramatically over a period of time. This makes the embeddings of nodes between adjacent snapshots as similar as possible. Even in a dynamic graph network with frequent additions and subtractions, the inherent features of the nodes will not change drastically. Therefore, the loss function is constructed by calculating the distance between the embeddings of adjacent snapshot nodes, as shown in formula (3).
[0095]
[0096] During the node evolution process, we use LSTM to predict nodes and construct the future form of node embedding. The predicted value of node embedding is consistent with the actual value learned by GCN. The similarity between the two is calculated and fed back to the GCN network to improve the evolution ability of GCN. The calculation method is shown in formula (4).
[0097]
[0098] Where P represents the predicted node embedding and X represents the node embedding.
[0099] To prevent overfitting during the learning process, we add L2 regularization to the loss function to reduce the size of the weight. As shown in formula (5), W represents the weight parameter in the LSTM model.
[0100]
[0101] The final loss function loss is defined as follows.
[0102]
[0103] Where α and β represent and The weight of .
[0104] To verify the effectiveness of the present invention, experiments were conducted on DEDGCN using publicly available basic datasets.
[0105] Dataset 1: Elliptic is a Bitcoin transaction graph, where nodes represent transactions and edges represent Bitcoin flows between transactions. The attributes of nodes in this graph are divided into two categories: legal and illegal, so we use this dataset to conduct experiments on node classification and node clustering tasks.
[0106] Dataset 2: Bitcoin Alpha is a platform for trading in Bitcoin. The transaction behaviors between users of the platform form graph data. The members of the platform rate other members on a scale of -10 to +10 to indicate the degree of trust of each member. Based on the ratings, we divide the ratings into two categories: trustworthy and untrustworthy. The transaction behaviors between users are constructed into a dynamic network, where nodes represent users and edges represent transactions between users, and the ratings of each user's transaction process are assigned a trust or untrustworthy label. We perform edge classification tasks and link prediction tasks on this graph dataset.
[0107] Dataset 3: Reddit Hyperlink Network extracts the link relationships in Reddit posts into a graph, each of which contains a time attribute and the sentiment (positive or negative) of the source post to the target post. We perform edge classification on this dataset to predict the sentiment relationship between unlinked posts.
[0108] Dataset 4: UCI is a graph of private messages sent on the UC-Irvine campus social platform. On the social platform, users can search for other people and then send conversations based on their profile information. An edge (u,v,t) represents a private message sent by user u to user v at time t. We perform link prediction on this dataset to capture the possible connections between users at the next moment.
[0109] Dataset 5: AS Network refers to a graph consisting of connections between autonomous systems. Nodes represent autonomous systems, and edges represent the connections between autonomous systems. We perform link prediction on this data to predict the connections of the network at the next moment.
[0110] The basic information of these data is shown in Table 1. According to the characteristics of different data sets, we divide them into snapshots at different time intervals and apply them to different tasks.
[0111] Table 1 Figure data information
[0112]
[0113] We will compare it with the following five basic methods.
[0114] Method 1: GCN is a static graph convolutional neural network that cannot extract dynamic features. We fuse all snapshots to form static graph data and conduct a comparative test on the importance of temporal features.
[0115] Method 2: GCN-GRU is a method that uses a fixed GCN to generate network node embeddings and uses GRU to evolve the temporal relationship of nodes.
[0116] Method 3: DynGEM is a dynamic unsupervised node representation method based on a deep autoencoder model, which trains the model at each moment by evolving the autoencoder model parameters.
[0117] Method 4: dyngraph2vec is a dynamic unsupervised node representation method that integrates autoencoder model, LSTM and MLP. This method includes three versions, dyngraph2vecAE, dyngraph2vecRNN and dyngraph2vecAERNN. dynagraph2vecAE is similar to DynGEM in architecture, so we use the second and third methods for comparison.
[0118] Method 5: EvolveGCN is similar to GCN-GRU. It uses RNN to evolve GCN parameters and learn the temporal relationship of the model.
[0119] As an implementable method, the evaluation methods used in the experiment include F1 value, MAP and MRR. The calculation of these three evaluation methods is introduced in detail below.
[0120] First, we need to make the following definitions:
[0121] True positive (TP): The number of samples whose predicted values are positive and whose true values are positive.
[0122] True negatives (TN): The number of samples whose predicted values are positive and whose true values are negative.
[0123] False positives (FP): The number of samples whose predicted values are negative and whose true values are positive.
[0124] False negatives (FN): The number of samples whose predicted values are negative and whose true values are negative.
[0125] Through the above four definitions, we can calculate the accuracy (Precision) and recall (Recall) of the prediction results. The formula is as follows:
[0126]
[0127]
[0128] The F1 value is the harmonic mean of precision and recall, which can objectively reflect the effectiveness of the prediction. The calculation formula is as follows:
[0129]
[0130] AP refers to the integral of the PR (Precision-Recall) curve, that is, the average precision of all recall values between 0 and 1. The calculation formula is as follows:
[0131]
[0132] MAP refers to the average value of AP of all categories. The calculation formula is as follows: K represents the number of AP categories.
[0133]
[0134] Specifically, the present invention verifies the effectiveness of the DEDGCN proposed in this paper in four aspects: node classification, edge classification and link prediction.
[0135] 1. Node classification: DEDGCN is used to learn the features of nodes with labels and predict the types of unlabeled nodes. This embodiment uses a feedforward neural network and a softmax function to perform probability calculations on node embeddings to determine the type of node u at time t. For node classification, this embodiment uses the F1 value to measure the effectiveness of the method.
[0136] 2. Edge classification: DEDGCN is used to learn the edge features with labels and predict the type of unlabeled edges. This embodiment uses a feedforward neural network (MLP) and a softmax function to perform probability calculations on node embeddings to determine the type of edge (u, v) at time t. The features of edge (u, v) are obtained by aggregating the features of node u and node v. The aggregation method uses the form of Hadamard product. The metric method for edge classification uses the F1 value.
[0137] 3. Link prediction: DEDGCN is used to calculate the embedding information of nodes u and v at time t and before, and the embedding information of nodes u and v at time t and before is used to predict whether the edge (u, v) exists at time t+1. We aggregate the features of nodes u and v, and then use MLP to obtain the probability of the existence of the edge. For link prediction, we use MAP (Mean Average Precision) for measurement.
[0138] Specifically, the experimental details are as follows:
[0139] 1. For any dataset, use one-hot node-degree as the input feature of the model.
[0140] 2. For all GCNs, we set the number of layers to 2; for all MLPs used in classification tasks, we set the number of layers to 2, and the classification probability is calculated using the softmax function. For classification tasks and link prediction tasks, the loss function of MLP uses cross entropy.
[0141] 3. For any dataset, we use a fixed-size embedding as the representation of the node, which is 100.
[0142] 4. Divide the data into training set, validation set and test set in a ratio of 8:1:1.
[0143] Specifically, the experimental results and analysis are as follows:
[0144] (1) Node classification
[0145] We use the Elliptic dataset for node classification tasks. For GCN parameters and node embedding evolution, we use a time window size of 5. The results of node classification are shown in Figure 4 .
[0146] from Figure 4 We can see that the static graph convolutional neural network has a poor representation effect on dynamic graph data. The F1 value is only 47% in the three-category task, and the two versions of EvolveGCN, version H and version O, are also not good for node classification of the Elliptic dataset, especially version H, with an F1 value of 44% for node classification. The reason is that the attributes of nodes on Elliptic are relatively stable, and it is difficult to convert between legal and illegal nodes. The GCN-GRU model and the DEDGCN model we proposed can evolve nodes, extract inherent features of nodes, and maintain the stability of nodes. In addition, the F1 value of DEDGCN using MLP for supervised classification reaches 77%. In addition to supervised classification, we use the clustering algorithm K-means to divide the types of nodes, and the effect is only 1% lower than the supervised method. This also shows that DEDGCN has a good effect on unsupervised clustering tasks.
[0147] (2) Edge classification
[0148] We use two datasets, Bitcoin Alpha and Reddit Hyperlink Network, for edge classification tasks. For GCN parameters and node embedding evolution, we use a time window size of 5. The edge classification results are shown in Figure 2. Figure 5 shown.
[0149] like Figure 5 As shown in the figure, in the edge classification task, the F1 value of DEDGCN on the Bitcoin Alpha dataset reaches 93%, and on the Reddit Hyperlink Network dataset it reaches 90%. At the same time, the F1 value of EvolveGCN is also higher than that of GCN and GCN-GRU. In graph data, nodes rely on edges to form graphs. Edges are an important component of the graph structure. The characteristics of edges mainly depend on the structure of the graph. In EvolveGCN and DEDGCN, the evolution of GCN parameters is relied on to dynamically capture graph features and more accurately grasp the graph structure, which enables good results in edge classification tasks. DEDGCN also considers the evolution law of the nodes themselves, and can grasp the development trend and interaction relationship of the nodes, making DEDGCN have better results in edge classification tasks.
[0150] (3) Link prediction
[0151] We conducted link prediction experiments on three datasets: Bitcoin Alpha, UCI, and AS Network. The results are shown in Table 2.
[0152] Table 2 MAP values for link prediction tasks
[0153]
[0154] As shown in Table 2, the MAP value of DEDGCN is greater than that of other methods, and remains above 0.15, which has a good effect in link prediction tasks. The MAP value is a comprehensive reflection of the accuracy and recall rate, and can reflect the global performance of the algorithm. The three datasets of BitcoinAlpha, UCI and AS Network belong to three different types of graph data: transaction network, social network and device network. DEDGCN has stable MAP values on different types of graph data, which directly shows that DEDGCN can be applied to most graph data and has good robustness.
[0155] The effectiveness of our proposed DEDGCN has been demonstrated through its performance in node classification, edge classification, and link prediction tasks. Based on the experimental results, we discussed the applicable task scenarios of GCN-RNN based on node evolution and EvloveGCN based on parameter evolution. DEDGCN combines the advantages of node evolution and parameter evolution, making the extracted node features dynamic while retaining the inherent stability characteristics of the nodes. It has achieved good results in various tasks, proving the wide applicability of DEDGCN. In addition, DEDGCN is an unsupervised graph representation model. We use unsupervised clustering methods to determine node types in the node classification task based on experimental data. The effect is only 1% lower than that of supervised classification, which also shows the effectiveness of DEDGCN in unsupervised tasks.
[0156] In summary, the present invention proposes a dual-layer evolutionary dynamic graph convolutional neural network model DEDGCN. Based on the GCN model, RNN is used to perform dual-layer evolution on the GCN network parameters and node embedding, ensuring that GCN can capture the ever-changing structural characteristics of graph data, and feed back the evolution results of node embedding to the GCN network, correct the GCN network parameters, and enable the model to learn the dynamic characteristics of nodes while capturing stability characteristics. On the other hand, the model uses the stability characteristics of nodes and the structural characteristics of graphs to construct a loss function, so that the model can be applied to unsupervised tasks and is suitable for application tasks of graph data in various scenarios.
[0157] The above is only a preferred embodiment of the present invention. It should be pointed out that a person skilled in the art can make several improvements and modifications without departing from the principle of the present invention. These improvements and modifications should also be regarded as within the scope of protection of the present invention.
Claims
1. A node classification method based on a two-layer evolving dynamic graph convolutional neural network model DEDGCN, characterized in that: include: The node features with labels in Elliptic are learned through DEDGCN, and the probability of node embedding is calculated through feedforward neural network and softmax function to determine the type of node u at time t. In Elliptic, nodes represent transactions, and the types of nodes include legal and illegal. The DEDGCN is constructed in the following manner: based on the graph convolutional neural network GCN, the recurrent neural network RNN is used to perform double-layer evolution on the network parameters and node embedding of the GCN respectively, to capture the changes in the structural characteristics of the graph data, and at the same time, to ensure the temporal stability of the node embedding, to construct the loss function using the node embedding evolution results and the results generated by GCN, to correct the GCN network parameters, to form an unsupervised dynamic graph representation learning framework, and to complete the construction of the double-layer evolutionary dynamic graph convolutional neural network model DEDGCN; The GCN consists of two graph convolutional layers, which extract the structural features of nodes in the network based on the features of neighbor nodes; t =(V t ,E t ), V t represents the set of nodes at the snapshot at time t, E t Represents the set below the snapshot at time t. The propagation rules between graph convolutional layers are as follows: in It represents the result obtained after the graph data at time t passes through the lth layer of graph convolution. Represents the adjacency matrix A of the graph data at time t t The regularized form of A t ′ represents the self-connection matrix of the graph data at time t, D represents the degree matrix, represents the parameters of the lth graph convolutional layer at time t, and σ(·) represents the activation function; The input of GCN is the vector of nodes in the graph data at time t. After L layers of graph convolution, the output of GCN is the learned node embedding. The GCN parameters of the lth layer at time t-1 are Input into LSTM to get the GCN parameters of the lth layer at time t The calculation method of GCN parameter evolution is as follows: where i (t) Indicates the calculation result of the input gate at time t, U (i) represents the weight parameter of the input gate, B (i) represents the bias parameter of the input gate, f (t) Indicates the calculation result of the forget gate at time t, U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t) Indicates the calculation result of the output gate at time t, U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t, U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t) represents the cell update result at time t, c (t-1) represents the cell update result at time t-1, and tanh() represents the activation function; The nodes obtained by GCN at different times are embedded into {X t-w+1 ,X t-w+2 ,…,X t } is input into LSTM to predict the embedding of the node at the next moment. The calculation method of GCN node evolution is as follows: P t+1 =LSTM(X t ,P t ) i (t+1) =sigmoid(W (i) x (t) +U (i) P t +B (i) ) f (t+1) =sigmoid(W (f) x (t) +U (f) P t +B (f) ) o (t+1) =sigmoid(W (o) x (t) +U (o) P t +B (o) ) Where w represents the window size of LSTM, P t represents the predicted node embedding at time t, i (t+1) represents the calculation result of the input gate at time t+1, W (i) , U (i) represents the weight parameter of the input gate, B (i) represents the bias parameter of the input gate, f (t+1) Indicates the calculation result of the forget gate at time t+1, W (f) , U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t+1) Indicates the calculation result of the output gate at time t+1, W (o) , U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t+1, W (c) , U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t+1) represents the cell update result at time t+1, c (t) Represents the cell update result at time t, and tanh() represents the activation function.
2. The node classification method based on the double-layer evolutionary dynamic graph convolutional neural network model DEDGCN according to claim 1 is characterized in that: The loss function is: Where loss represents the loss function, α and β represent and , P represents the predicted node embedding, X represents the node embedding, and W represents the weight parameter in the LSTM model.
3. An edge classification method based on a two-layer evolving dynamic graph convolutional neural network model DEDGCN, characterized in that: include: DEDGCN is used to learn the edge features with labels in the Reddit Hyperlink Network, and the probability of node embedding is calculated through a feedforward neural network and a softmax function to determine the type of edge (u,v) at time t. In the Reddit Hyperlink Network, each link relationship contains a time attribute and the positive or negative sentiment of the source post to the target post. DEDGCN is used to classify the edges of the Reddit Hyperlink Network to predict the sentiment relationship between unlinked posts. The DEDGCN is constructed in the following manner: based on the graph convolutional neural network GCN, the recurrent neural network RNN is used to perform double-layer evolution on the network parameters and node embedding of the GCN respectively, to capture the changes in the structural characteristics of the graph data, and at the same time, to ensure the temporal stability of the node embedding, to construct the loss function using the node embedding evolution results and the results generated by GCN, to correct the GCN network parameters, to form an unsupervised dynamic graph representation learning framework, and to complete the construction of the double-layer evolutionary dynamic graph convolutional neural network model DEDGCN; The GCN consists of two graph convolutional layers, which extract the structural features of nodes in the network based on the features of neighbor nodes; t =(V t ,E t ), V t represents the set of nodes at the snapshot at time t, E t Represents the set below the snapshot at time t. The propagation rules between graph convolutional layers are as follows: Among them, H t (l) It represents the result obtained after the graph data at time t passes through the lth layer of graph convolution. Represents the adjacency matrix A of the graph data at time t t The regularized form of A t ′ represents the self-connection matrix of the graph data at time t, D represents the degree matrix, and W t (l) represents the parameters of the lth graph convolutional layer at time t, and σ(·) represents the activation function; The input of GCN is the vector of nodes in the graph data at time t. After L layers of graph convolution, the output of GCN is the learned node embedding. The GCN parameters of the lth layer at time t-1 are Input into LSTM to get the GCN parameter W of the lth layer at time t t (l) , the calculation method of GCN parameter evolution is as follows: where i (t) Indicates the calculation result of the input gate at time t, U (i) represents the weight parameter of the input gate, B (i) represents the bias parameter of the input gate, f (t) Indicates the calculation result of the forget gate at time t, U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t) Indicates the calculation result of the output gate at time t, U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t, U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t) represents the cell update result at time t, c (t-1) represents the cell update result at time t-1, and tanh() represents the activation function; The nodes obtained by GCN at different times are embedded into {X t-w+1 ,X t-w+2 ,…,X t } is input into LSTM to predict the embedding of the node at the next moment. The calculation method of GCN node evolution is as follows: P t+1 =LSTM(X t ,P t ) i (t+1) =sigmoid(W (i) x (t) +U (i) P t +B (i) ) f (t+1) =sigmoid(W (f) x (t) +U (f) P t +B (f) ) o (t+1) =sigmoid(W (o) x (t) +U (o) P t +B (o) ) Where w represents the window size of LSTM, P t represents the predicted node embedding at time t, i (t+1) represents the calculation result of the input gate at time t+1, W (i) , U (i) represents the weight parameter of the input gate, B (i) represents the bias parameter of the input gate, f (t+1) Indicates the calculation result of the forget gate at time t+1, W (f) , U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t+1) Indicates the calculation result of the output gate at time t+1, W (o) , U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t+1, W (c) , U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t+1) represents the cell update result at time t+1, c (t) Represents the cell update result at time t, and tanh() represents the activation function.
4. The edge classification method based on the dual-layer evolutionary dynamic graph convolutional neural network model DEDGCN according to claim 3 is characterized in that: The loss function is: Where loss represents the loss function, α and β represent and , P represents the predicted node embedding, X represents the node embedding, and W represents the weight parameter in the LSTM model.
5. A link prediction method based on a two-layer evolving dynamic graph convolutional neural network model DEDGCN, characterized in that: include: DEDGCN is used to calculate the embedded information of nodes u and v at time t or before in UCI or AS Network. The embedded information of nodes u and v at time t or before is used to predict whether the edge (u, v) exists at time t+1. The features of nodes u and v are aggregated, and then the existence probability of the edge is obtained using a feedforward neural network. In UCI, edge (u, v, t) represents a private message sent by user u to user v at time t. DEDGCN is used to perform link prediction on UCI to capture the possible connections between users at the next moment. In AS Network, nodes represent autonomous systems and edges represent the communication relationship between autonomous systems. DEDGCN is used to perform link prediction on AS Network to predict the communication relationship of the network at the next moment. The DEDGCN is constructed in the following manner: based on the graph convolutional neural network GCN, the recurrent neural network RNN is used to perform double-layer evolution on the network parameters and node embedding of the GCN respectively, to capture the changes in the structural characteristics of the graph data, and at the same time, to ensure the temporal stability of the node embedding, to construct the loss function using the node embedding evolution results and the results generated by GCN, to correct the GCN network parameters, to form an unsupervised dynamic graph representation learning framework, and to complete the construction of the double-layer evolutionary dynamic graph convolutional neural network model DEDGCN; The GCN consists of two graph convolutional layers, which extract the structural features of nodes in the network based on the features of neighbor nodes; t =(V t ,E t ), V t represents the set of nodes at the snapshot at time t, E t Represents the set below the snapshot at time t. The propagation rules between graph convolutional layers are as follows: Among them, H t (l) It represents the result obtained after the graph data at time t passes through the lth layer of graph convolution. Represents the adjacency matrix A of the graph data at time t t The regularized form of A t ′ represents the self-connection matrix of the graph data at time t, D represents the degree matrix, and W t (l) represents the parameters of the lth graph convolutional layer at time t, and σ(·) represents the activation function; The input of GCN is the vector of nodes in the graph data at time t. After L layers of graph convolution, the output of GCN is the learned node embedding. The GCN parameters of the lth layer at time t-1 are Input into LSTM to get the GCN parameter W of the lth layer at time t t (l) , the calculation method of GCN parameter evolution is as follows: where i (t) Indicates the calculation result of the input gate at time t, U (i) represents the weight parameter of the input gate, B (i) represents the bias parameter of the input gate, f (t) Indicates the calculation result of the forget gate at time t, U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t) Indicates the calculation result of the output gate at time t, U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t, U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t) represents the cell update result at time t, c (t-1) represents the cell update result at time t-1, and tanh() represents the activation function; The nodes obtained by GCN at different times are embedded into {X t-w+1 ,X t-w+2 ,…,X t } is input into LSTM to predict the embedding of the node at the next moment. The calculation method of GCN node evolution is as follows: P t+1 =LSTM(X t ,P t ) i (t+1) =sigmoid(W (i) x (t) +U (i) P t +B (i) ) f (t+1) =sigmoid(W (f) x (t) +U (f) P t +B (f) ) o (t+1) =sigmoid(W (o) x (t) +U (o) P t +B (o) ) Where w represents the window size of LSTM, P t represents the predicted node embedding at time t, i (t+1) represents the calculation result of the input gate at time t+1, W (i) , U (i) represents the weight parameter of the input gate, B (i) represents the bias parameter of the input gate, f (t+1) Indicates the calculation result of the forget gate at time t+1, W (f) , U (f) represents the weight parameter of the forget gate, B (f) represents the bias parameter of the forget gate, o (t+1) Indicates the calculation result of the output gate at time t+1, W (o) , U (o) represents the weight parameter of the output gate, B (o) represents the bias parameter of the output gate, represents the candidate cell calculation result at time t+1, W (c) , U (c) represents the weight parameter of the candidate cell, B (c) represents the bias parameter of the candidate cell, c (t+1) represents the cell update result at time t+1, c (t) Represents the cell update result at time t, and tanh() represents the activation function.
6. The link prediction method based on the double-layer evolving dynamic graph convolutional neural network model DEDGCN according to claim 5, characterized in that: The loss function is: Where loss represents the loss function, α and β represent and , P represents the predicted node embedding, X represents the node embedding, and W represents the weight parameter in the LSTM model.
