Dynamic link prediction method and system based on time sequence heterogeneous graph attention network
Through the dynamic link prediction method based on the timing heterogeneous graph attention network, the calculation complexity and efficiency of link prediction in the dynamic heterogeneous network are solved, and efficient and accurate prediction of the timing heterogeneous network is achieved.
Patent Information
- Application Number
- CN202411043427.X
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-07-31
- Publication Date
- 2024-11-08
AI Technical Summary
When existing link prediction methods monitor and process data in real time in dynamic heterogeneous networks, the calculation complexity is high, the efficiency is insufficient, and it is difficult to effectively use sparse data for accurate prediction.
A dynamic link prediction method based on timing heterogeneous graph attention network is proposed. By obtaining timing heterogeneous graphs, learning node and path-level embeddings, using heterogeneous neighbor random walks and time encoders to capture heterogeneous information and timing Information, aggregate node neighborhood structure characteristics, calculate event attention scores, generate node time feature matrix, and make predictions.
It significantly improves the accuracy and efficiency of link prediction, can effectively process sparse data in dynamic heterogeneous networks, and realizes accurate prediction of timing heterogeneous networks.
Smart Images

Figure CN118916786A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of network link prediction, and in particular to a dynamic link prediction method and system based on a temporal heterogeneous graph attention network. Background Art
[0002] With the popularization of the Internet and the rise of various network applications, link prediction in network science has become an important research issue. Link prediction is to predict whether there is a link between two nodes in the network. The solution to this problem is of great significance for optimizing network resource allocation, social network analysis, etc. Networks in the real world are often heterogeneous, including different types of nodes and edges, and networks in the real world are usually dynamically changing and dynamically interacting. This feature makes the network highly complex and uncertain. Dynamic changes mean that information such as network topology, node attributes, and link attributes evolves with time and events, requiring real-time monitoring and adaptation. Dynamic interactions indicate that there are frequent interactions and information transfers between nodes and links in the network, resulting in the state and performance of the network changing at any time. When making predictions on dynamic network links, it is necessary to monitor the network state and topology changes in real time, consider the dynamic changes of link states in the network, and deal with challenges such as data sparsity and model generalization capabilities.
[0003] At present, traditional link prediction methods have some shortcomings. They require real-time monitoring of network status and topology changes, and the computational complexity is high, resulting in insufficient real-time performance and efficiency. The data in dynamic heterogeneous networks is usually sparse, and traditional data processing methods are difficult to effectively use these data for accurate link prediction. The attention mechanism is widely used in deep learning. By dynamically adjusting the degree of attention to the input, the expression ability and prediction performance of the model can be improved. In the link prediction problem, the introduction of the attention mechanism can help the network model better capture the correlation between nodes and improve the accuracy of link prediction. Summary of the invention
[0004] In view of the above problems, the present invention proposes a dynamic link prediction method and system based on a temporal heterogeneous graph attention network to solve the problem of heterogeneous network link prediction.
[0005] According to one aspect of the present invention, a dynamic link prediction method based on a temporal heterogeneous graph attention network is proposed, the method comprising the following steps:
[0006] Step 1: Obtain a time series heterogeneous graph of a time series heterogeneous network; the time series heterogeneous graph Where V is the set of nodes, E is the set of events, T is the set of timestamps, φ is the set of node types, and ψ is the set of edge types;
[0007] Step 2: Divide the time series heterogeneous graph into a training set and a test set according to the time sequence, and obtain the predicted time t;
[0008] Step 3: Training the link prediction model based on the training set, including:
[0009] Compute path-level embeddings for nodes in the training set And perform negative sampling to obtain negative samples of the same link type as the positive samples, and the number of positive and negative samples is the same; divide the positive samples and the corresponding negative samples into multiple batches, for each target node u in each batch:
[0010] First, we learn the node-level and path-level node information of the temporal heterogeneous graph before the prediction time t, aggregate the contextual information of the temporal heterogeneous graph, and obtain the node embedding x containing heterogeneous information. u (t);
[0011] Then, according to the top-N nearest sampling strategy, the first-order temporal neighbors of the target node are sampled in the temporal heterogeneous graph to obtain the temporal neighbor set N of the node t (u);
[0012] Then, according to the node features of the target node and the nodes in the temporal neighbor set and the time features of the events, the node time feature matrices Zs(t) and Zn(t) of the target node are obtained;
[0013] Then, the attention score of the event is calculated, and the features of the temporal neighbors are aggregated to obtain the neighborhood features of the target node;
[0014] Finally, the node features are processed and decoded to obtain the prediction results;
[0015] Step 4: Input the test set into the link prediction model trained in step 3 to perform prediction and obtain the prediction probability;
[0016] Step 5: Calculate the area under the ROC curve and the average precision value of the predicted probability corresponding to the test set;
[0017] Step 6: Iterate and repeat steps 3 to 5 until the maximum number of iterations is reached, calculate the average of multiple area values under the ROC curves and the average of multiple average precision values of multiple iterations as the final prediction result, and complete the training of the link prediction model;
[0018] Step 7: Input the time series heterogeneous network data to be predicted into the trained link prediction model for link prediction.
[0019] Furthermore, the path-level embedding in step 3 The calculation formula is:
[0020]
[0021] In the formula, LeakyReLU represents the activation function; and is a learnable parameter; represents the node level embedding of node v in the l-1th layer of the network.
[0022] Furthermore, the node information at the node level and path level of the temporal heterogeneous graph before the learning prediction time t in step 3 is aggregated to obtain the node embedding x containing heterogeneous information. u (t) include:
[0023] First, perform N random walks to obtain a path set P starting from the target node u. u ; For the path set P u Each path p i , aggregate the path-level embeddings of the nodes in the path, and obtain the path p for each path i Path-level embedding for:
[0024]
[0025] In the formula, f p is mean pooling; Represents node v i Path-level embedding of
[0026] Then, the personalized information of the target node u on each path is calculated
[0027]
[0028] In the formula, is the scaling vector, is the shift vector; Represents node u embedded at the node level of the l-1 layer network; then, all personalized information of the target node u is aggregated to obtain context information
[0029]
[0030] Where Mean represents mean aggregation; L(p) is the length of path p, e -λL(p) As a weighted scheme, shorter paths are favored;
[0031] Then, the context information Convert to node embedding x u (t):
[0032]
[0033] In the formula, and Represents a trainable parameter.
[0034] Furthermore, the step 3 described in which the node time feature matrices Zs(t) and Zn(t) of the target node are obtained according to the node features of the target node and the nodes in the temporal neighbor set and the temporal features of the event include: for each node in the temporal neighbor set, the mapping matrix M of the event is calculated. eu and Calculate the projection vector h of the two nodes corresponding to the event u (t) and Calculate the event features of two nodes on a certain event and z i (t i ); Calculate the time embedding TE(tt i ); Finally, the event features and time embedding are concatenated to obtain the time feature matrices Zs(t) and Zn(t) of the target node.
[0035] Furthermore, the mapping matrix M in step 3 eu and The calculation formula is:
[0036]
[0037]
[0038] In the formula, Projection vector representing the event type; n φ(u) and n φ(v) They represent the projection vector of node type φ(u) of node u and the projection vector of node type φ(v) of node v, respectively, both of which are trainable; I represents the identity matrix; d represents the matrix dimension;
[0039] The projection vector h u (t) and h v The calculation formula of (t) is:
[0040]
[0041]
[0042] In the formula, x u (t) and x v (t) are the node information of nodes u and v respectively;
[0043] Event Characteristics Indicates that node v0 is in event e iThe event characteristics on , the calculation formula is:
[0044]
[0045] Where LayerNorm represents the normalization layer; i represents the i-th temporal neighbor of the node; represents the projection vector of node v0; 0,i (t i ) represents event e i The edge features of
[0046] Event characteristics i (t i ) is calculated as:
[0047]
[0048] In the formula, Represents node v i The projection vector of
[0049] The temporal embedding TE(tt i ) is calculated as:
[0050]
[0051] In the formula, w1, is a learnable parameter; t i Indicates event e i Timestamp of T represents the embedding dimension;
[0052] The calculation formula of the time feature matrix Zs(t) and Zn(t) of the target node is:
[0053]
[0054] Zn(t)=[z1(t1)||TE(t-t1),…,z N (t N )||TE(t- t N)] T
[0055] In the formula, Represents the event features of node v0 on event e1; Indicates that node v0 is in event e N TE(0) represents the time embedding of the event at time 0; z1(t1) represents the event feature of the first temporal neighbor of node v0 on event e1; N (t N ) indicates that the Nth temporal neighbor of node v0 has NThe event characteristics on the time; TE(t-t1) represents the time embedding of the event at time t1; TE(tt N ) indicates t N Temporal embedding of events at moments.
[0056] Furthermore, the step 3 of calculating the attention score of the event and aggregating the features of the temporal neighbors to obtain the neighborhood features of the target node includes: projecting the temporal feature matrices Zs(t) and Zn(t) to the Q, K, V matrices, where Q is the query matrix, K is the index matrix, and V is the value matrix; calculating the attention weight αi of the target node to the event; performing weighted summation on the event features of the neighborhood of the target node to obtain the neighborhood feature y u (t); where the target node’s attention weight α for the event i The calculation formula is as follows:
[0057]
[0058] In the formula, Q i represents the i-th row of the query matrix Q, K i represents the i-th row of the index matrix K; Indicates that the node type φ(v0) has an edge type of The attention coefficient of the event is a learnable parameter; d represents the number of columns of the value matrix V; Indicates event e i The event type, φ(v0) represents the node type of node v0;
[0059] Neighborhood feature y u (t) is calculated as:
[0060]
[0061] Where Linear represents the linear projection function; V i Represents the i-th row of the value matrix V; N represents the number of temporal neighbor nodes.
[0062] Furthermore, the processing node features in step 3, decoding the features, and obtaining the prediction results include: representing the neighborhood y u (t) is combined with the source node features and passed to the fully connected neural network to obtain the output embedding of the node Use a fully connected neural network to calculate the probability of positive and negative sample events occurring at the target node Calculate the loss value, use Adam as the optimizer, and back-propagate to update the parameter set.
[0063] Furthermore, the neighborhood representation y in step 3 is u(t) is combined with the source node features and passed to the fully connected neural network to obtain the output embedding of the node The calculation formula is:
[0064]
[0065] In the formula, FFN represents fully connected neural network; represents the node embedding of node u at layer l; represents the mapping matrix, Represents the mapping vector, which are all trainable parameters; ReLU represents the activation function;
[0066] The probability of positive and negative sample events occurring at the target node The calculation formula is:
[0067]
[0068] In the formula, u and v represent the target node and the node to be predicted respectively; r γ represents the embedding of the link type to be predicted; r represents the embedding of the event type, and γ represents the event type.
[0069] Furthermore, in step 3, the loss value is calculated according to the following formula:
[0070]
[0071] Where ε represents the total set of positive and negative triplets; σ represents the logistic sigmoid function, y represents the sample label, the positive sample takes the value of 1, and the negative sample takes the value of 0; θ represents the model parameter, and λ is the parameter that controls L2 regularization.
[0072] According to another aspect of the present invention, a dynamic link prediction system based on a temporal heterogeneous graph attention network is proposed, the system comprising:
[0073] A data acquisition module is configured to acquire a temporal heterogeneous graph of a temporal heterogeneous network; the temporal heterogeneous graph Where V is a node set, E is an event set, T is a timestamp set, φ is a node type set, and ψ is an edge type set; the temporal heterogeneous graph is divided into a training set and a test set in chronological order to obtain a predicted time t;
[0074] A model training module is configured to train a link prediction model based on a training set, including: calculating a path-level embedding of nodes in the training set And perform negative sampling to obtain negative samples of the same link type as the positive samples, and the number of positive and negative samples is the same; divide the positive samples and the corresponding negative samples into multiple batches, and for each target node u in each batch: first, learn the node level and path level node information of the temporal heterogeneous graph before the prediction time t, aggregate the context information of the temporal heterogeneous graph, and obtain the node embedding x containing heterogeneous information u (t); Then, according to the top-N nearest sampling strategy, the first-order temporal neighbors of the target node are sampled in the temporal heterogeneous graph to obtain the temporal neighbor set N of the node t (u); then, according to the node features of the target node and the nodes in the temporal neighbor set and the temporal features of the event, the node temporal feature matrix Zs(t) and Zn(t) of the target node are obtained; then, the attention score of the event is calculated, and the features of the temporal neighbors are aggregated to obtain the neighborhood features of the target node; finally, the node features are processed and decoded to obtain the prediction results;
[0075] The test set is input into the trained link prediction model for prediction to obtain the prediction probability; the area under the ROC curve and the average precision value of the prediction probability corresponding to the test set are calculated;
[0076] Iteratively repeat the training step and the testing step until the maximum number of iterations is reached, and calculate the average of multiple area values under the ROC curves and the average of multiple average precision values of multiple iterations as the final prediction result to complete the training of the link prediction model;
[0077] The link prediction module is configured to input the time-series heterogeneous network data to be predicted into the trained link prediction model to perform link prediction.
[0078] The beneficial technical effects of the present invention are:
[0079] The present invention proposes a dynamic link prediction method and system based on a temporal heterogeneous graph attention network. For heterogeneity, heterogeneous neighbor random walks are used to capture heterogeneous information; for temporal information, a temporal encoder is used to encode temporal information; for new nodes, an inductive graph representation learning method is used to aggregate the structural features of the node neighborhood, thereby quickly generating node embeddings in new subgraphs. Baseline model comparison experiments were conducted with six classic frontier baseline models on four public temporal heterogeneous datasets, and ablation experiments and parameter comparison experiments were performed on the models. The results of the baseline model comparison experiments show that the present invention has significant advantages in the task of link prediction in temporal heterogeneous networks; the ablation experiments verify the effectiveness of the heterogeneous neighbor random walk module, node dynamic embedding generation module, and temporal heterogeneous graph attention layer of the present invention; the parameter comparison experiment fully analyzes the impact of the number of neighbor samples and the number of attention heads on the present invention. BRIEF DESCRIPTION OF THE DRAWINGS
[0080] The present invention can be better understood by referring to the description given below in conjunction with the accompanying drawings, which together with the following detailed description are included in this specification and form a part of this specification, and are used to further illustrate the preferred embodiments of the present invention and explain the principles and advantages of the present invention.
[0081] Figure 1 It is a framework diagram of the dynamic link prediction method based on the temporal heterogeneous graph attention network described in the present invention.
[0082] Figure 2 1 is a histogram of ablation experiment results in an embodiment of the present invention; wherein FIG. (a) corresponds to the AUC indicator result; and FIG. (b) corresponds to the AP indicator result.
[0083] Figure 3 2 is an example diagram of the impact of the number of neighbor sampling values on the AUC in an embodiment of the present invention; (a) corresponds to the Cora dataset; (b) corresponds to the DBLP-ac dataset; (c) corresponds to the DBLP-co dataset; and (d) corresponds to the TKY dataset.
[0084] Figure 4 2 is an example diagram of the impact of the value of the number of neighbor sampling on the AP in an embodiment of the present invention; (a) corresponds to the Cora dataset; (b) corresponds to the DBLP-ac dataset; (c) corresponds to the DBLP-co dataset; and (d) corresponds to the TKY dataset.
[0085] Figure 5 1 is an example diagram of the impact of the number of attention heads on AUC in an embodiment of the present invention; (a) corresponds to the Cora dataset; (b) corresponds to the DBLP-ac dataset; (c) corresponds to the DBLP-co dataset; and (d) corresponds to the TKY dataset.
[0086] Figure 6 2 is an example diagram of the impact of the number of attention heads on AP in an embodiment of the present invention; (a) corresponds to the Cora dataset; (b) corresponds to the DBLP-ac dataset; (c) corresponds to the DBLP-co dataset; and (d) corresponds to the TKY dataset. DETAILED DESCRIPTION
[0087] In order to enable those skilled in the art to better understand the scheme of the present invention, exemplary implementations or embodiments of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described implementations or embodiments are only implementations or embodiments of a part of the present invention, not all of them. Based on the implementations or embodiments of the present invention, all other implementations or embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.
[0088] The types of nodes and edges in temporal heterogeneous graphs are diverse and semantically rich, and the edges of the graph are time-related, that is, each event is timestamped when it occurs. Therefore, it is necessary to convert the temporal information into dynamic features and use temporal constraints to avoid violating the temporal causal relationship between interactions. The dynamic nature of nodes in temporal heterogeneous graphs means that new nodes will appear in the future, which do not exist during training, and many practical applications require their embeddings to be generated in a timely manner. Therefore, it is necessary to construct an inductive modeling method to generalize the optimized representation to new temporal subgraphs. For new nodes, the inductive graph representation learning method identifies the structural features of the node neighborhood by learning a trainable aggregation function, thereby quickly generating node embeddings in the new subgraph. A large number of studies have attempted to address the above challenges, but few methods can address these challenges simultaneously. Therefore, the present invention proposes a dynamic link prediction method and system based on a temporal heterogeneous graph attention network, wherein a dynamic link prediction model is proposed, which is named ATHGNN. The dynamic link prediction model mainly includes five parts: a heterogeneous neighbor random walk module, a temporal heterogeneous neighbor sampling module, a node dynamic embedding generation module, a temporal heterogeneous graph attention module, and a prediction module; for heterogeneity, a heterogeneous neighbor random walk is used to capture heterogeneous information; for temporal information, a temporal encoder is used to encode temporal information; for new nodes, an inductive graph representation learning method is used to aggregate the structural features of the node neighborhood, thereby quickly generating node embeddings in new subgraphs. And the present invention is experimented on four real temporal heterogeneous network datasets.
[0089] The embodiment of the present invention proposes a dynamic link prediction method based on a temporal heterogeneous graph attention network, the method comprising the following steps:
[0090] Step 1: Obtain a time series heterogeneous graph of a time series heterogeneous network; the time series heterogeneous graph Where V is the set of nodes, E is the set of events, T is the set of timestamps, φ is the set of node types, and ψ is the set of edge types;
[0091] Step 2: Divide the time series heterogeneous graph into a training set and a test set according to the time sequence, and obtain the predicted time t;
[0092] Step 3: Training the link prediction model based on the training set, including:
[0093] Compute path-level embeddings for nodes in the training set And perform negative sampling to obtain negative samples of the same link type as the positive samples, and the number of positive and negative samples is the same; divide the positive samples and the corresponding negative samples into multiple batches, for each target node u in each batch:
[0094] First, we learn the node-level and path-level node information of the temporal heterogeneous graph before the prediction time t, aggregate the contextual information of the temporal heterogeneous graph, and obtain the node embedding x containing heterogeneous information. u (t);
[0095] Then, according to the top-N nearest sampling strategy, the first-order temporal neighbors of the target node are sampled in the temporal heterogeneous graph to obtain the temporal neighbor set N of the node t (u);
[0096] Then, according to the node features of the target node and the nodes in the temporal neighbor set and the time features of the events, the node time feature matrices Zs(t) and Zn(t) of the target node are obtained;
[0097] Then, the attention score of the event is calculated, and the features of the temporal neighbors are aggregated to obtain the neighborhood features of the target node;
[0098] Finally, the node features are processed and decoded to obtain the prediction results;
[0099] Step 4: Input the test set into the link prediction model trained in step 3 to perform prediction and obtain the prediction probability;
[0100] Step 5: Calculate the area under the ROC curve and the average precision value of the predicted probability corresponding to the test set;
[0101] Step 6: Iterate and repeat steps 3 to 5 until the maximum number of iterations is reached, calculate the average of multiple area values under the ROC curves and the average of multiple average precision values of multiple iterations as the final prediction result, and complete the training of the link prediction model;
[0102] Step 7: Input the time series heterogeneous network data to be predicted into the trained link prediction model for link prediction.
[0103] The present invention is described in detail below.
[0104] The method starts from step 1. In step 1, a time sequence heterogeneous graph of a time sequence heterogeneous network is obtained; the time sequence heterogeneous graph Where V is the set of nodes, E is the set of events, T is the set of timestamps, φ is the set of node types, and ψ is the set of edge types.
[0105] According to an embodiment of the present invention, first, a time sequence heterogeneous graph of a time sequence heterogeneous network is obtained. The time sequence heterogeneous graph Where V is the node set, E is the edge set also known as the event set, and T is the timestamp set. Each node v in the node set has a type φ(v), and each edge e in the edge set has a type ψ(e). The sets of node types and edge types are represented by and Indicates that |T v |+|T e |>1.
[0106] Then, step 2 is executed. In step 2, the time series heterogeneous graph is divided into a training set and a test set according to the time sequence to obtain the predicted time t.
[0107] According to an embodiment of the present invention, the Divide into training set and test set, and obtain the predicted time t.
[0108] Then, step 3 is executed. In step 3, a link prediction model is trained based on the training set, including:
[0109] Compute path-level embeddings for nodes in the training set And perform negative sampling to obtain negative samples of the same link type as the positive samples, and the number of positive and negative samples is the same; divide the positive samples and the corresponding negative samples into multiple batches, for each target node u in each batch:
[0110] First, we learn the node-level and path-level node information of the temporal heterogeneous graph before the prediction time t, aggregate the contextual information of the temporal heterogeneous graph, and obtain the node embedding x containing heterogeneous information. u (t);
[0111] Then, according to the top-N nearest sampling strategy, the first-order temporal neighbors of the target node are sampled in the temporal heterogeneous graph to obtain the temporal neighbor set N of the node t (u);
[0112] Then, according to the node features of the target node and the nodes in the temporal neighbor set and the time features of the events, the node time feature matrices Zs(t) and Zn(t) of the target node are obtained;
[0113] Then, the attention score of the event is calculated, and the features of the temporal neighbors are aggregated to obtain the neighborhood features of the target node;
[0114] Finally, the node features are processed and decoded to obtain the prediction results.
[0115] According to an embodiment of the present invention, the link prediction model ATHGNN includes a heterogeneous neighbor random walk module, a temporal heterogeneous neighbor sampling module, a node dynamic embedding generation module, a temporal heterogeneous graph attention module and a prediction module.
[0116] 1) The heterogeneous neighbor random walk module is used to learn the node-level and path-level node information of the heterogeneous graph before the prediction time t, and aggregate the context information of the heterogeneous graph: the initial node embedding h vGet the path-level embedding s of a node v Then, a heterogeneous random walk is performed to obtain a path set, and the path encoder aggregates the node information in the path to obtain the path embedding Decode the path information to obtain the personalized information h of the target node in each path v|p , aggregate the personalized information of the nodes and obtain the node embedding x that integrates heterogeneous information v (t).
[0117] Specifically, there are two embeddings for a node v in a heterogeneous graph: node embedding Path Embedding h v Capture the basic information and structural information of nodes in heterogeneous graphs. For each node v, s v It is from h v It is extracted from the node, indicating the type information of the node and the single-hop or multi-hop relationship information between the nodes.
[0118]
[0119] in, and is a learnable parameter. LeakyReLU represents the activation function; represents the node level embedding of node v in the l-1th layer of the network.
[0120] In heterogeneous graphs, paths may carry different information due to heterogeneous multi-hop relationships between nodes. Therefore, a path encoder is used to fuse the node-level embedding associated with the path into the path-level embedding. Specifically, a random walk is first performed to sample a set of paths. Starting from each target node, N random walks are performed, and the paths of different lengths obtained each time can capture different ranges of structural information. Next, for each path p i , which is encoded by the path encoder to generate a path embedding Let P u represents the set of sampled paths starting from the target node u. If there is a path p i ∈P u So that p i ends at node v, then v is called the context node of u or simply the context of u. For example, given the path set for Figure 1 The target node u0 in the graph is i1, u1, u4, which carry different semantics for node u0. In the first layer, the path encoder f is used. p Get each path p i Embed
[0121]
[0122] in, Represents node v i Path-level embedding of p It is mean pooling, which takes the mean of the path-level embeddings of all nodes in the path as the path embedding. Then each path is embedded Use the activation function to decode and obtain the personalized information of the target node u on the path p
[0123]
[0124]
[0125]
[0126] in, and are learnable parameters, where is the scaling vector, is the shift vector and LeakyReLU(·) is the activation function. Represents the node u embedded at the node level of the l-1th layer network.
[0127] Then, the personalized information of the target node is aggregated to obtain the context information of node v
[0128]
[0129] Among them, Mean represents mean aggregation; L(p) is the length of path p, e -λL(p) As a weighted scheme, it prefers shorter paths. Then, node information is obtained based on context information.
[0130]
[0131] in,
[0132] 2) The temporal heterogeneous neighbor sampling module is used to sample the temporal neighbors of the target node according to the top-N nearest sampling strategy for a given prediction time t, and obtain the temporal neighbor set N of the target node t (v0).
[0133] Specifically, in order to improve the induction and generalization performance of the model, the temporal heterogeneous graph attention network does not select all neighbors from the temporal neighbors, but selects a certain number of neighbors as input. Given a node v0 and a prediction time t, N neighbors are sampled from its first-order temporal neighbors, denoted as N t (v0)={v1,...,v n}.
[0134] There are currently two neighbor sampling strategies: uniform random sampling, which randomly selects all temporal neighbors with equal probability; top-N recent sampling, which calculates the time difference between t and all events of the target node and sorts them in ascending order, and then selects temporal neighbors among the top N events. Intuitively, recent interactions better reflect the current state of the node than interactions a long time ago, and have a greater impact on future events. On the contrary, events that happened a long time ago may bring noise. Therefore, the top-N recent sampling strategy is used to sample neighbors.
[0135] In a time-heterogeneous graph, the number of different types of events varies greatly, which can easily lead to an unbalanced distribution of sampled neighbor types. In order to avoid sampling bias as much as possible, the number of samples of each event type is limited to no more than M. If the total number of event types associated with the source node is γ (γ≤|R|), then the total number of sampled neighbors N satisfies N≤γ*M.
[0136] 3) The node dynamic embedding generation module is used to obtain the node time feature matrix Zs(t) and Zn(t) of the target node based on the node features of the target node and the nodes in the temporal neighbor set and the time features of the event. eu The node features x in each event u (t) Mapping is performed to obtain the projection vector h of the node u (t), by node v i The projection vector h i (t i ) and event e i The relationship vector χ 0,i (t i )Get node v i In the event i Event features on z i (t i ). Use Bochner-type function time coding to obtain time embedding TE(t), and transform the event feature z of the node on the event i (t i ) is merged with the time embedding TE(t) to obtain the node time feature matrices Zs and Zn.
[0137] Specifically, the main task of the node dynamic embedding generation module is to map the node features in the event, obtain the projection vector of the node in each event, and then encode the time information to obtain the time embedding of the event, and merge the node projection vector with the time embedding to obtain the node time feature matrix. For different nodes, it is assumed that they have the same feature distribution and share the parameter model of the node, which does not hold in heterogeneous graphs. In addition, in the real world, there may be multiple types of edges between two nodes, and different types may correspond to different vector distributions. Therefore, when propagating information, not only the diversity of nodes but also the diversity of edges should be considered. A simple solution is to map the features of different distributions to the same semantic space through a transfer matrix. However, as the number of types increases, more parameters will be introduced in the model. In order to reduce the number of training parameters and avoid large-scale matrix multiplication calculations, the node features are projected from the node type space to the event type space by dynamically calculating the transfer matrix of the two matrices.
[0138] Given an event e = (u, v, t) and its meta-relation Define the dynamic mapping matrix as:
[0139]
[0140]
[0141] in, and n φ(u) ∈R d Represent the projection vectors of event type and node type respectively, both of which are trainable, M eu ∈R d×d ; I represents the identity matrix; d represents the matrix dimension.
[0142] The projection vector of nodes u and v in event e is:
[0143]
[0144]
[0145] where x u (t) and x v (t) are the node information of nodes u and v, respectively, and the projection vector h u (t),h v (t)∈R d .
[0146] Different events in a temporally heterogeneous graph may have different characteristics. For example, in a question-answering network, an answer interaction can be regarded as an event whose characteristics are determined by the content. Events can propagate information when aggregated. The model adds event information to node embeddings and adjusts event features to the same dimension as node embeddings through a normalization layer (such as LayerNorm). Given an event e i =(v0,v i ,t i ), the output is node v i Event characteristics on this event:
[0147]
[0148] Where i represents the i-th temporal neighbor of the target node v0, Represents node v i The projection vector, z i (t i )∈R d . χ 0,i (t i )∈R d Represents edge e i Here, χ 0,0 (t) is set to the zero vector. Represents the projection vector of node v0.
[0149] Transformer uses position encoding to model relative position relationships, thus solving the problem that the attention mechanism cannot capture the sequential relationship between entities. In the time graph, the time interval between the event occurrence time and the prediction time is mapped to d T dimensional vector to replace the position encoding. This model uses Bochner-type function time encoding:
[0150]
[0151] where {w i} is a learnable parameter; d T Represents the embedding dimension. The projection vector of the node is merged with the time embedding to obtain the node time feature matrix:
[0152]
[0153] Zn(t)=[z1(t1)||TE(t-t1),…,z N (t N )||TE(tt N )] T (15)
[0154] For event e i=(v0,v i ,t i ), Indicates that node v0 is in event e i The event characteristics on z i Represents node v i In the event i The event characteristics on , || represents the "connect" operation. Indicates that node v0 is in event e N TE(0) represents the time embedding of the event at time 0; z1(t1) represents the event feature of the first temporal neighbor of node v0 on event e1; N (t N ) indicates that the Nth temporal neighbor of node v0 has N The event characteristics on the time; TE(t-t1) represents the time embedding of the event at time t1; TE(tt N ) indicates t N Temporal embedding of events at moments.
[0155] 4) The temporal heterogeneous graph attention module is used to calculate the attention score of the event and aggregate the features of the temporal neighbors to obtain the neighborhood features of the target node. Zs and Zn are Linear projection is Q, K, V matrix, using Q, K matrix and different types of events e i Attention Weight Calculate the event e i The attention coefficient α i , get node v i The hidden feature of i V i , and then aggregate the implicit features of the temporal neighbors to obtain the neighborhood features y0(t) of the target node.
[0156] Specifically, the main work of the temporal heterogeneous graph attention module is to calculate the attention score of the event and aggregate the features of the temporal neighbors to obtain the neighborhood features of the target node. The temporal feature matrices Zs and Zn of the nodes are forwarded to three different linear projections to obtain the query matrix Q, the index matrix K, and the value matrix V:
[0157]
[0158]
[0159]
[0160] where e i Represents v0 and v i The events between and Represents the projection matrix of Q, K, V. Due to the heterogeneity of edges, projection matrices cannot be shared directly, so different types of matrices are used to distinguish different events while capturing the semantics of event types. Attention weight α i Given by:
[0161]
[0162] In the formula, Q i represents the i-th row of the query matrix Q, K i represents the i-th row of the index matrix K; Indicates that the node type φ(v0) has an edge type of The attention coefficient of the event is a learnable parameter; d represents the number of columns of the value matrix V; Indicates event e i The event type of v0, φ(v0) represents the node type of node v0. This formula reveals that v i How to pass the event i To focus on the features of v0. In addition, not all types of events contribute equally to the source node, so a learnable tensor is set Adaptively adjust the level of attention paid to different types of events.
[0163] Self-attention aggregates the features of temporal neighbors and obtains the node v i The implicit embedding of i V i , which can capture node features and topological information. The next step is to map the implicit embeddings from the event feature space back to the node feature space so that they can be fused with the node features. Using Linear for linear projection, the neighborhood features of node v0 are:
[0164]
[0165] In the formula, represents the linear projection function; V i Represents the i-th row of the value matrix V; N represents the number of temporal neighbor nodes.
[0166] 5) The prediction module is used to process node features, decode node features, and obtain prediction results. u (t) and node feature x u (t) Splice to obtain the node output vector Use the decoder to decode and output the prediction result of the link to be tested.
[0167] Specifically, to combine the neighborhood representation with the source node features, the ATHGNN model concatenates them and passes them to a feed-forward neural network, i.e., a fully connected neural network:
[0168]
[0169] In the formula, FFN represents fully connected neural network; represents the node embedding of node u at layer l; represents the mapping matrix, Represents the mapping vector, which are all trainable parameters; ReLU represents the activation function.
[0170] Multi-head attention can effectively improve model performance and stability. The model can be extended to support multi-head settings. Assume that the self-attention output comes from P attention heads, that is, y i ≡Attn i (Q, K, V), i = 1, 2, ... P. First, connect these neighborhood representations with the features of the source node:
[0171]
[0172] in, is the final output representation of node u at time t, which can be used for link prediction tasks.
[0173] The goal of the heterogeneous graph decoder is to reconstruct the heterogeneous edges of the graph based on the node representation. It does this through the function Score edge triplets, where d r Represents the dimension of edge type embedding. The node representation is calculated through the l-layer encoder, and the feedforward neural network, i.e., the fully connected neural network, is used as the scoring function. Therefore, the probability of an event (u, v, t) of type r occurring is:
[0174]
[0175] Where u and v represent the target node and the node to be predicted respectively; represents the edge type embedding to be predicted. r represents the event type embedding, and γ represents the event type.
[0176] Use negative sampling to train the model. For each observed example, construct a non-existent event as a negative sample, so the number of positive samples is the same as the number of negative samples. The cross entropy loss is optimized as:
[0177]
[0178] Where ε represents the total set of positive and negative triplets, σ represents the logistic sigmoid function, y represents the sample label, the positive sample value is 1, the negative sample value is 0, θ represents the model parameter, and λ controls the L2 regularization.
[0179] Then execute step 4. In step 4, the test set is input into the link prediction model trained in step 3 for prediction to obtain the prediction probability.
[0180] Then, step five is executed, in which the area under the ROC curve and the average precision value of the predicted probability corresponding to the test set are calculated.
[0181] According to an embodiment of the present invention, AUC is a commonly used evaluation index, which measures the ability of a classifier to distinguish between positive samples and negative samples. AP is the area under the curve of PR, which is used to determine whether the link prediction is accurate. The link prediction problem refers to predicting whether there is a connection between two nodes in a network, which can be regarded as a binary classification problem. There are two types of sample labels for the link prediction problem: positive samples and negative samples, and there are two types of prediction results: connected and unconnected. According to the classification of sample labels and prediction results, the prediction results can be divided into four groups, represented by a confusion matrix, as shown in Table 1, where rows represent prediction values and columns represent label values.
[0182] Table 1 Confusion matrix of link prediction task
[0183]
[0184] True Positives (TP): The number of positive samples that are correctly predicted to be connected.
[0185] False Positives (FP): The number of positive samples that are incorrectly predicted as having no connection.
[0186] False Negatives (FN): The number of negative samples that are incorrectly predicted to be connected.
[0187] True Negatives (TN): The number of negative samples that are correctly predicted as having no connection.
[0188] It can be concluded that TP and TN are samples with accurate predictions, while FP and FN are samples with incorrect predictions.
[0189] AUC is the area under the ROC curve. In the link prediction model, to make the ROC curve, we first need to sort the probability values of the prediction results of each sample that are connected in descending order, and then combine the probability values and labels into a table. Starting from the highest probability value in the sorting, the probability value of each sample is regarded as a threshold one by one. For each such threshold, all samples above or equal to this threshold are considered positive samples, and samples below this threshold are considered negative samples. For each threshold, two key indicators need to be calculated: the true positive rate TPR and the false positive rate FPR. The calculation formula is as follows:
[0190]
[0191]
[0192] Using TPR as the vertical axis and FPR as the horizontal axis, we can get the ROC curve. AUC is the area under the ROC curve, ranging from 0.5 to 1. The closer the value is to 1, the better the performance of the model.
[0193] AP is the area under the PR curve and is used to determine whether the link prediction is accurate.
[0194] The calculation formulas for precision and recall are as follows:
[0195]
[0196]
[0197] With Recall as the horizontal axis and Precision as the vertical axis, we can get the PR curve. AP is the area under the PR curve, ranging from 0 to 1. The closer the value is to 1, the better the performance of the model.
[0198] Then execute step six. In step six, iteratively repeat steps three to five until the maximum number of iterations is reached, and the average value of the area under the multiple ROC curves and the average value of the multiple average precision values of multiple iterations are calculated as the final prediction result to complete the training of the link prediction model.
[0199] The goal of the ATHGNN model is to capture the complex structural information of the temporal heterogeneous network, convert the temporal information into dynamic features, and generate the embedding of newly emerged nodes in a timely manner, thereby completing the link prediction on THG. The pseudo code of the dynamic link prediction method based on the temporal heterogeneous graph attention network is shown in Algorithm 1.
[0200]
[0201]
[0202]
[0203] In lines 1 and 2 of the algorithm, the positive samples and the corresponding negative samples are divided into multiple batches according to the batch size batchsize. During the training process, the model learns the representation vectors of the nodes in the training set, scores the training results, updates the parameter set according to the back propagation of the loss function, optimizes the model multiple times, and outputs the test results of the model after each epoch of training.
[0204] Lines 3 to 15 of the algorithm correspond to the heterogeneous neighbor random walk module. The main task of this module is to obtain the node embedding x containing heterogeneous information. u (t). First, the node embedding is converted into the path-level embedding of the node according to formula (1): Negative sampling is performed according to the type of the target link to obtain negative samples of the same link type as the positive sample, and the number of positive and negative samples is the same. The parameters that need to be trained here are Secondly, starting from each target node, perform N random walks to obtain a path set P with a maximum path length of L u ={p1,p2…p n Then, the path-level embeddings of the nodes in the aggregated paths are obtained according to formula (2) to obtain the path embedding of each path. According to formulas (3) to (5), the personalized information of the target node on each path is calculated The set of parameters that need to be trained in this process is Finally, the personalized information of the target node is aggregated according to formula (6) to obtain the context information According to formula (7), the context information is converted into node embedding x u (t), the set of parameters that need to be trained here is The set of all parameters of this module is denoted as θ W .
[0205] Line 16 of the algorithm corresponds to the time-heterogeneous neighbor sampling module. This module samples the first-order temporal neighbors of the target node in the time graph according to the top-N nearest sampling strategy to obtain the temporal neighbor set N of the node. t (u), there are no parameters that need to be trained in this process.
[0206] Lines 17 to 29 of the algorithm correspond to the dynamic embedding mapping module, which obtains the node time feature matrices Zs(t) and Zn(t) of the target node based on the node features of the target node and the nodes in the temporal neighbor set and the temporal features of the event. First, the mapping matrix M of the two nodes in the event is calculated according to formulas (8) to (9): eu and Secondly, the projection vector h of the two nodes in the event is calculated according to formulas (10) and (11): u (t) and Again, according to formula (12), the event features of the two nodes on this event are calculated: and z i (t i Then, the time embedding TE(tt) of the event is calculated according to formula (13): i ), the set of parameters that need to be trained in this process is Finally, according to formulas (14) and (15), the event features and time embedding are concatenated to obtain the time feature matrices Zs(t) and Zn(t) of the target node. The set of parameters that need to be trained in this module is denoted as θ D .
[0207] Lines 30 to 35 of the algorithm correspond to the time-heterogeneous graph attention layer. First, the temporal feature matrices Zs(t) and Zn(t) are projected onto the Q, K, and V matrices according to formulas (16) to (18). The set of parameters that need to be trained here is Then, the attention weight α of the target node to the event is calculated according to formula (19): i , where the parameter set to be trained is {μ}. Finally, according to formula (20), the event features of the neighborhood of the target node are weighted and summed to obtain the neighborhood feature y u (t). The set of parameters that need to be trained in this module is denoted as θ A .
[0208] Lines 36 to 39 of the algorithm correspond to the prediction module. First, the neighborhood representation is combined with the source node features according to formula (21) and passed to the fully connected neural network to obtain the output embedding of the node The set of parameters that need to be trained in this process is Then, according to formula (23), the fully connected neural network is used to calculate the probability of positive and negative sample events of the target node Line 39 of the algorithm calculates the Loss value, and the optimizer uses Adam to back-propagate and update the parameter set θ = {θ W ,θ D ,θ A ,θ P}.
[0209] Lines 41 to 46 of the algorithm input the test set into the trained link prediction model for prediction, obtain the prediction probability, and calculate the area under the ROC curve and the average precision value of the prediction probability corresponding to the test set; the iteration is repeated until the maximum number of iterations is reached, and the average value of the area under the ROC curve and the average value of the average precision values of multiple iterations are calculated as the final prediction result to complete the training of the link prediction model.
[0210] Finally, step seven is executed. In step seven, the time series heterogeneous network data to be predicted is input into the trained link prediction model for link prediction. Specific embodiment 1
[0212] As shown in Table 2, the Cora dataset, a temporal heterogeneous network dataset, is a paper citation network. The nodes in the network represent papers, and the edges represent that one paper cites another paper. The type of each node is the research field to which the paper belongs. The node types include AI (artificial intelligence), DB (database), ML (machine learning), human-computer interaction, IR (information retrieval), natural language processing, SE (software engineering), RE (reinforcement learning), bioinformatics, and neural network. The edge type is "citation" or "citation relationship". The timestamp of the edge records the year when the paper was cited.
[0213] Table 2
[0214] Dataset Number of nodes Number of edges Node Type Edge Type Timestamp Average Cora 12588 47675 10 1 39 7.57
[0215] In the paper citation network, the present invention can help understand and discover the citation patterns in the academic community. It mainly solves the following problems: identifying potential citation relationships, predicting citation relationships that do not yet exist but may appear, and helping to understand which papers may be cited; predicting cooperation opportunities between fields, identifying possible co-authorship relationships, and revealing potential academic cooperation.
[0216] When performing link prediction, the time-series heterogeneous network data to be predicted is the time-series heterogeneous graph of the current network, including edge and node information and timestamp information. Link prediction is performed using the trained link model. The edge and node information of the time-series heterogeneous graph of the current network represents the existing paper citation relationship, and the timestamp information can reflect the temporal order of citations of papers in different fields. The prediction results include: predicted edges, that is, the citation relationships in different fields that are expected to appear in the future; predicted probabilities or weights, each predicted edge is usually accompanied by a probability or weight value, indicating the possibility of this edge appearing in the future. Through these prediction results, insights into future academic development can be obtained, guiding the selection of research directions and the establishment of cooperation. Specific embodiment 2
[0218] As shown in Table 3, the DBLP-co dataset is an author collaboration network. The nodes in the network represent authors, and the edges represent collaborations between authors. The node type is the author's research field, including computer science, artificial intelligence, database, machine learning, neural network, human-computer interaction, information retrieval, natural language processing, software engineering, and bioinformatics. The edge type is a "collaboration" or "co-authorship" relationship. The timestamp records the year of the author's collaboration.
[0219] Table 3
[0220] Dataset Number of nodes Number of edges Node Type Edge Type Timestamp Average DBLP-co 28085 236894 10 1 27 16.87
[0221] In the author collaboration network, the present invention can solve the following problems: identify potential collaborations, predict collaborations between authors that have not yet occurred, and identify possible future co-authors. This helps researchers find potential collaborators and thus expand their academic networks; identify collaborations between different research fields and map out interdisciplinary collaboration trends.
[0222] When performing link prediction, the time-series heterogeneous network data to be predicted is the time-series heterogeneous graph of the current network, including edge and node information and timestamp information, etc. Link prediction is performed using a trained link model, and the model is trained using historical cooperation data. According to the current network structure and timestamp information, known author nodes and cooperation edges between them, including the types of articles co-authored by authors, and the time information of cooperation between authors, the evolution of cooperation relationships over time is analyzed. The prediction results include: predicting missing edges, that is, new cooperation relationships that may appear, indicating which cooperation relationships between authors may be formed in the future; calculating probabilities or scores for predicted edges, and providing a probability value or score for possible cooperation relationships, indicating the possibility of the relationship being formed within a specific time period. Using the present invention, researchers can better plan cooperation strategies and formulate research directions. Specific embodiment three
[0224] As shown in Table 4, the time series heterogeneous network dataset-DBLP-ac is also an academic network. In the embodiment, data from May 2014 are selected. The network contains two types of nodes: authors (A) and conferences (C). The edge types represent the cooperation relationship between authors and the relationship between authors and conferences in publishing papers. The types of the dataset are extremely uneven, and the number of authors is 1624 times the number of conferences. The timestamp of each edge records the year of interaction between nodes.
[0225] Table 4
[0226] Dataset Number of nodes Number of edges Node Type Edge Type Timestamp Average DBLP-ac 164174 845485 2 2 46 10.30
[0227] In the DBLP-ac-type academic network, the present invention can solve the following problems: predicting author collaboration networks, by analyzing the collaboration history between authors, predicting collaborations that have not yet been established, which helps identify potential collaborators and promote scientific research collaboration; predicting the trend of paper publication, predicting which authors may publish papers at specific conferences in the future, which helps understand the hot spots and development trends in the academic field; predicting new connections between authors and conferences, identifying which authors may choose new conferences as a publication channel, and helping conference organizers predict potential participants and research topics.
[0228] When performing link prediction, the time-series heterogeneous network data to be predicted is the time-series heterogeneous graph of the current network, including edge and node information and timestamp information. Link prediction is performed using the trained link model. The known data information includes: the current network structure, known author nodes and the cooperation edges between them (for example, which authors have co-authored papers), known publication edges between authors and conferences (for example, which authors have published papers at which conferences); timestamp information, time information about cooperation and publication relationships, so as to understand how they evolve over time; time series of historical data, including previous cooperation records and publication records. The prediction results include: prediction of new cooperation relationships, predicting which authors may establish new cooperation relationships, and giving possible scores or probabilities to indicate the possibility of cooperation; potential publication relationships, identifying possible new publication relationships between authors and conferences, that is, which authors may choose to publish papers at specific conferences in the future; predicted edge sets, generating a possible edge set, representing the predicted author cooperation relationships and new publication relationships between authors and conferences. Through these prediction results, researchers and conference organizers can optimize cooperation strategies, thereby promoting academic development and the transformation of scientific research results. Specific embodiment 4
[0230] As shown in Table 5, the Time Series Heterogeneous Network Dataset-TKY is a user check-in graph dataset provided in the THINE model proposed in the literature [1]. This dataset records the check-in behavior of users in Tokyo. The network contains two types of nodes: users (U) and places (P), and the edge type indicates that the user has checked in at a certain place. The type of the dataset is also unbalanced, and the number of users is 27 times the number of places. The timestamp of the edge indicates the time of check-in.
[0231] Table 5
[0232] Dataset Number of nodes Number of edges Node Type Edge Type Timestamp Average TKY 64151 573703 2 1 555437 17.89
[0233] In a time-series heterogeneous network of the same type as TKY, the present invention can predict the user's check-in behavior and, based on the user's historical check-in records, predict the location where the user may check in in the future, which helps to understand the user's behavior patterns and preferences.
[0234] When performing link prediction, the time-series heterogeneous network data to be predicted is the time-series heterogeneous graph of the current network, including edge and node information and timestamp information, etc. Link prediction is performed using the trained link model. The known information includes: the current network structure, known user nodes and the check-in edges between them and location nodes (for example, which users checked in at which locations at what time); timestamp information, recording the time of each check-in, so as to analyze the time pattern of user check-in behavior; historical check-in records, the user's past check-in behavior, to form a training set for the model. The prediction results include: prediction of potential check-in locations, generating a list of possible check-in locations for each user, indicating the locations and probabilities where the user may check in in the future; time prediction of check-in behavior, predicting the locations where the user may check in within a specific time period, which helps to optimize service provision; prediction of new connections between users and locations: identifying new edges that may be established between users and locations, that is, locations where users may check in in the future. The present invention can help researchers better understand user behavior, optimize marketing strategies, and improve user experience.
[0235] The technical effect of the present invention is further verified through experiments.
[0236] In order to verify the effectiveness of the present invention, six popular graph representation learning methods are selected for comparison. These methods can be divided into four categories: embedding method for static homogeneous graphs: GAT; embedding method for static heterogeneous graphs: Metapath2vec; embedding method for dynamic homogeneous graphs: TDGNN; methods for dynamic heterogeneous graphs: DyHATR, STHE and THGAT. The specific methods are introduced as follows:
[0237] GAT: A method for learning node embeddings in static homogeneous networks. This method learns the relationship weights between nodes through attention mechanism and multi-head attention mechanism, weights the features of neighbor nodes, obtains node representation, and thus achieves link prediction tasks.
[0238] Metapath2vec: A method for learning node embeddings in static heterogeneous networks. It defines different types of metapaths to describe the relationships between nodes, and then uses these metapaths to capture the relationships between different types of nodes and generate node embeddings to achieve link prediction tasks.
[0239] TDGNN: A method for link prediction in dynamic homogeneous networks that combines graph neural networks and temporal dependencies. TDGNN uses graph neural networks to learn node representations and combines temporal information to capture temporal dependencies between nodes, which can handle link prediction tasks in continuous time environments.
[0240] DyHATR: A method for link prediction in dynamic heterogeneous networks. It uses a hierarchical attention mechanism to model different types of nodes and edges in dynamic networks and combines temporal RNN to process time series information to complete link prediction tasks in dynamic heterogeneous networks.
[0241] STHE: It is a method for link prediction in continuous-time heterogeneous networks. It combines implicit edge type information and time information by simplifying the temporal heterogeneous network, applies block technology to aggregate block features of long sequences, and uses Transformer to encode block features to obtain node representation, thereby achieving link prediction on temporal heterogeneous graphs.
[0242] THGAT: A method for representation learning of dynamic heterogeneous graphs, combining the dynamics of heterogeneous graphs with neighborhood type modeling. By taking advantage of the heterogeneous nature of graphs and considering the different neighborhood types of nodes, the representation learning of dynamic heterogeneous graphs is achieved by dynamically adjusting the graph structure and neighborhood type modeling.
[0243] Experiment 1: Baseline model comparison test: Because some baseline methods cannot process heterogeneous information or timing information, the timing heterogeneous network data is processed during the experiment. When running GAT, the timing information and heterogeneous information in the data are removed, and the data is degraded to static homogeneous network data; when running Metapath2Vec, the timing information in the data is removed, and the data is degraded to static heterogeneous network data; when running TDGNN, the heterogeneous information in the data is removed, and the data is degraded to dynamic homogeneous network data.
[0244] In the experiment, the dataset is divided into training set and test set in chronological order with a ratio of 8:2. When performing negative sampling, negative samples with the same edge type as the positive samples are sampled, and the number of positive and negative samples is the same. The experimental results are shown in Tables 6 and 7.
[0245] Table 6 AUC results of ATHGNN model compared with baseline model (%)
[0246]
[0247] Table 7 AP results of ATHGNN model compared with baseline model (%)
[0248]
[0249] Experiment 1 Analysis: From Tables 2 and 3, it can be seen that ATHGNN has significant advantages in the AUC and AP indicators of the four data sets. GAT for static homogeneous networks cannot process heterogeneous information and temporal information, so GAT performs poorly on the two indicators overall. Although Metapath2Vec for static heterogeneous networks can process heterogeneous information, because there is no meta-path input, heterogeneous information is not well utilized, so the overall effect is not as good as GAT. Although TDGNN for dynamic homogeneous networks can process temporal information, TDGNN mainly focuses on the short-term temporal dependencies between nodes and has limited modeling capabilities for long-term temporal dependencies. Therefore, TDGNN's performance in the data set is unstable. It performs better than GAT on the Cora and DBLP-ac data sets, but it performs worse than GAT on DBLP-co and TKY. DyHATR uses RNN to process temporal information. Even if variant structures such as LSTM or GRU are used, DyHATR still has limited processing capabilities for long-term memory, which affects its processing of temporal information. Therefore, DyHATR's performance is not optimal. STHE simplifies the network data when processing time-series heterogeneous network data. The simplification operation causes the loss of node type information of the network, so STHE performs poorly. Although THGAT models the neighborhood type, it does not fully capture the changes of nodes and edges in the dynamic graph, and its modeling ability is limited when processing dynamic heterogeneous graphs, so THGAT's performance is not optimal. The ATHGNN model proposed in the present invention has better results than all baseline methods on three data sets in terms of the evaluation indicator AUC, and is always better than all baseline methods in terms of the evaluation indicator AP, which shows that ATHGNN effectively learns the heterogeneous information and timing information of the time-series heterogeneous network to represent the link to be tested.
[0250] Experiment 2 Ablation experiment: In order to verify the effectiveness of each module in ATHGNN, the present invention sets up a comparative experiment of three models: no heterogeneous neighbor random walk, no dynamic embedding, and no temporal heterogeneous graph attention layer. Specifically, ATHGNN removes the heterogeneous neighbor random walk module as a variant of the non-heterogeneous neighbor random walk, denoted as ATHGNN -struct ; ATHGNN removes the node dynamic embedding generation module as a variant of nodeless dynamic embedding, denoted as ATHGNN -time ; ATHGNN removes the event attention weights and is a variant of the non-temporal heterogeneous graph attention layer, denoted as ATHGNN -type The experimental results are shown in Table 8, Table 9 and Figure 2 shown.
[0251] Table 8 Ablation experiment AUC results (%)
[0252]
[0253] Table 9 Ablation experiment AP results (%)
[0254]
[0255] Experiment 2 Analysis: Experimental results show that the complete ATHGNN model outperforms other variant models. The heterogeneous information learned by heterogeneous neighbor random walks is helpful for link prediction tasks in temporal heterogeneous graphs. Among them, the experimental results of the variant without dynamic embedding mapping are the most different from those of the complete ATHGNN, which shows that temporal embedding plays an important role in temporal heterogeneous graph representation learning and the dynamic embedding mapping module is effective. Setting different attention weights for different events helps to learn heterogeneous semantic information, so the attention layer of the temporal heterogeneous graph is meaningful.
[0256] Experiment 3 Parameter comparison experiment: In order to study the robustness of ATHGNN and find the most suitable hyperparameters, this paper analyzes the impact of the number of neighbor samples N and the number of attention heads H on four data sets. For the sake of fairness, the number of neighbor samples is selected from 4, 6, 8, and 10, and the number of attention heads is selected from 1, 2, 4, and 8. The remaining parameters are the same as the experimental settings of ATHGNN in the comparative experiment. The experimental results are shown in Figure 2. Figure 3-6 shown.
[0257] Experiment 3 analysis: On the one hand, according to Figure 3 and 4 It can be concluded that the scores of AUC and AP improve with the increase of the number of neighbor samples, but show a downward trend on the DBLP-ac dataset, which may be due to the dense connections between nodes. Sampling more neighbors may introduce more noise, resulting in smooth node representation. On the other hand, Figure 5 and 6 It shows that the number of attention heads affects the performance of the model. Multi-head attention helps to obtain representations of different aspects from different subspaces, thereby enhancing the expressiveness.
[0258] Another embodiment of the present invention provides a dynamic link prediction system based on a temporal heterogeneous graph attention network, the system comprising:
[0259] A data acquisition module is configured to acquire a temporal heterogeneous graph of a temporal heterogeneous network; the temporal heterogeneous graph Where V is a node set, E is an event set, T is a timestamp set, φ is a node type set, and ψ is an edge type set; the temporal heterogeneous graph is divided into a training set and a test set in chronological order to obtain a predicted time t;
[0260] A model training module is configured to train a link prediction model based on a training set, including: calculating a path-level embedding of nodes in the training set And perform negative sampling to obtain negative samples of the same link type as the positive samples, and the number of positive and negative samples is the same; divide the positive samples and the corresponding negative samples into multiple batches, and for each target node u in each batch: first, learn the node level and path level node information of the temporal heterogeneous graph before the prediction time t, aggregate the context information of the temporal heterogeneous graph, and obtain the node embedding x containing heterogeneous information u (t); Then, according to the top-N nearest sampling strategy, the first-order temporal neighbors of the target node are sampled in the temporal heterogeneous graph to obtain the temporal neighbor set N of the node t (u); then, according to the node features of the target node and the nodes in the temporal neighbor set and the temporal features of the event, the node temporal feature matrix Zs(t) and Zn(t) of the target node are obtained; then, the attention score of the event is calculated, and the features of the temporal neighbors are aggregated to obtain the neighborhood features of the target node; finally, the node features are processed and decoded to obtain the prediction results;
[0261] The test set is input into the trained link prediction model for prediction to obtain the prediction probability; the area under the ROC curve and the average precision value of the prediction probability corresponding to the test set are calculated;
[0262] Iteratively repeat the training step and the testing step until the maximum number of iterations is reached, and calculate the average of multiple area values under the ROC curves and the average of multiple average precision values of multiple iterations as the final prediction result to complete the training of the link prediction model;
[0263] The link prediction module is configured to input the time-series heterogeneous network data to be predicted into the trained link prediction model to perform link prediction.
[0264] The function of a dynamic link prediction system based on a temporal heterogeneous graph attention network described in this embodiment can be explained by the aforementioned dynamic link prediction method based on a temporal heterogeneous graph attention network. Therefore, for the parts not described in detail in this embodiment, please refer to the above method embodiments and will not be repeated here.
[0265] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments are contemplated within the scope of the invention thus described. The disclosure of the present invention is intended to be illustrative rather than restrictive of the scope of the invention, which is defined by the appended claims.
[0266] The documents cited in the present invention are as follows:
[0267] [1]Yang,D,et al."Modeling User Activity Preference by Leveraging UserSpatial Temporal Characteristics in LBSNs."IEEE Transactions onSystemsMan&Cybernetics Systems 45.1(2014):129-142.
Claims
1. A dynamic link prediction method based on temporal heterogeneous graph attention network, characterized in that: The following steps are involved: Step 1: Obtain a time-series heterogeneous graph of a time-series heterogeneous network; The time series heterogeneous graph Where V is the set of nodes, E is the set of events, T is the set of timestamps, φ is the set of node types, and ψ is the set of edge types; Step 2: Divide the time series heterogeneous graph into a training set and a test set according to the time sequence, and obtain the predicted time t; Step 3: Training the link prediction model based on the training set, including: For each event in the training set, calculate the path-level embedding of the node And perform negative sampling to obtain negative samples of the same link type as the positive samples, and the number of positive and negative samples is the same; divide the positive samples and the corresponding negative samples into multiple batches, for each target node u in each batch: First, we learn the node-level and path-level node information of the temporal heterogeneous graph before the prediction time t, aggregate the contextual information of the temporal heterogeneous graph, and obtain the node embedding x containing heterogeneous information. u (t); Then, according to the top-N nearest sampling strategy, the first-order temporal neighbors of the target node are sampled in the temporal heterogeneous graph to obtain the temporal neighbor set N of the node t (u); Then, according to the node features of the target node and the nodes in the temporal neighbor set and the time features of the events, the node time feature matrices Zs(t) and Zn(t) of the target node are obtained; Then, the attention score of the event is calculated, and the features of the temporal neighbors are aggregated to obtain the neighborhood features of the target node; Finally, the node features are processed and decoded to obtain the prediction results; Step 4: Input the test set into the link prediction model trained in step 3 to perform prediction and obtain the prediction probability; Step 5: Calculate the area under the ROC curve and the average precision value of the predicted probability corresponding to the test set; Step 6: Iterate and repeat steps 3 to 5 until the maximum number of iterations is reached, calculate the average of multiple area values under the ROC curves and the average of multiple average precision values of multiple iterations as the final prediction result, and complete the training of the link prediction model; Step 7: Input the time series heterogeneous network data to be predicted into the trained link prediction model for link prediction.
2. According to claim 1, a dynamic link prediction method based on a temporal heterogeneous graph attention network is characterized in that: The path-level embedding described in step 3 The calculation formula is: In the formula, LeakyReLU represents the activation function; and is a learnable parameter; represents the node level embedding of node v in the l-1th layer of the network.
3. According to claim 2, a dynamic link prediction method based on a temporal heterogeneous graph attention network is characterized in that: In step 3, we learn the node-level and path-level node information of the temporal heterogeneous graph before the prediction time t, aggregate the context information of the heterogeneous graph, and obtain the node embedding x containing heterogeneous information. u (t) include: First, perform multiple random walks to obtain a path set P starting from the target node u. u ; For the path set P u Each path p i , aggregate the path-level embeddings of the nodes in the path, and obtain the path p for each path i Path-level embedding for: In the formula, f p is mean pooling; Represents node v i Path-level embedding of Then, the personalized information of the target node u on each path is calculated In the formula, is the scaling vector, is the shift vector; represents the node u embedded at the node level of the l-1 layer network; Then, all personalized information of the target node u is aggregated to obtain the context information Where Mean represents mean aggregation; L(p) is the length of path p, e -λL(p) As a weighted scheme, shorter paths are favored; Then, the context information Convert to node embedding x u (t): In the formula, and Represents a trainable parameter.
4. According to claim 3, a dynamic link prediction method based on a temporal heterogeneous graph attention network is characterized in that: The step 3 described in which the node time feature matrices Zs(t) and Zn(t) of the target node are obtained according to the node features of the target node and the nodes in the temporal neighbor set and the temporal features of the event include: for each node in the temporal neighbor set, the event mapping matrix M is calculated. eu and Calculate the projection vector h of the two nodes corresponding to the event u (t) and Calculate the event features of two nodes on a certain event and z i (t i ); Calculate the time embedding TE(tt i ); Finally, the event features and time embedding are concatenated to obtain the time feature matrices Zs(t) and Zn(t) of the target node.
5. According to claim 4, a dynamic link prediction method based on a temporal heterogeneous graph attention network is characterized in that: The mapping matrix M described in step 3 eu and The calculation formula is: In the formula, Projection vector representing the event type; n φ(u) and n φ(v) They represent the projection vector of node type φ(u) of node u and the projection vector of node type φ(v) of node v, respectively, both of which are trainable; I represents the identity matrix; d represents the matrix dimension; The projection vector h u (t) and h v The calculation formula of (t) is: In the formula, x u (t) and x v (t) are the node information of nodes u and v respectively; Event Characteristics Indicates that node v0 is in event e i The event characteristics on , the calculation formula is: Where LayerNorm represents the normalization layer; i represents the i-th temporal neighbor of the node; represents the projection vector of node v0; 0,i (t i ) represents event e i The edge features of Event characteristics i (t i ) is calculated as: In the formula, Represents node v i The projection vector of The temporal embedding TE(tt i ) is calculated as: In the formula, w1, is a learnable parameter; t i Indicates event e i Timestamp of T represents the embedding dimension; The calculation formula of the time feature matrix Zs(t) and Zn(t) of the target node is: Zn(t)=[z1(t1)||THE(t-t1),…,z N (t N )||TE(tt N )] T In the formula, Represents the event features of node v0 on event e1; Indicates that node v0 is in event e N The characteristics of the events above; TE(0) represents the temporal embedding of the event at time 0; z1(t1) represents the event feature of the first temporal neighbor of node v0 at event e1; N (t N ) indicates that the Nth temporal neighbor of node v0 has N The characteristics of the events above; TE(t-t1) represents the time embedding of the event at time t1; TE(tt N ) indicates t N Temporal embedding of events at moments.
6. According to claim 5, a dynamic link prediction method based on a temporal heterogeneous graph attention network is characterized in that: The calculation of the attention score of the event in step 3 and the aggregation of the features of the temporal neighbors to obtain the neighborhood features of the target node include: projecting the temporal feature matrices Zs(t) and Zn(t) to the Q, K, V matrices, where Q is the query matrix, K is the index matrix, and V is the value matrix; calculating the attention weight α of the target node to the event i ; Perform weighted summation on the event features of the target node’s neighborhood to obtain the neighborhood feature y u (t); where the target node’s attention weight α for the event i The calculation formula is as follows: In the formula, Q i represents the i-th row of the query matrix Q, K i represents the i-th row of the index matrix K; Indicates that the node type φ(v0) has an edge type of The attention coefficient of the event is a learnable parameter; d represents the number of columns of the value matrix V; Indicates event e i The event type, φ(v0) represents the node type of node v0; Neighborhood feature y u (t) is calculated as: In the formula, represents the linear projection function; V i Represents the i-th row of the value matrix V; N represents the number of temporal neighbor nodes.
7. According to claim 6, a dynamic link prediction method based on a temporal heterogeneous graph attention network is characterized in that: The processing node features described in step 3, decoding the features, and obtaining the prediction results include: representing the neighborhood y u (t) is combined with the source node features and passed to the fully connected neural network to obtain the output embedding of the node Use a fully connected neural network to calculate the probability of positive and negative sample events occurring at the target node Calculate the loss value, use Adam as the optimizer, and back-propagate to update the parameter set.
8. According to claim 7, a dynamic link prediction method based on a temporal heterogeneous graph attention network is characterized in that: As described in step 3, the neighborhood is represented by y u (t) is combined with the source node features and passed to the fully connected neural network to obtain the output embedding of the node The calculation formula is: In the formula, FFN represents fully connected neural network; represents the node embedding of node u at layer l; represents the mapping matrix, Represents the mapping vector, which are all trainable parameters; ReLU represents the activation function; The probability of positive and negative sample events occurring at the target node The calculation formula is: In the formula, u and v represent the target node and the node to be predicted respectively; r γ represents the embedding of the link type to be predicted; r represents the embedding of the event type, and γ represents the event type.
9. A dynamic link prediction method based on a temporal heterogeneous graph attention network according to claim 8, characterized in that: In step 3, the loss value is calculated according to the following formula: Where ε represents the total set of positive and negative triplets; σ represents the logistic sigmoid function, y represents the sample label, the positive sample takes the value of 1, and the negative sample takes the value of 0; θ represents the model parameter, and λ is the parameter that controls L2 regularization.
10. A dynamic link prediction system based on temporal heterogeneous graph attention network, characterized in that: include: A data acquisition module configured to acquire a temporal heterogeneous graph of a temporal heterogeneous network; The time series heterogeneous graph Where V is a node set, E is an event set, T is a timestamp set, φ is a node type set, and ψ is an edge type set; the time series heterogeneous graph is divided into a training set and a test set in chronological order to obtain a predicted time t; A model training module is configured to train a link prediction model based on a training set, including: calculating a path-level embedding of nodes in the training set And perform negative sampling to obtain negative samples of the same link type as the positive samples, and the number of positive and negative samples is the same; divide the positive samples and the corresponding negative samples into multiple batches, and for each target node u in each batch: first, learn the node level and path level node information of the temporal heterogeneous graph before the prediction time t, aggregate the context information of the temporal heterogeneous graph, and obtain the node embedding x containing heterogeneous information u (t); Then, according to the top-N nearest sampling strategy, the first-order temporal neighbors of the target node are sampled in the temporal heterogeneous graph to obtain the temporal neighbor set N of the node t (t); then, according to the node features of the target node and the nodes in the temporal neighbor set and the temporal features of the event, the node temporal feature matrix Zs(t) and Zn(t) of the target node are obtained; then, the attention score of the event is calculated, and the features of the temporal neighbors are aggregated to obtain the neighborhood features of the target node; finally, the node features are processed and decoded to obtain the prediction results; The test set is input into the trained link prediction model for prediction to obtain the prediction probability; the area under the ROC curve and the average precision value of the prediction probability corresponding to the test set are calculated; Iteratively repeat the training step and the testing step until the maximum number of iterations is reached, and calculate the average of multiple area values under the ROC curves and the average of multiple average precision values of multiple iterations as the final prediction result to complete the training of the link prediction model; The link prediction module is configured to input the time-series heterogeneous network data to be predicted into the trained link prediction model to perform link prediction.
Citation Information
Cited By
Method and device for reconstructing tracking link of digital service network
CN119892710A
Target detection tracking method combining YOLOv8 detection algorithm and KCF tracking algorithm
CN120147608A
A target detection and tracking method combining a YOLOv8 detection algorithm and a KCF tracking algorithm
CN120147608B
Computing task scheduling method and system based on time sequence diagram network resource state prediction
CN120492131A
Analysis method and system based on large model and heterogeneous graph neural network
CN122475949A
