A static heterogeneous network link prediction method and system based on a graph attention network

Abstract

The application discloses a static heterogeneous network link prediction method and system based on a graph attention network, and relates to the technical field of network link prediction.The technical points of the application include: obtaining a heterogeneous network training dataset and a test dataset, wherein the dataset comprises a heterogeneous graph; performing random walk on the heterogeneous graph with the edge type being r by using a preset meta path to generate a path set, and obtaining a sample batch from the path set; training a link prediction model based on the training dataset and the test dataset; and inputting to-be-predicted heterogeneous network data into the trained link prediction model to perform link prediction.The application has significant advantages in the task of static heterogeneous network link prediction.

Description

Technical Field

[0001] This invention relates to the field of network link prediction technology, specifically to a static heterogeneous network link prediction method and system based on graph attention networks. Background Technology

[0002] With the popularization of the Internet and the rise of various network applications, network science has become an important research field. In this field, link prediction is a crucial problem—predicting whether a link exists between two nodes in a network. Solving this problem is of great significance for optimizing network resource allocation and social network analysis.

[0003] Real-world networks are often heterogeneous, containing different types of nodes and edges. Examples include social networks, biological protein networks, and computer networks, which contain various types of nodes, attributes, and relationships. Therefore, when performing link prediction on real-world heterogeneous networks, it is necessary to consider the characteristics of different types of nodes and links, as well as the complex relationships between them, and to fully explore network structure and attribute information.

[0004] Traditional link prediction methods have several shortcomings. They often lack comprehensive consideration of network structure, node attributes, and link attributes, resulting in prediction results limited by only a portion of the information. Heterogeneous networks contain various types of nodes and edges. Link prediction on heterogeneous networks presents multiple challenges, as each node pair can have various types of relationships. It is crucial to leverage the strengths of different relationships and learn a unified embedding. In link prediction methods for static heterogeneous networks, graph attention network (GNN)-based methods have shown good performance. However, in GNN-based methods, a small number of GNN layers cannot capture high-order features of the network; as the number of GNN layers increases, training complexity rises, leading to oversmoothing and performance degradation. While random walk-based methods can flexibly sample neighbors, the complex neighbor and relationship types in heterogeneous graphs can easily cause sampling imbalance. Furthermore, heterogeneous networks contain complex and diverse relationships. Extracting effective information from these relationships presents a challenge. Due to oversmoothing and the vanishing gradient problem, graph neural networks (GNNs) are difficult to deepen significantly. Reasonably increasing the number of GNN layers and capturing both high-order and low-order structural information from heterogeneous networks with complex structures pose significant challenges to link prediction in heterogeneous networks. Attention mechanisms are widely used in deep learning; by dynamically adjusting the degree of attention given to inputs, the expressive power and predictive performance of the model can be improved. In the link prediction problem, introducing attention mechanisms can help the network model better capture the correlations between nodes, thereby improving the accuracy of link prediction. Summary of the Invention

[0005] In view of the above problems, this invention proposes a static heterogeneous network link prediction method and system based on graph attention network to solve the link prediction problem in heterogeneous networks.

[0006] According to one aspect of the present invention, a static heterogeneous network link prediction method based on graph attention networks is proposed, the method comprising the following steps:

[0007] Step 1: Obtain the heterogeneous network training dataset and test dataset. The dataset includes a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, where V is the set of nodes, E is the set of edges, φ is the set of node types, and ψ is the set of edge types.

[0008] Step 2: Use the preset meta-path to perform a random walk on the heterogeneous graph with edge type r to generate a path set, and obtain sample batches from the path set.

[0009] Step 3: Train the link prediction model based on the sample batches corresponding to the training dataset. For each training sample batch, perform the following steps:

[0010] First, the basic embedding of the target node is obtained by generating shared embeddings and aggregating the edge embeddings of neighbors;

[0011] Then, based on the node sequence generated by the random walk, skip-gram is performed on the node sequence to learn the node structure embedding, so as to obtain the semantic relationships and structural features between various types of nodes in the heterogeneous graph;

[0012] Then, edge type information in the heterogeneous graph is learned through a graph attention network layer to obtain an output vector that integrates structural information and edge type information.

[0013] Then, based on the output vector of the target node, the existence of an edge of type r between nodes is predicted to obtain the prediction probability;

[0014] Then, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters;

[0015] Step 4: Input the batches of samples corresponding to the test dataset into the link prediction model trained in Step 3 for prediction, and obtain the prediction probability for each batch;

[0016] Step 5: Calculate the area under the ROC curve and the average reciprocal ranking value of the predicted probabilities corresponding to the test dataset;

[0017] Step 6: Iterate and repeat steps 3 to 5 until the maximum number of iterations is reached. Calculate the average of the area under multiple ROC curves and the average of multiple average inverse ranking values ​​as the final prediction result, and complete the training of the link prediction model.

[0018] Step 7: Input the heterogeneous network data to be predicted into the trained link prediction model for link prediction.

[0019] Furthermore, the step two, which involves using preset meta-paths to perform random walks on the heterogeneous graph with edge type r to generate a path set, includes: dividing the heterogeneous graph G into multiple layers according to edge type, performing random walks on each layer according to preset meta-paths, and generating a path set P. r .

[0020] Furthermore, step three, which describes obtaining the basic embedding of the target node by generating shared embeddings and aggregating the edge embeddings of neighbors, includes:

[0021] First, use the mean aggregator to aggregate the target node v i The edge embeddings of k-order neighbors on a layer of type r are aggregated using mean value to obtain the edge embeddings of that edge type.

[0022]

[0023] In the formula, σ is the activation function. It is a mapping matrix; mean represents mean aggregation, N i,r It is node v i The set of neighbors on edges of type r; Represents node v i Neighbor node v on the layer with edge type r j k-1 order edge embedding;

[0024] Then, combine the edge embeddings of all edge types u i,r Get the target node v i edge feature matrix U i ;

[0025] Then, the side feature matrix U is computed using a self-attention mechanism. i The coefficient a of the linear combination of vectors on edge type r i,r :

[0026]

[0027] In the formula, softmax is the activation function, and w r and W r These are the trainable parameters on the edges of type r, and the superscript T indicates the transpose of the vector or matrix;

[0028] Then, based on coefficient a i,r Edge embedding matrix U i Perform a weighted summation and integrate the shared embedded b i Obtain target node v iThe basic embedding z on each edge type i,r :

[0029]

[0030] In the formula, b i Represents node v i The shared embedding is randomly initialized by the model; ξ r λ is a hyperparameter representing the importance of edge embeddings to the overall embedding; p Indicates a i,r The p-th element; u i,p Represents node v i The edge embeddings of type p are defined as edge embeddings; R represents the total number of edge types.

[0031] Furthermore, step three, which involves performing skip-gram on the node sequence generated by the random walk to learn the node structure embedding, includes:

[0032] First, the negative sampling optimization objective function E1 is calculated:

[0033]

[0034] In the formula, σ is the sigmoid activation function; L is the number of negative samples corresponding to the positive training samples; z i,r z represents the basic embedding of node i on edge type r. j,r This represents the basic embedding of node j on edge type r; Indicates from node v j The corresponding node set V t The noise distribution P defined above t (v) are randomly selected from v. k The function; v k From node v j The corresponding node set V t The noise distribution P defined above t (v) were randomly selected;

[0035] Then, concatenate the target node v. i The basic embedding z of each edge type i,r Obtain node structure embedding

[0036]

[0037] In the formula, ∥ represents the splicing operation; R represents the total number of edge types.

[0038] Furthermore, the step three, which involves learning edge type information in heterogeneous graphs through graph attention network layers, includes:

[0039] First, attention scores are computed using edge-type embeddings and node-structure embeddings.

[0040]

[0041] In the formula, LeakyReLU represents the activation function; α represents the learnable mapping vector; r ψ(<i,j>) Represents edge type characteristics; h j The structure embedding of node j is indicated by h. k Represents the structural embedding of node k; ψ(<i,j> ) represents the type of the edge between node i and node j; W and W r It is a learnable mapping matrix used to map node embeddings and edge type embeddings; N i Represents node v i The set of neighbors;

[0042] Then, in attention scores Add residual connections to obtain the overall attention coefficient.

[0043]

[0044] In the formula, the hyperparameter β∈[0,1] is the scaling factor; Represents node v i For node v j Attention score in the l-th layer of the network;

[0045] Then, based on the comprehensive attention coefficient Aggregate the l-th layer network:

[0046]

[0047] In the formula, Represents node v i The structural embedding of the network in layer l, W (l) This represents the trainable mapping matrix of the l-th layer network;

[0048] Finally, structural embedding Perform L2 normalization to obtain the output embedding.

[0049] Furthermore, in step three, the prediction of whether there is an edge of type r between nodes is made according to the output vector of the target node using the following formula:

[0050]

[0051] In the formula, sigmoid is the activation function; R r It is a learnable square matrix of edge type r; o uand o v These represent the output embeddings of nodes u and v, respectively; Prob r This represents the probability that there is an edge of type r between nodes u and v.

[0052] Furthermore, in step three, the loss value is calculated according to the following formula:

[0053]

[0054] In the formula, μ is a hyperparameter; L t p is the set of target links to be predicted. l y is the probability that link l exists. l ∈{0,1} is a label indicating whether a link is a positive or negative sample. 1 indicates a positive sample and the link exists, while 0 indicates a negative sample and the link does not exist.

[0055] According to another aspect of the present invention, a static heterogeneous network link prediction system based on graph attention networks is proposed, the system comprising:

[0056] The data acquisition module is configured to acquire training and testing datasets for heterogeneous networks. The datasets include a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, where V is the set of nodes, E is the set of edges, φ is the set of node types, and ψ is the set of edge types.

[0057] The data preprocessing module is configured to perform random walks on a heterogeneous graph with edge type r using preset meta-paths to generate a path set and obtain sample batches from the path set.

[0058] The prediction model training module is configured to train the link prediction model based on the sample batches corresponding to the training dataset. For each training sample batch, the following steps are performed: First, the basic embedding of the target node is obtained by generating shared embeddings and edge embeddings of aggregated neighbors; then, based on the node sequence generated by random walk, skip-gram is performed on the node sequence to learn the node structure embedding, so as to obtain the semantic relationships and structural features between various types of nodes in the heterogeneous graph; then, edge type information in the heterogeneous graph is learned through a graph attention network layer to obtain an output vector that integrates structural information and edge type information; then, based on the output vector of the target node, the existence of an edge of type r between nodes is predicted to obtain the prediction probability; finally, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters.

[0059] Input the batches of samples corresponding to the test dataset into the trained link prediction model above for prediction, and obtain the prediction probability of each batch; calculate the area under the ROC curve and the average reciprocal rank value of the prediction probability corresponding to the test dataset.

[0060] Repeat the above training and testing steps iteratively until the maximum number of iterations is reached. Calculate the average of the area under multiple ROC curves and the average of multiple average inverse ranking values ​​from multiple iterations as the final prediction result, thus completing the training of the link prediction model.

[0061] The link prediction module is configured to input heterogeneous network data to be predicted into a trained link prediction model for link prediction.

[0062] The beneficial technical effects of this invention are:

[0063] This invention proposes a static heterogeneous network link prediction method and system based on graph attention networks. The method includes the following steps: acquiring a heterogeneous network training dataset and a test dataset, the dataset including a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, where V is the node set, E is the edge set, φ is the node type set, and ψ is the edge type set; using a preset meta-path to perform a random walk on the heterogeneous graph with edge type r to generate a path set, and obtaining sample batches from the path set; training a link prediction model based on the sample batches corresponding to the training dataset; inputting the sample batches corresponding to the test dataset into the trained link prediction model for prediction, and obtaining the prediction probability of each batch; calculating the area under the ROC curve and the average reciprocal rank value of the prediction probability corresponding to the test dataset; iteratively repeating the process until the maximum number of iterations is reached, calculating the average of the area under the ROC curves and the average of the average reciprocal rank values ​​of multiple iterations as the final prediction result, completing the training of the link prediction model; inputting the heterogeneous network data to be predicted into the trained link prediction model for link prediction. This invention was compared with six classic state-of-the-art baseline models on four public datasets, and parameter comparison and ablation experiments were conducted. The baseline model comparison results show that this invention has significant advantages over other baseline methods. The ablation experiments verified the effectiveness of the AHGNN model's global node embedding generation module, heterogeneous structure information extraction module, and edge type information fusion module. The parameter comparison experiments thoroughly investigated the impact of edge embedding dimension, edge type embedding dimension, and the path length of the Skip-gram algorithm on the AHGNN model. Attached Figure Description

[0064] The present invention can be better understood by referring to the description given below in conjunction with the accompanying drawings, which together with the following detailed description are included in and form part of this specification, and are used to further illustrate preferred embodiments of the invention and explain the principles and advantages of the invention.

[0065] Figure 1 This is a framework diagram of a static heterogeneous network link prediction method based on graph attention network according to an embodiment of the present invention.

[0066] Figure 2 This is an example diagram illustrating the impact of parameter values ​​on AUC in embodiments of the present invention; where (a) corresponds to the edge embedding dimension of the Amazon dataset; (b) corresponds to the edge type embedding dimension of the Amazon dataset; (c) corresponds to the skip-gram algorithm path length of the Amazon dataset; (d) corresponds to the edge embedding dimension of the LastFM dataset; (e) corresponds to the edge type embedding dimension of the LastFM dataset; (f) corresponds to the skip-gram algorithm path length of the LastFM dataset; (g) corresponds to the edge embedding dimension of the PubMed dataset; (h) corresponds to the edge type embedding dimension of the PubMed dataset; (i) corresponds to the skip-gram algorithm path length of the PubMed dataset; (j) corresponds to the edge embedding dimension of the YouTube dataset; (k) corresponds to the edge type embedding dimension of the YouTube dataset; and (l) corresponds to the skip-gram algorithm path length of the YouTube dataset.

[0067] Figure 3 This is an example diagram illustrating the impact of parameter values ​​on MRR in embodiments of the present invention; where (a) corresponds to the edge embedding dimension of the Amazon dataset; (b) corresponds to the edge type embedding dimension of the Amazon dataset; (c) corresponds to the skip-gram algorithm path length of the Amazon dataset; (d) corresponds to the edge embedding dimension of the LastFM dataset; (e) corresponds to the edge type embedding dimension of the LastFM dataset; (f) corresponds to the skip-gram algorithm path length of the LastFM dataset; (g) corresponds to the edge embedding dimension of the PubMed dataset; (h) corresponds to the edge type embedding dimension of the PubMed dataset; (i) corresponds to the skip-gram algorithm path length of the PubMed dataset; (j) corresponds to the edge embedding dimension of the YouTube dataset; (k) corresponds to the edge type embedding dimension of the YouTube dataset; and (l) corresponds to the skip-gram algorithm path length of the YouTube dataset. Detailed Implementation

[0068] To enable those skilled in the art to better understand the present invention, exemplary embodiments or examples of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described embodiments or examples are merely some, not all, of the embodiments or examples of the present invention. All other embodiments or examples obtained by those skilled in the art based on the embodiments or examples of the present invention without inventive effort should fall within the scope of protection of the present invention.

[0069] This invention proposes a static heterogeneous network link prediction method and system based on graph attention network, and names the proposed link prediction model: AHGNN, to capture rich node information, edge type information and structural information to learn a unified embedding from different relations, and to perform link prediction using multiple topologies from different node types.

[0070] This invention proposes a static heterogeneous network link prediction method based on graph attention networks, which includes the following steps:

[0071] Step 1: Obtain the heterogeneous network training dataset and test dataset. The dataset includes a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, where V is the set of nodes, E is the set of edges, φ is the set of node types, and ψ is the set of edge types.

[0072] Step 2: Use the preset meta-path to perform a random walk on the heterogeneous graph with edge type r to generate a path set, and obtain sample batches from the path set.

[0073] Step 3: Train the link prediction model based on the sample batches corresponding to the training dataset. For each training sample batch, perform the following steps:

[0074] First, the basic embedding of the target node is obtained by generating shared embeddings and aggregating the edge embeddings of neighbors;

[0075] Then, based on the node sequence generated by the random walk, skip-gram is performed on the node sequence to learn the node structure embedding, so as to obtain the semantic relationships and structural features between various types of nodes in the heterogeneous graph;

[0076] Then, edge type information in the heterogeneous graph is learned through a graph attention network layer to obtain an output vector that integrates structural information and edge type information.

[0077] Then, based on the output vector of the target node, the existence of an edge of type r between nodes is predicted to obtain the prediction probability;

[0078] Then, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters;

[0079] Step 4: Input the batches of samples corresponding to the test dataset into the link prediction model trained in Step 3 for prediction, and obtain the prediction probability for each batch;

[0080] Step 5: Calculate the area under the ROC curve and the average reciprocal ranking value of the predicted probabilities corresponding to the test dataset;

[0081] Step 6: Iterate and repeat steps 3 to 5 until the maximum number of iterations is reached. Calculate the average of the area under multiple ROC curves and the average of multiple average inverse ranking values ​​as the final prediction result, and complete the training of the link prediction model.

[0082] Step 7: Input the heterogeneous network data to be predicted into the trained link prediction model for link prediction.

[0083] The method begins with step one. In step one, a heterogeneous network training dataset and a test dataset are obtained, and the dataset includes heterogeneous graphs.

[0084] According to an embodiment of the present invention, a heterogeneous graph G = {V, E, φ, ψ}, where V is a set of nodes and E is a set of edges. Each node v has a type φ(v), and each edge e has a type ψ(e). The sets of node types and edge types are respectively determined by... and It means, |T v |+|T e |>1.

[0085] Then, step two is executed. In step two, a set of paths is generated by performing a random walk on the heterogeneous graph with edge type r using a preset meta-path, and a batch of samples is obtained from the set of paths.

[0086] According to an embodiment of the present invention, the heterogeneous graph G = {V, E, φ, ψ} is partitioned into |T| groups based on the edge type ψ(e). e There are 10 layers, and then a random walk is performed on each layer based on the metapath. For example, in layer G with edge type r... r =(V,E) r In the given metapath T: V1→V2→…V t →…V l Where l is the length of the meta-path, and n random walks are performed to generate a path set P. r Then from P r Obtain training samples {(v i ,v j ,r)}.

[0087] Then proceed to step three, in which the link prediction model is trained based on the sample batches corresponding to the training dataset. For each training sample batch, the following steps are performed:

[0088] First, the basic embedding of the target node is obtained by generating shared embeddings and aggregating the edge embeddings of neighbors;

[0089] Then, based on the node sequence generated by the random walk, skip-gram is performed on the node sequence to learn the node structure embedding, so as to obtain the semantic relationships and structural features between various types of nodes in the heterogeneous graph;

[0090] Then, edge type information in the heterogeneous graph is learned through a graph attention network layer to obtain an output vector that integrates structural information and edge type information.

[0091] Then, based on the output vector of the target node, the existence of an edge of type r between nodes is predicted to obtain the prediction probability;

[0092] Then, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters.

[0093] According to embodiments of the present invention, such as Figure 1 As shown, the link prediction model AHGNN consists of four parts: a global node embedding generation module, a heterogeneous structure information extraction module, an edge type information fusion module, and a result prediction module. The global node embedding generation module is used to concatenate shared embeddings and edge embeddings to learn a unified embedding from different relationships; the heterogeneous structure information extraction module is used to obtain low-order and high-order structural features of heterogeneous networks; the edge type information fusion module introduces node residual connections to fuse edge type information and solve the problems of over-smoothing and gradient vanishing in GNN training.

[0094] 1) Overall node embedding generation module: Obtains the basic embedding of the target node by generating shared embeddings and aggregating the edge embeddings of neighbors; specifically, initializes the shared embedding b of nodes in the heterogeneous graph. i and edge embedding matrix U i U, the mean aggregation of first-order neighbors i Using self-attention mechanism to U i Perform a weighted summation, then add b. i Obtain the basic embedding z of the node on each type of edge. i,r .

[0095] Network embedding refers to generating low-dimensional embeddings by modeling the topology and node attribute information. Network embedding can transform complex network structures into easily processed vector representations, thus facilitating link prediction. Figure 1 In a heterogeneous network, node v i The global embedding on an edge of type r consists of two parts: shared embedding and edge embedding. Node v i The shared embedding is shared between different edge types; that is, when a heterogeneous graph is transformed into a homogeneous graph by removing edge and node types, the information carried by the nodes is retained. Node v i Edge embedding of type r It is formed by embedding and aggregating the edges of its neighbors:

[0096]

[0097] Where, d u Let N be the dimension of the edge embedding. i,r It is node v i The set of neighbors on edges of type r, for each node v i and the initial edge embedding for each edge type The aggregator function is randomly generated by the model. It uses a mean aggregator, as shown below:

[0098]

[0099] Where σ is the activation function, It is a mapping matrix, combining nodes v i All types of edges are embedded as nodes v i edge embedding matrix Where d u It is the dimension of the edge embedding, |T e | represents the number of edge types; mean indicates mean aggregation. Represents node v i Neighbor node v on the layer with edge type r j The (k-1)th order edge embedding;

[0100] Combine all edge types of edge embedding u i,r Get the target node v i edge feature matrix U i :

[0101]

[0102] Mean aggregator can be converted into matrix multiplication:

[0103]

[0104] Where, N r ∈R |V|×|V| It is the normalized adjacency matrix of edges of type r. express The i-th column, U r ={u 1,r ,u 2,r ,u 3,r …u m,r} is the set of edge embeddings for all nodes on edges of this type. Where |V| represents the number of nodes in the graph.

[0105] U is calculated using a self-attention mechanism. i The coefficients of the linear combination of vectors in for:

[0106]

[0107] in, and These are the trainable parameters on edges of type r, where the superscript T denotes the transpose of a vector or matrix, and a i,r This represents the weights of each type of edge embedding. Softmax is the activation function.

[0108] Therefore, node v i The basic embedding of type r edge is:

[0109]

[0110] in, Represents node v i The shared embedding, ξ, is randomly initialized by the model. r λ is a hyperparameter representing the importance of edge embeddings to the overall embedding. p Indicates a i,r The p-th element:

[0111]

[0112] u i,p Represents node v i The edge embeddings of type p are defined as edge embeddings; R represents the total number of edge types.

[0113] 2) Heterogeneous Structure Information Extraction Module: Based on the node sequences generated by random walks, skip-gram is performed on the node sequences to learn node structure embeddings, thereby obtaining semantic relationships and structural features among various types of nodes in the heterogeneous graph. Specifically, on the node sequences obtained based on meta-path random walks, the Skip-Gram algorithm is used to obtain the structural information of the heterogeneous network and optimize the basic embedding information z. i,r The basic embedding z on the edges of each type of node. i,r The structural embedding h of the nodes is obtained by splicing. i .

[0114] Random walks are used to generate node sequences, and then skip-grams are performed on these sequences to learn embeddings. Since each view of the input network is heterogeneous, meta-path-based random walks are employed. Specifically, given a network view r, G... r =(V,E) r and metapath T: V1→V2→…V t →…V l Let l be the length of the meta-path, and the transition probability at step t be defined as follows:

[0115]

[0116] Among them, v i ∈V t N i,r Represents node v i In view G r The neighborhood of the node is determined. A random walk is performed according to a predefined meta-path T. The meta-path-based random walk strategy ensures that the semantic relationships between different types of nodes can be correctly incorporated into the Skip-Gram model.

[0117] Suppose a random walk of type r with edge length l follows a path Context Where c is the radius of the window size. Therefore, given a node v i Given its path context C, the objective of negative sampling is to minimize the following negative log-likelihood:

[0118]

[0119] Where θ represents all the above parameters. A heterogeneous softmax function is used, which is based on node v. j The node type is normalized. Specifically, given v i time v j The probability is defined as follows, where v j ∈V t ,

[0120]

[0121] Finally, node v i The negative sampling objective function -logP θ (v j |v i Approximately:

[0122]

[0123] Where σ(x) = 1 / (1 + exp(-x)) is the sigmoid function, L is the number of negative samples corresponding to the positive training samples, and v k From node v j The corresponding node set V t The noise distribution P defined above t (v) was randomly selected. i,r z represents the basic embedding of node i on edge type r. j,r This represents the basic embedding of node j on edge type r; Indicates from node v j The corresponding node set Vt The noise distribution P defined above t (v) are randomly selected from v. k The function; v k From node v j The corresponding node set V t The noise distribution P defined above t (v) was randomly selected.

[0124] Therefore, the objective function can update the neighbor sets of nodes in heterogeneous graphs in various types of network views, thereby capturing more semantic information between different node types, making the structural information more complete and the node information more perfect, so that the model is more balanced during training.

[0125] 3) Edge type information fusion module: This module learns edge type information in heterogeneous graphs through a graph attention network layer to obtain an output vector that fuses structural and edge type information; specifically, it initializes the edge type feature r. ψ(<i,j>) h is embedded through the structure of the node i and edge type feature r ψ(<i,j>) Calculate the attention coefficient, add residual connections based on this, aggregate neighborhood structure features, and output the final embedding o for each node after L2 normalization. i .

[0126] While GAT possesses powerful representational capabilities for modeling isomorphic graphs, its performance on heterogeneous graphs may not be optimal due to its neglect of node or edge types. To address this issue, the model extends the original graph attention mechanism by incorporating edge type information into the attention computation. Specifically, at each layer, for each edge type ψ(e)∈T... e Edge type embedding Attention scores are calculated using node embeddings and edge type embeddings. As shown below:

[0127]

[0128] Where, ψ(<i,j> The ) indicates the type of the edge between node i and node j. and It is a learnable mapping matrix used to map node embeddings and edge type embeddings. LeakyReLU represents the activation function; α represents the learnable mapping vector; r ψ(<i,j>) Represents edge type characteristics; h j The structure embedding of node j is indicated by h. k N represents the structural embedding of node k; i Represents node v i The set of neighbors.

[0129] To address the over-smoothing and vanishing gradient issues in GNNs, the AHGNN model introduces residual connections. Cross-layer node representations incorporate pre-activated residual connections. The aggregation at layer l can be represented as:

[0130]

[0131] in, It is node v i and v j The attention weights between the edges, where σ is an activation function. It is formed by splicing together the basic embeddings of nodes on all types of edges:

[0132]

[0133] therefore dimensional d h =d u ×|T e | Represents node v i Structural embedding of the network in layer l, W (l) Let represent the trainable mapping matrix of the l-th layer network.

[0134] Residual connections on the attention score can effectively solve the problems of over-smoothing and vanishing gradients. The original attention score is obtained through formula (12). Then, add residual joins:

[0135]

[0136] The hyperparameter β∈[0,1] is the scaling factor. Represents node v i For node v j Attention score in the l-th layer of the network.

[0137] Similar to GAT, the algorithm employs multi-head attention to enhance the model's expressive power. Specifically, it executes a K-independent attention mechanism according to formula (12) and concatenates the results as the final representation. The corresponding update rule is:

[0138]

[0139]

[0140]

[0141] Where ∥ represents a splicing operation, It is the l-th linear transformation The attention score is calculated based on equation (15). The output size cannot usually be precisely divided by the number of heads. After GAT, no further concatenation is performed; instead, an average representation is used in the final (L) layer.

[0142]

[0143] L2 normalization is performed on the output embedding:

[0144]

[0145] in, It is node v i The L2-normalized output embedding, This is the final expression of formula (19).

[0146] 4) Result prediction module: Based on the output vector of the target node, predict whether there is an edge of type r between the nodes, obtain the prediction probability, and use the DistMult decoder to parse the embedding to obtain the prediction result.

[0147] Specifically, the final output embedding o of node i is obtained through the above steps. i Link prediction requires decoding the embedding. DistMult outperforms direct dot product. The prediction formula for determining whether there exists an edge of type r between node pairs u and v is as follows:

[0148]

[0149] Among them, Prob r This represents the probability that there is an edge of type r between nodes u and v; Let d be a learnable square matrix of edge type r, where d o =Kd h ;o u and o v Let represent the output embeddings of nodes u and v, respectively. Link prediction loss function. Using formula (22):

[0150]

[0151] Among them, L t p is the set of target links to be predicted. l y is the probability that link l exists. l Let {0, 1} be the label indicating whether a link is a positive or negative sample, where 1 indicates a positive sample and the link exists, and 0 indicates a negative sample and the link does not exist. Then the total loss function... It can be expressed by formula (23), where μ is a hyperparameter:

[0152]

[0153] Then proceed to step four, in which the sample batches corresponding to the test dataset are input into the link prediction model trained in step three for prediction, and the prediction probability of each batch is obtained.

[0154] Then proceed to step five, in which the area under the ROC curve and the average reciprocal rank value of the predicted probability corresponding to the test dataset are calculated.

[0155] According to embodiments of the present invention, AUC is a commonly used evaluation metric that measures the classifier's ability to distinguish between positive and negative samples. MRR is a metric that evaluates model performance based on the link ranking results predicted by the model. The link prediction problem refers to predicting whether a connection exists between two nodes in a network, which can be viewed as a binary classification problem. The sample labels in the link prediction problem have two classes: positive and negative, and the prediction results have two classes: connected and not connected. Based on the sample labels and the classification of the prediction results, the prediction results can be divided into four groups, represented by a confusion matrix, as shown in Table 1, where rows represent predicted values ​​and columns represent label values.

[0156] Table 1. Confusion Matrix of Link Prediction Task

[0157]

[0158] True Positives (TP): The number of positive samples that are correctly predicted as connected samples.

[0159] False Positives (FP): The number of positive samples that are incorrectly predicted as unconnected samples.

[0160] False Negatives (FN): The number of negative samples that are incorrectly predicted as connected samples.

[0161] True Negatives (TN): The number of negative samples that are correctly predicted as unconnected samples.

[0162] Therefore, TP and TN are the samples that are predicted accurately, while FP and FN are the samples that are predicted incorrectly.

[0163] AUC is the area under the ROC curve. In link prediction models, creating the ROC curve first involves sorting the probability values ​​of each sample's predicted outcome as belonging to a connected link in descending order. Simultaneously, the probability values ​​and labels are combined into a table. Starting with the highest probability value in the sorted table, each sample's probability value is treated as a threshold. For each threshold, all samples above or equal to this threshold are considered positive samples, while samples below the threshold are considered negative samples. For each threshold, two key metrics need to be calculated: True Positive Rate (TPR) and False Positive Rate (FPR), as shown in the formulas below.

[0164]

[0165]

[0166] Using TPR as the vertical axis and FPR as the horizontal axis, we can obtain the ROC curve. AUC is the area under the ROC curve, with a value ranging from 0.5 to 1. The closer the value is to 1, the better the model's performance.

[0167] MRR is a standard metric for ranking, treating link prediction as a link retrieval problem. Since exhaustive computation for all node pairs is too cumbersome, positive and negative samples of the target node are retrieved as a group. The MRR is calculated by sorting the probability values ​​of the target node's samples in descending order based on the link prediction results, finding the rank of the first positive sample, calculating the reciprocal of that position, and then averaging the reciprocals of the rank of the first positive sample for all nodes. The formula is shown below.

[0168]

[0169] Where n is the number of nodes, rank i It represents the rank of the first positive sample at the i-th node. The MRR value ranges from 0 to 1. The closer the value is to 1, the higher the accuracy of the algorithm's prediction for the first positive sample and the better the performance.

[0170] Then proceed to step six. In step six, steps three to five are iteratively repeated until the maximum number of iterations is reached. The average of the area under multiple ROC curves and the average of multiple average inverse ranking values ​​are calculated as the final prediction result, thus completing the training of the link prediction model.

[0171] The goal of the AHGNN model is to capture effective structural information, including edge type information and neighborhood node information, and effective node information, including node type information, in static heterogeneous networks, thereby completing the link prediction task. The pseudocode for the heterogeneous network link prediction algorithm based on graph attention networks is shown in Algorithm 1. The algorithm requires the following inputs: meta-path T, training set (Train), test set (Test), maximum number of iterations (Epoch), and batch size of each training sample. During model training, the training set data is divided into multiple batches according to the batch size, and the model is trained iteratively. The model's performance is tested after each epoch of training.

[0172]

[0173]

[0174] Lines 1 to 3 of the algorithm describe the generation of training samples using random walks based on meta-paths. First, the heterogeneous graph G = {V, E, φ, ψ} is partitioned into |T| groups according to the edge type ψ(e). e There are 10 layers, and then a random walk is performed on each layer based on the metapath. For example, in layer G with edge type r... r =(V,E) r In the given metapath T: V1→V2→…V t →…V l Where l is the length of the meta-path, and n random walks are performed to generate a path set P. r Then from P r Obtain training samples {(v i ,v j ,r)}.

[0175] Lines 6 to 10 of the algorithm describe the module for generating global node embeddings, which generates shared embeddings b. i and the edge embedding of aggregated neighbors u i,r Obtain the basic embedding z of the target node i,r First, according to formula (2), the target node v i The edge embeddings of k-order neighbors on a layer of type r are obtained by mean aggregation. In the implementation, the length of the meta-path designed for the dataset is 2, thus obtaining the first-order and second-order neighbors of the target node, i.e., k=2. As shown in the formula, the parameter requiring backpropagation training is the mapping matrix. Secondly, combine all types of edge embeddings u i,r Get node v i edge feature matrix U i Then, U is calculated using the self-attention mechanism according to formula (5). iThe coefficient α of the linear combination of vectors on edge type r i,r As can be seen from the formula, the parameters that need to be trained by backpropagation are the weight matrix. Finally, according to formula (6), the edge embedding matrix U i Perform a weighted summation and integrate the shared embedded b i Obtain the basic embedding z of the target node on each edge type. i,r The module that generates the overall node embedding captures basic node information, including shared information and neighborhood information.

[0176] Lines 11 to 15 of the algorithm correspond to the heterogeneous Skip-Gram module. Based on the node sequence generated by the random walk, skip-gram is performed on the node sequence to learn the node embedding. First, a path P of length l with edge of type r and the context C of the target node are obtained by random walk based on the meta-path T; then, the transition probability p(v) of the next step of the target node is obtained by formula (8). j |v i According to formula (11), the negative sampling optimization objective function E1 is obtained, and the optimization objective function guides the basic embedding z. i,r The learning process; finally, according to formula (14), the basic embeddings z of each edge type of the target node are spliced ​​together. i,r Obtain node structure embedding This serves as input to the next module. This module has no parameters that require backpropagation training. The heterogeneous Skip-Gram module can capture the semantic relationships and structural features between various types of nodes in a heterogeneous graph.

[0177] Lines 16 to 20 of the algorithm describe the heterogeneous GAT module of AHGNN, which learns edge type information in the heterogeneous graph through the GAT layer. First, according to formula (12), the edge type embedding r is used. ψ (e) and node structure embedding h i To calculate attention score The parameters that need to be trained via backpropagation in this formula are: This module sets different numbers of GAT layers for sparse and dense networks. For sparse heterogeneous networks, three GAT layers are set, i.e., n=3; for dense heterogeneous networks, two GAT layers are set, i.e., n=2. Secondly, to address the over-smoothing and gradient vanishing problems of GNNs, the AHGNN model introduces residual connections and, according to formula (15), adjusts the original attention score... Add residual connections. The algorithm uses multi-head attention to enhance the model's expressive power, therefore the parameter set to be trained needs to be updated to... Where K represents the number of attention heads. This module learns to obtain an output vector o that integrates structural and edge type information.i .

[0178] Lines 21 and 22 of the algorithm correspond to the result prediction module. Using formula (22), it predicts whether there are edges of type r between nodes based on the output vector of the target node. The parameters that need to be trained are... In summary, the parameters that need to be trained in AHGNN are θ = {θ1, θ2, θ3, θ4}. Line 22 of the algorithm calculates the loss value, the optimizer uses Adam, and backpropagation updates θ.

[0179] Finally, step seven is executed, in which the heterogeneous network data to be predicted is input into the trained link prediction model for link prediction. Specific Implementation Example 1

[0181] As shown in Table 2, the heterogeneous network dataset - Amazon product dataset includes product metadata and links between products. This invention only uses product metadata for the electronics category, including product attributes and links for shared viewing and purchasing between products. Product attributes include price, sales ranking, brand, and category, and include electronics products. Node type is product, and edge type is links between them that allow for shared viewing and purchasing.

[0182] Table 2

[0183] Dataset Number of nodes Number of node types Number of sides Number of edge types average degree Amazon 10099 1 148659 2 14.72

[0184] In heterogeneous networks similar to Amazon, this invention can solve a variety of problems: optimize product recommendations by leveraging users' shared viewing and purchasing data to recommend products that users may not have discovered but could be interested in, thereby increasing product exposure and sales opportunities; identify potential new connections by predicting new edges that may form in the future (such as new shared viewing or purchasing relationships) to help merchants identify market demand and user interests in advance; perform product classification and clustering analysis by analyzing the connectivity between products to identify which products are similar or can be grouped into the same category, which can help improve product classification and display strategies; and facilitate market strategy formulation by enabling targeted marketing activities based on link prediction results, thereby improving marketing effectiveness.

[0185] When performing static heterogeneous network link prediction, the heterogeneous network data to be predicted is the heterogeneous graph of the current heterogeneous network. A trained link prediction model is used for link prediction, given the current product nodes (product A, product B, etc.) and their known links (edges for shared viewing and shared purchase). Link prediction is performed on the current network structure, and the results include: predicting which products are likely to form new shared viewing or shared purchase relationships, assigning confidence scores to these potential links, identifying similar products or product groups that can be bundled, and improving the effectiveness of product combinations. This invention can optimize product recommendations, improve sales conversion rates, and thus gain an advantage in a highly competitive market. Specific Implementation Example 2

[0187] As shown in Table 3, the heterogeneous network dataset - LastFM is an online music website. Using the LastFM dataset from reference [1], the dataset was preprocessed by filtering out users and tags with only one link. This dataset comes from the Last.fm7 online music system. Its users are connected to each other in a social network generated by Last.fm "friends". Each user has a list of most popular music artists, tags, i.e., a tuple [user, tag, artist], and friend relationships in the dataset's social network. Each artist has a Last.fm URL and an image URL. The node types are user, tag, and music artist, and the edge types are user-music artist, user-tag, and music artist-tag.

[0188] Table 3

[0189] Dataset Number of nodes Number of node types Number of sides Number of edge types average degree LastFM 20612 3 141521 3 6.87

[0190] In heterogeneous networks similar to LastFM, the heterogeneous network data to be predicted is the heterogeneous graph of the current heterogeneous network. Link prediction is performed using a trained link prediction model. Based on the current heterogeneous network structure, the following link prediction tasks can be completed, namely, the prediction results include: predicting user-music artist edges, predicting which users may be interested in some music artists they have not yet heard; predicting user-tag edges, predicting which tags users may like or add to their preferences; and predicting artist-tag edges, predicting which tags some music artists may be associated with in order to better describe their music style. This invention addresses the following issues: user recommendation, generating a list of recommended music artists for each user based on their historical behavior and tagged preferences to help them discover new music; tag recommendation, providing each user with a list of relevant tags to encourage them to use these tags to describe their favorite music, enhancing user interaction with the platform; artist tag relationships, providing corresponding tag recommendations for each music artist to help them attract more listeners and optimize their performance on the platform; potential edge identification, identifying edges in the current network that have not yet been established but have potential connections, such as possible relationships between users and music artists, and between users and tags, thereby further optimizing the recommendation algorithm; and user behavior pattern identification, analyzing user behavior patterns based on prediction results to identify potential preferences, thus providing data support for marketing campaigns or adjustments to the recommendation system. Through these predictions and analyses, online music systems can more effectively meet user needs, improve user experience, and increase the discoverability and relevance of music. This not only improves user satisfaction but also enhances platform operational efficiency and user stickiness. Specific Implementation Example 3

[0192] As shown in Table 4, the heterogeneous network dataset - PubMed is a biomedical literature library that uses the PubMed dataset from reference [2]. This dataset is constructed from PubMed into a network consisting of genes, diseases, chemicals and species. The node types are gene, disease, chemical substance, and species, and the edge types are as follows: gene-disease edge, indicating the correlation between a gene and a specific disease, such as the relationship between gene mutation and related diseases; gene-chemical substance edge, indicating the interaction between a gene and a chemical substance (such as a drug or compound), usually used to describe the mechanism of drug action; gene-species edge, indicating the conservation or expression of a gene in different species, which helps to compare gene functions; disease-chemical substance edge, indicating the relationship between a specific disease and a chemical substance (drug) used to treat that disease; disease-species edge, indicating the manifestation and research of a disease in different species, such as the correlation between animal models and human diseases; chemical substance-species edge, indicating the research and application of a chemical substance in a specific species, especially in pharmacology and toxicology; gene-gene edge, indicating the interaction or regulatory relationship between genes, such as co-expressed genes or regulatory networks; disease-disease edge, indicating the similarity or comorbidity between diseases, such as multiple diseases may have common biomarkers; chemical substance-chemical substance edge, indicating the similarity or interrelationship between different chemical substances, such as the structural similarity or functional association of compounds; species-species edge, indicating the similarity between species, usually based on the similarity of their genome, phenotype, or ecological function.

[0193] Table 4

[0194] Dataset Number of nodes Number of node types Number of sides Number of edge types average degree PubMed 63109 4 244986 10 3.88

[0195] In PubMed's heterogeneous networks, the heterogeneous network data to be predicted is the heterogeneous graph of the current heterogeneous network. A trained link prediction model is used to predict links; this can solve several important problems in the biomedical field: gene-disease relationship prediction (predicting potential associations between genes and unconfirmed diseases); drug-disease association prediction (predicting which drugs may be effective for specific diseases); gene function prediction (inferring the function of unstudied genes, especially in disease research); disease comorbidity prediction (identifying diseases that may occur simultaneously to improve clinical decision-making); chemical substance interaction prediction (predicting possible interactions between chemical substances for use in drug combination therapy); and interspecies conservation analysis (exploring similar biological mechanisms and disease models in different species).

[0196] Based on the known network structure, missing edges can be predicted. The prediction results include: missing links between genes and diseases (a gene may be associated with multiple unidentified diseases but not recorded); links between genes and drugs (a drug may act on an unstudied gene); links between diseases and chemical substances (known drugs may affect new diseases or symptoms); and unknown links between species and diseases (the manifestation of a specific disease in other species). Based on the existing network structure, new edge predictions can be obtained through link prediction, such as gene predictions of associations with specific diseases and potential new indications for drugs. This invention, based on the prediction results, will yield richer node relationships in the network, thereby enhancing the overall network usability and supporting new directions in biomedical research. By identifying new associations, it can guide future experiments, such as drug development targeting specific genes or research into the mechanisms of diseases. Specific Implementation Example 4

[0198] As shown in Table 5, the heterogeneous network dataset - YouTube dataset is a multi-way bidirectional network dataset consisting of five interaction types between 15,088 YouTube users. The node type in the YouTube dataset is user, and the edge types include contacts, shared friends, shared subscriptions, shared subscribers, and shared favorite videos between users.

[0199] Table 5

[0200] Dataset Number of nodes Number of node types Number of sides Number of edge types average degree Youtube 2000 1 1310544 5 655.27

[0201] In the YouTube dataset, the heterogeneous network data to be predicted is a heterogeneous graph of the current heterogeneous network. A trained link prediction model is used to predict links. This invention can solve several problems, particularly in social network analysis and user behavior prediction: It can predict user relationships, identifying which users are likely to become friends or contacts, thereby enhancing the connectivity of social networks; it can predict shared subscriptions and preferences, predicting channels or videos that users may jointly subscribe to, improving user experience and personalized content recommendations; it can perform influence propagation analysis, predicting which users are likely to have the greatest influence on specific content, thus helping to formulate marketing strategies or optimize information dissemination; it can discover potential communities, identifying potential communities or user groups based on the interaction types between users, for better content recommendation or social promotion; and it can be used for content recommendation, recommending relevant videos or channels to users based on the relationships and interaction types between them.

[0202] Based on the current network structure, missing edges can be predicted. The prediction results include: potentially unestablished connections between users (e.g., two users may not be contacts but could establish a connection based on other interaction types); missing shared friend relationships (e.g., two users may have mutual friends but are not yet connected); potentially shared channels (e.g., one user and another may share a specific channel); and potentially shared favorite videos (e.g., one user and another may share the same interests but these are not recorded). The prediction results of this invention in this type of scenario include the following aspects: newly established user relationships, by predicting possible connections, new friendships or interactions between users can be discovered; recommended shared subscription channels, predicting channels that new users may share, which can be used in personalized recommendation systems; interest-matching recommendations, recommending potentially interesting movies or channels through shared preferences to improve user experience; and enhanced social networks, through link prediction, social networks will become closer, and user interactions will increase. Through the static heterogeneous network link prediction technology proposed in this invention, we can better understand user behavior patterns, improve the quality of social interactions, and enhance the accuracy of content recommendations, thereby improving the overall user experience.

[0203] The technical effects of the present invention were further verified through experiments.

[0204] To evaluate the accuracy of link prediction based on the AHGNN model, six of the most representative and state-of-the-art heterogeneous network link prediction methods were selected as baseline methods for comparative experiments. The specific methods are described below:

[0205] GCN: A deep learning model based on graph-structured data. This method learns node representations by performing convolution operations on the graph structure. GCN aggregates the feature vectors of each node with the feature vectors of its neighboring nodes to obtain a new representation of the node for link prediction.

[0206] GAT: A deep learning model based on graph attention mechanism. This method learns the relationship weights between nodes through attention mechanism and multi-head attention mechanism, weights the features of neighboring nodes, obtains node representation, and thus realizes the link prediction task.

[0207] RGCN: A deep learning model for processing relational graph data. This method assigns a weight matrix to each relation type to learn the feature representation of that relation type. RGCN extends GCN to relational (polygonal) graphs, performing convolutional operations on the graph structure to learn node representations. It updates node representations using weight matrices for different relation types, thereby enabling link prediction.

[0208] HetGNN: A deep learning model for processing heterogeneous graphs. This method sets different representations and features for each type of node and edge, representing the heterogeneous graph as a multi-layer structure. It learns the node representation by passing information across different layers, thereby performing link prediction tasks.

[0209] HGT: A deep learning model for processing heterogeneous graphs based on the Transformer architecture. This method utilizes the Transformer architecture, including self-attention and multi-head attention mechanisms. By passing information across Transformer blocks in different layers, it learns node representations to complete the link prediction task.

[0210] LHGNN: A method for link prediction in latent heterogeneous graphs. This method maps the original heterogeneous graph to a low-dimensional latent space to obtain a representation of the latent heterogeneous graph. LHGNN performs link prediction by acquiring node-level and edge-level latent semantic information even when the types of network nodes and edges are unknown.

[0211] Experiment 1: Baseline Model Comparison Experiment. A comparative experiment was conducted on four datasets to examine the proposed AHGNN model and six baseline methods. In this invention, edges present in the heterogeneous network are called positive samples, and edges not present are called negative samples. The training, validation, and test sets all contain both positive and negative samples. Positive samples are formed by shuffling the edge sets of the heterogeneous network and dividing them proportionally, with 81% used for training, 9% for validation, and 10% for testing. In real-world scenarios, it is usually necessary to distinguish between positive and negative samples with similar characteristics, rather than random negative samples. Therefore, when performing negative sampling, edges not present between the target node and its two-hop neighbors are selected as negative samples. To ensure training balance, the number of positive and negative samples in the training set is equal, and the ratio of positive to negative samples in the test set is also 1:1. Ten experiments were conducted, and the average value was taken as the final result. The experimental results are shown in Tables 6 and 7.

[0212] Table 6. AUC results (%) of the AHGNN method and the corresponding model.

[0213]

[0214] Table 7. MRR results (%) of the AHGNN model compared with the baseline model.

[0215]

[0216] Analysis of Experiment 1: The results show that a simple homogeneous GAT can match the best heterogeneous neural network in most cases. AHGNN, which inherits from the heterogeneous GAT, outperforms all state-of-the-art heterogeneous neural network methods in link prediction on the three datasets. R-GCN learns node representations by introducing relation feature matrices and multi-layer GCN structures, but its accuracy is not as high as that of a simple GCN because R-GCN is based on local neighbor information for node representation learning, which has the limitation of failing to capture global graph structure information. In tasks that require better capture of global information, traditional GCNs perform better. HetGNN assigns GNN weight matrices separately for edge type and node type, but the number of nodes and edges of different types varies greatly. For relation types with low co-occurrence frequency, it is difficult to learn accurate weights for them, so its performance is not as good as traditional GNNs. HGT models heterogeneous attention for each edge and implicitly learns meta-paths, but when the relation types are imbalanced, the individual information of nodes in the larger class is ignored, so it does not achieve optimal results. LHGNN can acquire heterogeneous node-level and edge-level information even when the types of network nodes and edges are unknown, thus performing well in networks with imbalanced node and edge types. The AHGNN model proposed in this invention outperforms all state-of-the-art heterogeneous network link prediction methods on three datasets in terms of the AUC metric, and consistently outperforms all baseline methods in terms of the MRR metric. This indicates that AHGNN can learn more effective feature vectors to represent the links to be predicted.

[0217] Experiment 2 Ablation Experiment: To verify the effectiveness of each module in AHGNN, an ablation experiment without global node embedding (i.e., AHGNN) was set up. -add ), heterogeneous Skip-Gram (i.e., AHGNN) -skip ), heterogeneous GAT (i.e., AHGNN) -HGAT A comparative experiment of three models. Specifically, the sum of shared embeddings and edge embeddings in AHGNN was transformed into a model using only shared embeddings as the AHGNN model. -add The negative sampling optimization function of Skip-Gram is removed as the AHGNN. -skip Replace the heterogeneous GAT layer with a linear layer as the AHGNN. -HGAT The experimental results are shown in Table 8.

[0218] Table 8 Ablation Experiment Results (%)

[0219]

[0220] Experiment 2 Analysis: The results show that the complete AHGNN model performs best in both metrics, demonstrating the effectiveness of generating global node embeddings, heterogeneous Skip-Gram, and heterogeneous GAT modules. The model lacking heterogeneous GAT performs the worst overall, indicating the need to aggregate edge type information to complete node information in heterogeneous graphs. The models lacking global node embeddings and heterogeneous Skip-Gram also show decreased performance compared to the complete AHGNN model, proving the effectiveness of aggregating neighborhood node features for link prediction tasks and that the heterogeneous Skip-Gram algorithm reduces some structural information loss.

[0221] Experiment 3: Parameter Comparison Experiment: In order to optimize the parameters of the AHGNN method, this invention selects the edge embedding vector dimension d. u In heterogeneous GAT, the dimension d of the edge-type embedding vector is... r The path length l of the Skip-gram algorithm s These three parameters were compared in an experiment. Specifically, for d... u Parameter comparison, fixed d r For 32, l s If d is 10, then d u The values ​​are set to 16, 32, 64, 128, 256, and 512 respectively. For d... r Parameter comparison, fixed d u For 200, l s If d is 10, then d r The values ​​are set to 4, 8, 16, 32, and 64 respectively. s The experimental setup is similar, with d fixed. u 200, d r For 32, sequentially l s The values ​​were set to 4, 8, 16, 32, and 64. The experimental results are as follows: Figure 2 and Figure 3 As shown.

[0222] Analysis of Experiment 3: Experimental results show that the edge embedding vector dimension d u The model achieves optimal performance on most datasets when set to 128, and best performs on the MRR metric when set to 32 on LastFM. For the edge-type embedding vector dimension d in heterogeneous GAT... r The model achieved its optimal AUC metric when set to 16 on most datasets. r When the AUC metric is 32, the AHGNN model performs best on the Amazon dataset. r The model achieved its optimal performance on the MRR metric when set to 32 on most datasets. rThe model with a path length of 16 l in the Skip-gram algorithm showed the best MRR performance on the LastFM dataset. s As can be seen, the model performs best in AUC and MRR metrics on most datasets when the path length is 32. On Amazon, the model performs best in AUC when the path length is 16, and the model achieves the best MRR performance when the path length is 8.

[0223] Another embodiment of the present invention proposes a static heterogeneous network link prediction system based on graph attention networks, the system comprising:

[0224] The data acquisition module is configured to acquire training and testing datasets for heterogeneous networks. The datasets include a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, where V is the set of nodes, E is the set of edges, φ is the set of node types, and ψ is the set of edge types.

[0225] The data preprocessing module is configured to perform random walks on a heterogeneous graph with edge type r using preset meta-paths to generate a path set and obtain sample batches from the path set.

[0226] The prediction model training module is configured to train the link prediction model based on the sample batches corresponding to the training dataset. For each training sample batch, the following steps are performed: First, the basic embedding of the target node is obtained by generating shared embeddings and edge embeddings of aggregated neighbors; then, based on the node sequence generated by random walk, skip-gram is performed on the node sequence to learn the node structure embedding, so as to obtain the semantic relationships and structural features between various types of nodes in the heterogeneous graph; then, edge type information in the heterogeneous graph is learned through a graph attention network layer to obtain an output vector that integrates structural information and edge type information; then, based on the output vector of the target node, the existence of an edge of type r between nodes is predicted to obtain the prediction probability; finally, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters.

[0227] Input the batches of samples corresponding to the test dataset into the trained link prediction model above for prediction, and obtain the prediction probability of each batch; calculate the area under the ROC curve and the average reciprocal rank value of the prediction probability corresponding to the test dataset;

[0228] Repeat the above training and testing steps iteratively until the maximum number of iterations is reached. Calculate the average of the area under multiple ROC curves and the average of multiple average inverse ranking values ​​from multiple iterations as the final prediction result, thus completing the training of the link prediction model.

[0229] The link prediction module is configured to input heterogeneous network data to be predicted into a trained link prediction model for link prediction.

[0230] The function of the static heterogeneous network link prediction system based on graph attention network described in this embodiment can be explained by the aforementioned static heterogeneous network link prediction method based on graph attention network. Therefore, for the parts not described in detail in this embodiment, please refer to the above method embodiments, and they will not be repeated here.

[0231] Although the invention has been described with respect to a limited number of embodiments, those skilled in the art will understand from the foregoing description that other embodiments are conceivable within the scope of the invention described herein. The disclosure of the invention is illustrative and not restrictive, and the scope of the invention is defined by the appended claims.

[0232] The following documents are cited in this invention:

[0233] [1]Cantador I, Brusilovsky P, Kuflik T.Second workshop on informationheterogeneity and fusion in recommender systems(HetRec2011)[C] / / Proceedings of the fifth ACM conference on Recommender systems.2011:387-388.

[0234] [2]Dong Y, Hu Z, Wang K, et al. Heterogeneous network representation learning [C] / / IJCAI.2020,20:4861-4867.

Claims

1. A method for predicting links in static heterogeneous networks based on graph attention networks, characterized in that, Includes the following steps: Step 1: Obtain the heterogeneous network training and testing datasets. The datasets include heterogeneous graphs; heterogeneous graphs ,in It is a set of nodes. It is an edge set. It is a set of node types. It is a set of edge types; Step 2: Utilize the preset metapath in the edge type... Random walks are performed on the heterogeneous graph to generate a set of paths, and sample batches are obtained from the set of paths; Step 3: Train the link prediction model based on the sample batches corresponding to the training dataset. For each training sample batch, perform the following steps: First, the basic embedding of the target node is obtained by generating shared embeddings and aggregating the edge embeddings of its neighbors; this includes: using a mean aggregator to embed the target node... The type is On the layer The edge embeddings of the order neighbors are aggregated by mean to obtain the edge embeddings of that edge type. : ； In the formula, It is an activation function. It is a mapping matrix; Indicates mean aggregation, It is a node In type The set of neighbors on the edge; Represents a node In edge type Neighboring nodes on the layer The Edge embedding; edge embedding that combines all edge types Get the target node edge feature matrix The edge feature matrix is ​​computed using a self-attention mechanism. In edge type coefficients of a linear combination of vectors on : ； In the formula, It is an activation function. and They are of type Trainable parameters on the edges, superscript Represents the transpose of a vector or matrix; based on coefficients Edge embedding matrix Perform weighted summation and integrate shared embedded data. Obtain the target node Basic embeddings on various edge types : ； In the formula, Represents a node The shared embeddings are randomly initialized by the model; It is a hyperparameter that represents the importance of edge embeddings to the overall embedding; express The One element; Represents a node The edge embeddings of type p; R represents the total number of edge types; Then, based on the node sequence generated by the random walk, skip-gram is performed on the node sequence to learn the node structure embedding, so as to obtain the semantic relationships and structural features between various types of nodes in the heterogeneous graph; Then, edge type information in the heterogeneous graph is learned through a graph attention network layer to obtain an output vector that integrates structural information and edge type information. Then, based on the output vector of the target node, determine whether there is a type between the nodes. We predict the edges to obtain the predicted probabilities; Then, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters; Step 4: Input the batches of samples corresponding to the test dataset into the link prediction model trained in Step 3 for prediction, and obtain the prediction probability for each batch; Step 5: Calculate the area under the ROC curve and the average reciprocal ranking value of the predicted probabilities corresponding to the test dataset; Step 6: Iterate and repeat steps 3 to 5 until the maximum number of iterations is reached. Calculate the average of the area under multiple ROC curves and the average of multiple average inverse ranking values ​​as the final prediction result, and complete the training of the link prediction model. Step 7: Input the heterogeneous network data to be predicted into the trained link prediction model for link prediction.

2. The static heterogeneous network link prediction method based on graph attention network according to claim 1, characterized in that, Step two describes using a preset meta-path on edges of type [missing information]. Random walks on heterogeneous graphs to generate a set of paths include: [The text abruptly ends here, likely due to an incomplete sentence or a formatting error.] The system is divided into multiple layers based on edge type. On each layer, a random walk is performed based on a preset meta-path to generate a path set. .

3. The static heterogeneous network link prediction method based on graph attention network according to claim 1, characterized in that, Step three, which describes learning node structure embeddings by performing skip-gram on the node sequence generated by the random walk, includes: First, the negative sampling optimization objective function is calculated. : ； In the formula, yes Activation function; It is the number of negative samples corresponding to the positive training samples; This represents the basic embedding of node i on edge type r. This represents the basic embedding of node j on edge type r; Indicates from node Corresponding node set The noise distribution defined above Randomly selected from The function; From node Corresponding node set The noise distribution defined above Randomly selected from; Then, concatenate the target node. Basic embeddings of various edge types Obtain node structure embedding : ； In the formula, This indicates a splicing operation.

4. The static heterogeneous network link prediction method based on graph attention network according to claim 3, characterized in that, Step three, which involves learning edge type information in heterogeneous graphs through graph attention network layers, includes: First, attention scores are computed using edge-type embeddings and node-structure embeddings. : ； In the formula, Indicates the activation function; ; Indicates edge type characteristics; This indicates the structural embedding of node j; This represents the structural embedding of node k; Indicates the type of the edge between node i and node j; and It is a learnable mapping matrix used to map node embeddings and edge type embeddings; Represents a node The set of neighbors; Then, in attention scores Add residual connections to obtain the overall attention coefficient. : ； In the formula, hyperparameters ∈[0,1] is the scaling factor; Represents a node For nodes Attention score in the l-th layer of the network; Then, based on the comprehensive attention coefficient Aggregate the l-th layer network: ； In the formula, Represents a node Embedded in the structure of the network at layer l. This represents the trainable mapping matrix of the l-th layer network; Finally, structural embedding Perform L2 normalization to obtain the output embedding.

5. The static heterogeneous network link prediction method based on graph attention network according to claim 4, characterized in that, Step three describes determining whether there is a type of [missing information] between nodes based on the output vector of the target node. The edges are predicted according to the following formula: In the formula, It is an activation function; It is a learnable square matrix of edge type 𝑟; and These represent the output embeddings of nodes u and v, respectively. This represents the probability that there is an edge of type r between nodes u and v.

6. The static heterogeneous network link prediction method based on graph attention network according to claim 5, characterized in that, In step three, the loss value is calculated using the following formula: ； In the formula, It's a hyperparameter; , It is the set of target links to be predicted. It is a link The probability of existence The label indicates whether the link is a positive or negative sample. 1 indicates a positive sample and the link exists, while 0 indicates a negative sample and the link does not exist.

7. A static heterogeneous network link prediction system based on graph attention networks, characterized in that, include: The data acquisition module is configured to acquire training and testing datasets for heterogeneous networks. The datasets include heterogeneous graphs. ,in It is a set of nodes. It is an edge set. It is a set of node types. It is a set of edge types; The data preprocessing module is configured to utilize preset meta-paths on edges of type [missing information]. Random walks are performed on the heterogeneous graph to generate a set of paths, and sample batches are obtained from the set of paths; The prediction model training module is configured to train the link prediction model based on the sample batches corresponding to the training dataset. For each training sample batch, the following steps are performed: First, the basic embedding of the target node is obtained by generating shared embeddings and aggregating the edge embeddings of neighbors, including: using a mean aggregator to aggregate the edge embeddings of the target node. The type is On the layer The edge embeddings of the order neighbors are aggregated by mean to obtain the edge embeddings of that edge type. : ； In the formula, It is an activation function. It is a mapping matrix; Indicates mean aggregation, It is a node In type The set of neighbors on the edge; Represents a node In edge type Neighboring nodes on the layer The Edge embedding; edge embedding that combines all edge types Get the target node edge feature matrix The edge feature matrix is ​​computed using a self-attention mechanism. In edge type coefficients of a linear combination of vectors on : ； In the formula, It is an activation function. and They are of type Trainable parameters on the edges, superscript Represents the transpose of a vector or matrix; based on coefficients Edge embedding matrix Perform weighted summation and integrate shared embedded data. Obtain the target node Basic embeddings on various edge types : ； In the formula, Represents a node The shared embeddings are randomly initialized by the model; It is a hyperparameter that represents the importance of edge embeddings to the overall embedding; express The One element; Represents a node Let R represent the edge embeddings of type p on the edges; R represents the total number of edge types; then, based on the node sequence generated by the random walk, skip-gram is performed on the node sequence to learn the node structure embeddings, so as to obtain the semantic relationships and structural features between nodes of various types in the heterogeneous graph; then, edge type information in the heterogeneous graph is learned through a graph attention network layer to obtain an output vector that integrates structural information and edge type information; then, based on the output vector of the target node, the presence or absence of type p between nodes is determined. The edges are predicted to obtain the predicted probabilities; then, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters; Input the batches of samples corresponding to the test dataset into the trained link prediction model above for prediction, and obtain the prediction probability of each batch; calculate the area under the ROC curve and the average reciprocal rank value of the prediction probability corresponding to the test dataset. Repeat the above training and testing steps iteratively until the maximum number of iterations is reached. Calculate the average of the area under multiple ROC curves and the average of multiple average inverse ranking values ​​from multiple iterations as the final prediction result, thus completing the training of the link prediction model. The link prediction module is configured to input heterogeneous network data to be predicted into a trained link prediction model for link prediction.

Images