Static heterogeneous network link prediction method and system based on graph attention network

By applying a graph attention network in heterogeneous network, combining random walk and Skip-gram to integrate edge type information, the problem of structural information capture and excessive smoothing in heterogeneous network link prediction is solved, and more accurate links are achieved Predict the effect.

CN119004255AActive Publication Date: 2024-11-22HARBIN INST OF TECH

Patent Information

Application Number
CN202411043419.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2024-07-31
Publication Date
2024-11-22
Estimated Expiration
2044-07-31

AI Technical Summary

Technical Problem

The existing technology lacks comprehensive consideration of network structure, node attributes and link attributes in heterogeneous network link prediction, resulting in limitations in prediction results. Graph neural networks face excessive smoothing and gradient vanishing problems when capturing higher-order features and deep learning. It is difficult to effectively capture the complex structural information of heterogeneous networks.

Method used

提出一种基于图注意力网络的静态异构网络链路预测方法,通过随机游走生成路径集合,使用共享嵌入和聚合邻居边嵌入获取节点嵌入,结合Skip-gram学习节点结构嵌入,并通过图 Attention network layer combines edge type information, uses the Adam optimizer to update parameters, calculate the area value under the ROC curve and the average reciprocal ranking value to optimize the model.

Benefits of technology

The accuracy and model performance of link prediction are significantly improved, and the effectiveness of the overall node embedding generation module, heterogeneous structure information extraction module and edge type information fusion module of the AHGNN model is better than the six classic baseline models, improving The effect of heterogeneous network link prediction is achieved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN119004255A_ABST
    Figure CN119004255A_ABST
Patent Text Reader

Abstract

The invention discloses a static heterogeneous network link prediction method and system based on a graph attention network, and relates to the technical field of network link prediction. According to the technical key points, the method comprises the steps of obtaining a heterogeneous network training data set and a test data set, wherein the data sets comprise heterogeneous graphs; randomly walking on the heterogeneous graph of which the edge type is r by utilizing a preset meta-path to generate a path set, and obtaining a sample batch from the path set; training a link prediction model based on the training data set and the test data set; and inputting to-be-predicted heterogeneous network data into the trained link prediction model for link prediction. The static heterogeneous network link prediction method has remarkable advantages in static heterogeneous network link prediction tasks.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the technical field of network link prediction, and in particular to a static heterogeneous network link prediction method and system based on a graph attention network. Background Art

[0002] With the popularization of the Internet and the rise of various network applications, network science has become an important research field. In this field, link prediction is an important problem, that is, predicting whether there is a link between two nodes in the network. The solution to this problem is of great significance for optimizing network resource allocation, social network analysis, etc.

[0003] Networks in the real world are often heterogeneous, including different types of nodes and edges. For example, social networks, biological protein networks, and computer networks contain various types of nodes, attributes, and relationships. Therefore, when performing link prediction on real heterogeneous networks, it is necessary to consider the characteristics of different types of nodes and links in heterogeneous networks, as well as the complex associations between them, and to fully explore the network structure and attribute information.

[0004] At present, traditional link prediction methods have some shortcomings. They often lack comprehensive consideration of network structure, node attributes, link attributes and other information, resulting in prediction results limited to the influence of partial information. Heterogeneous networks contain multiple types of nodes and multiple types of edges. Link prediction on heterogeneous networks has multiple challenges, and each node pair can have multiple different types of relationships. It is very important to be able to borrow advantages from different relationships and learn a unified embedding. Among the link prediction methods for static heterogeneous networks, the method based on graph attention network-GNN has a good effect. In the GNN-based method, the number of GNN layers is too small to capture the high-order features of the network; as the number of layers of the GNN network increases, the training complexity increases, resulting in over-smoothing problems, resulting in reduced performance. In the random walk-based method, although the neighbors can be sampled flexibly, the neighbor types and relationship types in the heterogeneous graph are complex, which is prone to sampling imbalance. In addition, heterogeneous networks contain complex and diverse relationships. How to extract effective information from the various relationships in heterogeneous networks? Due to the over-smoothing and gradient vanishing problems, graph neural networks (GNNs) are difficult to become very deep. How to reasonably increase the number of GNN layers and how to capture high-order and low-order structural information of heterogeneous networks with complex structures pose challenges to heterogeneous network link prediction. The attention mechanism is widely used in deep learning. By dynamically adjusting the degree of attention to the input, the model's expressiveness and prediction performance can be improved. In the link prediction problem, the introduction of the attention mechanism can help the network model better capture the correlation between nodes and improve the accuracy of link prediction. Summary of the invention

[0005] In view of the above problems, the present invention proposes a static heterogeneous network link prediction method and system based on graph attention network to solve the problem of heterogeneous network link prediction.

[0006] According to one aspect of the present invention, a static heterogeneous network link prediction method based on a graph attention network is proposed, and the method comprises the following steps:

[0007] Step 1: Obtain a heterogeneous network training data set and a test data set, where the data set includes a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, where V is a node set, E is an edge set, φ is a node type set, and ψ is an edge type set;

[0008] Step 2: Use the preset meta-path to perform random walks on the heterogeneous graph with edge type r to generate a path set, and obtain a sample batch from the path set;

[0009] Step 3: Train the link prediction model based on the sample batches corresponding to the training data set. Perform the following steps for each training sample batch:

[0010] First, the base embedding of the target node is obtained by generating a shared embedding and aggregating the edge embeddings of neighbors;

[0011] Then, according to the node sequence generated by random walk, skip-gram is performed on the node sequence to learn the node structure embedding to obtain the semantic relationship and structural features between multiple types of nodes in the heterogeneous graph;

[0012] Then, the edge type information in the heterogeneous graph is learned through the graph attention network layer to obtain an output vector that combines structural information and edge type information;

[0013] Then, based on the output vector of the target node, predict whether there is an edge of type r between the nodes and obtain the predicted probability;

[0014] Then, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters;

[0015] Step 4: Input the sample batches corresponding to the test data set into the link prediction model trained in step 3 for prediction, and obtain the prediction probability of each batch;

[0016] Step 5: Calculate the area under the ROC curve and the average reciprocal ranking value of the predicted probability corresponding to the test data set;

[0017] Step 6: Iterate and repeat steps 3 to 5 until the maximum number of iterations is reached, calculate the average of multiple area values ​​under the ROC curves of multiple iterations and the average of multiple average reciprocal ranking values ​​as the final prediction result, and complete the training of the link prediction model;

[0018] Step 7: Input the heterogeneous network data to be predicted into the trained link prediction model for link prediction.

[0019] Furthermore, the step 2 of performing random walks on a heterogeneous graph with edge type r using a preset meta-path to generate a path set includes: dividing the heterogeneous graph G into multiple layers according to the edge type, performing random walks on each layer according to the preset meta-path, and generating a path set P r .

[0020] Furthermore, the step 3 of obtaining the base embedding of the target node by generating the shared embedding and the edge embedding of the aggregated neighbors includes:

[0021] First, use the mean aggregator to aggregate the target node v i The edge embeddings of the k-order neighbors on the layer of type r are averaged and aggregated to obtain the edge embedding of this edge type.

[0022]

[0023] Where σ is the activation function, is the mapping matrix; mean represents mean aggregation, N i,r is node v i The set of neighbors on the edge of type r; Represents node v i Neighbor node v on layer with edge type r j The k-1 order edge embedding of;

[0024] Then, combine the edge embedding u of all edge types i,r Get the target node v i The edge feature matrix U i ;

[0025] Then, the self-attention mechanism is used to calculate the edge feature matrix U i coefficient a of the linear combination of vectors on edge type r i,r :

[0026]

[0027] In the formula, softmax is the activation function, w r and W r They are the trainable parameters on the edge of type r, and the superscript T represents the transpose of a vector or matrix;

[0028] Then, based on the coefficient a i,r The edge embedding matrix U i Perform weighted summation and integrate the shared embedding b i Get the target node v iBasic embedding z on each edge type i,r :

[0029]

[0030] Where b i Represents node v i The shared embedding of ξ is randomly initialized by the model; r is a hyperparameter indicating the importance of edge embedding to the overall embedding; p Indicates a i,r The pth element of i,p Represents node v i is the edge embedding of edges of type p; R represents the total number of edge types.

[0031] Furthermore, in step 3, performing skip-gram on the node sequence to learn the node structure embedding according to the node sequence generated by random walk includes:

[0032] First, calculate the negative sampling optimization objective function E1:

[0033]

[0034] Where σ is the sigmoid activation function; L is the number of negative samples corresponding to positive training samples; z i,r represents the basic embedding of node i on edge type r, z j,r represents the basic embedding of node j on edge type r; Represents the slave node v j The corresponding node set V t The noise distribution P defined above t (v) randomly select v k Function of v k is from node v j The corresponding node set V t The noise distribution P defined above t (v) randomly selected;

[0035] Then, concatenate the target node v i The basic embedding z of each edge type i,r Get node structure embedding

[0036]

[0037] In the formula, ∥ represents the concatenation operation; R represents the total number of edge types.

[0038] Furthermore, the step 3 of learning edge type information in a heterogeneous graph through a graph attention network layer includes:

[0039] First, the attention score is calculated using edge type embedding and node structure embedding

[0040]

[0041] In the formula, LeakyReLU represents the activation function; α represents the learnable mapping vector; r ψ(<i,j>) Indicates edge type characteristics; h j represents the structural embedding of node j; h k represents the structural embedding of node k; ψ(<i,j> ) represents the type of edge between node i and node j; W and W r is a learnable mapping matrix used to map node embeddings and edge type embeddings; N i Represents node v i The set of neighbors of ;

[0042] Then, in the attention score Add residual connection to obtain comprehensive attention coefficient

[0043]

[0044] Where, the hyperparameter β∈[0,1] is the scaling factor; Represents node v i For node v j The attention score at the lth layer of the network;

[0045] Then, according to the comprehensive attention coefficient Aggregate the layer l network:

[0046]

[0047] In the formula, Represents node v i In the structure embedding of the l-th layer network, W (l) Represents the trainable mapping matrix of the l-th layer network;

[0048] Finally, the structure is embedded Perform L2 normalization to get the output embedding.

[0049] Furthermore, in step 3, the output vector of the target node is used to predict whether there is an edge of type r between the nodes according to the following formula:

[0050]

[0051] In the formula, sigmoid is the activation function; R r is a learnable matrix of edge type r; o uand v Represent the output embedding of node u and node v respectively; Prob r represents the probability that there is an edge of type r between node u and node v.

[0052] Furthermore, in step 3, the loss value is calculated according to the following formula:

[0053]

[0054] Where μ is a hyperparameter; L t is the set of target links to be predicted, p l is the probability that link l exists, y l ∈{0,1} is a label indicating whether the link is a positive or negative sample. 1 indicates that it is a positive sample and the link exists, and 0 indicates that it is a negative sample and the link does not exist.

[0055] According to another aspect of the present invention, a static heterogeneous network link prediction system based on a graph attention network is proposed, the system comprising:

[0056] A data acquisition module is configured to acquire a heterogeneous network training data set and a test data set, wherein the data set includes a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, wherein V is a node set, E is an edge set, φ is a node type set, and ψ is an edge type set;

[0057] A data preprocessing module, configured to perform a random walk on a heterogeneous graph with an edge type of r using a preset meta-path to generate a path set, and obtain a sample batch from the path set;

[0058] The prediction model training module is configured to train the link prediction model based on the sample batches corresponding to the training data set, and performs the following steps for each training sample batch: first, the basic embedding of the target node is obtained by generating the shared embedding and the edge embedding of the aggregated neighbors; then, according to the node sequence generated by the random walk, the node sequence is skip-gram-performed to learn the node structure embedding, so as to obtain the semantic relationship and structural features between multiple types of nodes in the heterogeneous graph; then, the edge type information in the heterogeneous graph is learned through the graph attention network layer to obtain the output vector that integrates the structural information and the edge type information; then, according to the output vector of the target node, whether there is an edge of type r between the nodes is predicted to obtain the prediction probability; then, the loss value is calculated, and the optimizer uses Adam to perform back-propagation to update the parameters;

[0059] Input the sample batches corresponding to the test data set into the above-mentioned trained link prediction model for prediction, and obtain the prediction probability of each batch; calculate the area under the ROC curve and the average reciprocal ranking value of the prediction probability corresponding to the test data set;

[0060] Iterate and repeat the above training and testing steps until the maximum number of iterations is reached, calculate the average of multiple area values ​​under the ROC curves and the average of multiple average reciprocal ranking values ​​of multiple iterations as the final prediction result, and complete the training of the link prediction model;

[0061] The link prediction module is configured to input the heterogeneous network data to be predicted into the trained link prediction model to perform link prediction.

[0062] The beneficial technical effects of the present invention are:

[0063] The present invention proposes a static heterogeneous network link prediction method and system based on a graph attention network, the method comprising the following steps: obtaining a heterogeneous network training data set and a test data set, the data set comprising a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, wherein V is a node set, E is an edge set, φ is a node type set, and ψ is an edge type set; using a preset meta-path to perform a random walk on a heterogeneous graph with an edge type of r to generate a path set, and obtaining a sample batch from the path set; training a link prediction model based on the sample batch corresponding to the training data set; inputting the sample batch corresponding to the test data set into the trained link prediction model for prediction, and obtaining the prediction probability of each batch; calculating the area under the ROC curve and the average reciprocal ranking value of the prediction probability corresponding to the test data set; iteratively repeating until the maximum number of iterations is reached, calculating the average value of multiple area under the ROC curve values ​​of multiple iterations and the average value of multiple average reciprocal ranking values ​​as the final prediction result, and completing the training of the link prediction model; inputting the heterogeneous network data to be predicted into the trained link prediction model for link prediction. The present invention has conducted comparative experiments with six classic frontier baseline models on four public data sets, and carried out parameter comparison experiments and ablation experiments on the models. The results of the baseline model comparison experiments show that the present invention has significant advantages over other baseline methods. The ablation experiments verify the effectiveness of the overall node embedding generation module, heterogeneous structure information extraction module and edge type information fusion module of the AHGNN model. The parameter comparison experiments fully investigate the impact of edge embedding dimension, edge type embedding dimension and path length of the Skip-gram algorithm on the AHGNN model. BRIEF DESCRIPTION OF THE DRAWINGS

[0064] The present invention can be better understood by referring to the description given below in conjunction with the accompanying drawings, which together with the following detailed description are included in this specification and form a part of this specification, and are used to further illustrate the preferred embodiments of the present invention and explain the principles and advantages of the present invention.

[0065] Figure 1 It is a framework diagram of a static heterogeneous network link prediction method based on a graph attention network described in an embodiment of the present invention.

[0066] Figure 2 1 is an example diagram of the influence of parameter values ​​on AUC in an embodiment of the present invention; wherein, (a) corresponds to the edge embedding dimension of the Amazon dataset; (b) corresponds to the edge type embedding dimension of the Amazon dataset; (c) corresponds to the skip-gram algorithm path length of the Amazon dataset; (d) corresponds to the edge embedding dimension of the LastFM dataset; (e) corresponds to the edge type embedding dimension of the LastFM dataset; (f) corresponds to the skip-gram algorithm path length of the LastFM dataset; (g) corresponds to the edge embedding dimension of the PubMed dataset; (h) corresponds to the edge type embedding dimension of the PubMed dataset; (i) corresponds to the skip-gram algorithm path length of the PubMed dataset; (j) corresponds to the edge embedding dimension of the YouTube dataset; (k) corresponds to the edge type embedding dimension of the YouTube dataset; (l) corresponds to the skip-gram algorithm path length of the YouTube dataset.

[0067] Figure 3 : is an example diagram of the impact of parameter values ​​on MRR in an embodiment of the present invention; wherein, (a) corresponds to the edge embedding dimension of the Amazon dataset; (b) corresponds to the edge type embedding dimension of the Amazon dataset; (c) corresponds to the skip-gram algorithm path length of the Amazon dataset; (d) corresponds to the edge embedding dimension of the LastFM dataset; (e) corresponds to the edge type embedding dimension of the LastFM dataset; (f) corresponds to the skip-gram algorithm path length of the LastFM dataset; (g) corresponds to the edge embedding dimension of the PubMed dataset; (h) corresponds to the edge type embedding dimension of the PubMed dataset; (i) corresponds to the skip-gram algorithm path length of the PubMed dataset; (j) corresponds to the edge embedding dimension of the YouTube dataset; (k) corresponds to the edge type embedding dimension of the YouTube dataset; (l) corresponds to the skip-gram algorithm path length of the YouTube dataset. DETAILED DESCRIPTION

[0068] In order to enable those skilled in the art to better understand the scheme of the present invention, exemplary implementations or embodiments of the present invention will be described below in conjunction with the accompanying drawings. Obviously, the described implementations or embodiments are only implementations or embodiments of a part of the present invention, not all of them. Based on the implementations or embodiments of the present invention, all other implementations or embodiments obtained by ordinary technicians in the field without creative work should fall within the scope of protection of the present invention.

[0069] The present invention proposes a static heterogeneous network link prediction method and system based on graph attention network, and names the proposed link prediction model as: AHGNN, to capture rich node information, edge type information and structural information to learn a unified embedding from different relationships, and utilize multiple topological structures from different node types for link prediction.

[0070] The embodiment of the present invention proposes a static heterogeneous network link prediction method based on a graph attention network, the method comprising the following steps:

[0071] Step 1: Obtain a heterogeneous network training data set and a test data set, where the data set includes a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, where V is a node set, E is an edge set, φ is a node type set, and ψ is an edge type set;

[0072] Step 2: Use the preset meta-path to perform random walks on the heterogeneous graph with edge type r to generate a path set, and obtain a sample batch from the path set;

[0073] Step 3: Train the link prediction model based on the sample batches corresponding to the training data set. Perform the following steps for each training sample batch:

[0074] First, the base embedding of the target node is obtained by generating a shared embedding and aggregating the edge embeddings of neighbors;

[0075] Then, according to the node sequence generated by random walk, skip-gram is performed on the node sequence to learn the node structure embedding to obtain the semantic relationship and structural features between multiple types of nodes in the heterogeneous graph;

[0076] Then, the edge type information in the heterogeneous graph is learned through the graph attention network layer to obtain an output vector that combines structural information and edge type information;

[0077] Then, based on the output vector of the target node, predict whether there is an edge of type r between the nodes and obtain the predicted probability;

[0078] Then, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters;

[0079] Step 4: Input the sample batches corresponding to the test data set into the link prediction model trained in step 3 for prediction, and obtain the prediction probability of each batch;

[0080] Step 5: Calculate the area under the ROC curve and the average reciprocal ranking value of the predicted probability corresponding to the test data set;

[0081] Step 6: Iterate and repeat steps 3 to 5 until the maximum number of iterations is reached, calculate the average of multiple area values ​​under the ROC curves of multiple iterations and the average of multiple average reciprocal ranking values ​​as the final prediction result, and complete the training of the link prediction model;

[0082] Step 7: Input the heterogeneous network data to be predicted into the trained link prediction model for link prediction.

[0083] The method starts from step 1. In step 1, a heterogeneous network training data set and a test data set are obtained, and the data sets include heterogeneous graphs.

[0084] According to an embodiment of the present invention, a heterogeneous graph G = {V, E, φ, ψ}, where V is a node set and E is an edge set. Each node v has a type φ(v), and each edge e has a type ψ(e). The sets of node types and edge types are represented by and Indicates that |T v |+|T e |>1.

[0085] Then, step 2 is executed. In step 2, a preset meta-path is used to perform a random walk on a heterogeneous graph with an edge type of r to generate a path set, and a sample batch is obtained from the path set.

[0086] According to an embodiment of the present invention, the heterogeneous graph G = {V, E, φ, ψ} is divided into |T e | layers, and then perform random walks on each layer according to the meta-path. For example, in layer G with edge type r r =(V,E r ), given a meta-path T: V1→V2→…V t →…V l , where l is the length of the meta-path, and n random walks are performed to generate a path set P r , then from P r Get the training samples {(v i ,v j ,r)}.

[0087] Then, step 3 is executed. In step 3, the link prediction model is trained based on the sample batches corresponding to the training data set. The following steps are executed for each training sample batch:

[0088] First, the base embedding of the target node is obtained by generating a shared embedding and aggregating the edge embeddings of neighbors;

[0089] Then, according to the node sequence generated by random walk, skip-gram is performed on the node sequence to learn the node structure embedding to obtain the semantic relationship and structural features between multiple types of nodes in the heterogeneous graph;

[0090] Then, the edge type information in the heterogeneous graph is learned through the graph attention network layer to obtain an output vector that combines structural information and edge type information;

[0091] Then, based on the output vector of the target node, predict whether there is an edge of type r between the nodes and obtain the predicted probability;

[0092] Then, the loss value is calculated and the optimizer uses Adam to perform backpropagation to update the parameters.

[0093] According to an embodiment of the present invention, Figure 1 As shown in the figure, the link prediction model AHGNN includes four parts: overall node embedding generation module, heterogeneous structure information extraction module, edge type information fusion module and result prediction module. The overall node embedding generation module is used to splice shared embedding and edge embedding, and learn a unified embedding from different relationships; the heterogeneous structure information extraction module is used to obtain low-order and high-order structural features of heterogeneous networks; the edge type information fusion module introduces node residual connections to fuse edge type information and solve the problems of over-smoothing and gradient disappearance in GNN training.

[0094] 1) Overall node embedding generation module: obtains the base embedding of the target node by generating shared embeddings and edge embeddings of aggregated neighbors; specifically, initializes the shared embedding b of the nodes in the heterogeneous graph i and the edge embedding matrix U i , mean aggregate first-order neighbors U i , using the self-attention mechanism for U i Perform a weighted sum and then add b i Get the basic embedding z of the node on each type of edge i,r .

[0095] Network embedding refers to the generation of low-dimensional embedding by modeling topological structures and node attribute information. Through network embedding, complex network structures can be converted into easy-to-process vector representations, thereby facilitating link prediction. Figure 1 In a heterogeneous network, node v i The overall embedding on the edge of type r is divided into two parts: shared embedding and edge embedding. i The shared embedding is shared between different edge types, that is, the information carried by the node when the heterogeneous graph removes the edge type and the node type is converted into a homogeneous graph. i Edge embedding of edges of type r Aggregated by the edge embeddings of neighbors:

[0096]

[0097] Among them, d u is the dimension of edge embedding, N i,r is node v i The set of neighbors on the edge of type r, for each node v i and the initial edge embeddings for each edge type is randomly generated by the model. The aggregator function uses the mean aggregator as follows:

[0098]

[0099] Where σ is the activation function, is the mapping matrix, combining nodes v i All types of edges are embedded as nodes v i The edge embedding matrix where d u is the dimension of edge embedding, |T e | is the number of edge types; mean represents mean aggregation, Represents node v i Neighbor node v on layer with edge type r j The k-1th order edge embedding of ;

[0100] Combine the edge embedding u of all edge types i,r Get the target node v i The edge feature matrix U i :

[0101]

[0102] The mean aggregator can be converted to a matrix multiplication:

[0103]

[0104] Among them, N r ∈R |V|×|V| is the normalized adjacency matrix of edges of type r, express The i-th column, U r = {u 1,r ,u 2,r ,u 3,r …u m,r} is the edge embedding set of all nodes on this type of edge, Where |V| represents the number of nodes in the graph.

[0105] Use the self-attention mechanism to calculate U i The coefficients of the linear combination of vectors in for:

[0106]

[0107] in, and are the trainable parameters on the edge of type r, the superscript T represents the transpose of a vector or matrix, and a i,r Represents the weight of each type of edge embedding. Softmax is the activation function.

[0108] Therefore, node v i The basic embedding on type r edges is:

[0109]

[0110] in, Represents node v i The shared embedding of is randomly initialized by the model, ξ r is a hyperparameter that represents the importance of edge embedding to the overall embedding, p Indicates a i,r The pth element of

[0111]

[0112] u i,p Represents node v i is the edge embedding of edges of type p; R represents the total number of edge types.

[0113] 2) Heterogeneous structural information extraction module: Based on the node sequence generated by random walk, skip-gram is performed on the node sequence to learn the node structure embedding to obtain the semantic relationship and structural features between multiple types of nodes in the heterogeneous graph; specifically, the Skip-Gram algorithm is used on the node sequence obtained based on the meta-path random walk to obtain the structural information of the heterogeneous network and optimize the basic embedding information z i,r , embed the basis of each type of edge of the node into z i,r Perform splicing to obtain the node structure embedding h i .

[0114] We use random walks to generate node sequences and then perform skip-gram on the node sequences to learn embeddings. Since each view of the input network is heterogeneous, we use meta-path based random walks. Specifically, given a network view r, G r =(V,E r ) and meta-path T: V1→V2→…V t →…V l , l is the length of the meta-path, and the transition probability at step t is defined as follows:

[0115]

[0116] Among them, v i ∈V t , N i,r Represents node v i In view G r The random walk strategy based on the meta-path ensures that the semantic relationship between nodes of different types can be correctly incorporated into the Skip-Gram model.

[0117] Suppose a random walk of length l on edges of type r follows a path Context where c is the radius of the window size. So, given a node v i and its path context C, the goal of negative sampling is to minimize the following negative log-likelihood:

[0118]

[0119] Among them, θ represents all the above parameters. A heterogeneous softmax function is used, which is based on the node v j Specifically, given v i Time j The probability definition of is as follows, where v j ∈V t ,

[0120]

[0121] Finally, the node v i Negative sampling objective function -logP θ (v j |v i ) is approximately:

[0122]

[0123] Where σ(x) = 1 / (1+exp(-x)) is the sigmoid function, L is the number of negative samples corresponding to positive training samples, and v k is from node v j The corresponding node set V t The noise distribution P defined above t (v) is randomly selected. i,r represents the basic embedding of node i on edge type r, z j,r represents the basic embedding of node j on edge type r; Represents the slave node v j The corresponding node set Vt The noise distribution P defined above t (v) randomly select v k Function of v k is from node v j The corresponding node set V t The noise distribution P defined above t (v) randomly selected.

[0124] Therefore, the objective function can update the neighbor sets of each order of nodes in the heterogeneous graph in each type of network view, so as to capture more semantic information between different node types, making the structural information more complete and the node information more perfect, so that the model is more balanced during the training process.

[0125] 3) Edge type information fusion module: The edge type information in the heterogeneous graph is learned through the graph attention network layer to obtain an output vector that combines the structural information and edge type information; specifically, the edge type feature r is initialized ψ(<i,j>) , through the structure of the node embedding h i and edge type feature r ψ(<i,j>) Calculate the attention coefficient, add residual connections on this basis, aggregate neighborhood structure features, and output the final embedding of each node after L2 normalization. i .

[0126] Although GAT has powerful representation ability in modeling homogeneous graphs, its performance on heterogeneous graphs may not be optimal due to its ignorance of node or edge types. To address this issue, the model extends the original graph attention mechanism by incorporating edge type information into the attention calculation. Specifically, at each layer, for each edge type ψ(e)∈T e Assign edge type embedding And use node embedding and edge type embedding to calculate attention score As shown below:

[0127]

[0128] Among them, ψ(<i,j> ) represents the type of edge between node i and node j, and is a learnable mapping matrix used to map node embedding and edge type embedding. LeakyReLU represents the activation function; α represents the learnable mapping vector; r ψ(<i,j>) Indicates edge type characteristics; h j represents the structural embedding of node j; h k represents the structural embedding of node k; N i Represents node v i The set of neighbors.

[0129] In order to solve the problem of over-smoothing and gradient vanishing of GNN, the AHGNN model introduces residual connections. The cross-layer node representation adds pre-activated residual connections. The aggregation of the lth layer can be expressed as:

[0130]

[0131] in, is node v i and v j The attention weight of the edge between them, σ is an activation function, It is formed by concatenating the basic embeddings of nodes on all types of edges:

[0132]

[0133] therefore The dimension d h =d u ×|T e |. Represents node v i In the structure embedding of the l-th layer network, W (l) Represents the trainable mapping matrix of the l-th layer network.

[0134] The residual connection on the attention score can effectively solve the problem of over-smoothing and gradient disappearance. The original attention score is obtained by formula (12): Finally, add the residual connection:

[0135]

[0136] where the hyperparameter β∈[0,1] is a scaling factor. Represents node v i For node v j The attention score at layer l of the network.

[0137] Similar to GAT, the algorithm uses multi-head attention to enhance the expressiveness of the model. Specifically, K independent attention mechanisms are executed according to formula (12), and their results are concatenated as the final representation. The corresponding update rule is:

[0138]

[0139]

[0140]

[0141] Among them, ∥ represents the splicing operation, is the lth linear transformation The attention score calculated according to equation (15). Usually the output size cannot be divided exactly by the number of heads. After GAT, no concatenation is performed, but the average representation is used in the final (L) layer.

[0142]

[0143] Perform L2 normalization on the output embedding:

[0144]

[0145] in, is node v i The L2-normalized output embedding of is the final expression of formula (19).

[0146] 4) Result prediction module: predict whether there is an edge of type r between nodes based on the output vector of the target node, obtain the prediction probability, and use the DistMult decoder to parse the embedding to obtain the prediction result.

[0147] Specifically, the final output embedding o of node i is obtained through the above steps: i , link prediction requires decoding of embeddings. The performance of DistMult is better than direct dot product. For whether there is an edge of type r between node pairs u and v, the prediction formula is as follows:

[0148]

[0149] Among them, Prob r represents the probability that there is an edge of type r between node u and node v; is a learnable matrix of edge type r, where d o =Kd h ;o u and v Represent the output embedding of node u and node v respectively. Link prediction loss function Using formula (22) it is expressed as:

[0150]

[0151] Among them, L t is the set of target links to be predicted, p l is the probability that link l exists, y l ∈{0,1} is the label indicating whether the link is a positive or negative sample, 1 indicates that it is a positive sample and the link exists, and 0 indicates that it is a negative sample and the link does not exist. Then the total loss function is It can be expressed by formula (23), where μ is a hyperparameter:

[0152]

[0153] Then execute step 4. In step 4, the sample batches corresponding to the test data set are input into the link prediction model trained in step 3 for prediction to obtain the prediction probability of each batch.

[0154] Then, step five is executed, in which the area under the ROC curve and the average reciprocal ranking value of the predicted probability corresponding to the test data set are calculated.

[0155] According to an embodiment of the present invention, AUC is a commonly used evaluation index, which measures the ability of a classifier to distinguish between positive samples and negative samples. MRR is an index for evaluating model performance based on the link ranking results predicted by the model. The link prediction problem refers to predicting whether there is a connection between two nodes in a network, which can be regarded as a binary classification problem. There are two types of sample labels for the link prediction problem: positive samples and negative samples, and there are two types of prediction results: connected and unconnected. According to the classification of sample labels and prediction results, the prediction results can be divided into four groups, represented by a confusion matrix, as shown in Table 1, where rows represent prediction values ​​and columns represent label values.

[0156] Table 1 Confusion matrix of link prediction task

[0157]

[0158] True Positives (TP): The number of positive samples that are correctly predicted to be connected.

[0159] False Positives (FP): The number of positive samples that are incorrectly predicted as having no connection.

[0160] False Negatives (FN): The number of negative samples that are incorrectly predicted to be connected.

[0161] True Negatives (TN): The number of negative samples that are correctly predicted as having no connection.

[0162] It can be concluded that TP and TN are samples with accurate predictions, while FP and FN are samples with incorrect predictions.

[0163] AUC is the area under the ROC curve. In the link prediction model, to make the ROC curve, we first need to sort the probability values ​​of the prediction results of each sample that are connected in descending order, and then combine the probability values ​​and labels into a table. Starting from the highest probability value in the sorting, the probability value of each sample is regarded as a threshold one by one. For each such threshold, all samples above or equal to this threshold are considered positive samples, and samples below this threshold are considered negative samples. For each threshold, two key indicators need to be calculated: the true positive rate TPR and the false positive rate FPR. The calculation formula is as follows:

[0164]

[0165]

[0166] Using TPR as the vertical axis and FPR as the horizontal axis, we can get the ROC curve. AUC is the area under the ROC curve, ranging from 0.5 to 1. The closer the value is to 1, the better the performance of the model.

[0167] MRR is a standard metric for ranking, which treats link prediction as a link retrieval problem. Since exhaustive calculations for all node pairs are too cumbersome, the positive and negative samples of the target node are retrieved as a group. MRR is calculated by sorting the probability values ​​of the samples of the target node in descending order based on the link prediction results, finding the ranking of the first positive sample, calculating the reciprocal of the position, and then averaging the reciprocals of the rankings of the first positive samples of all nodes to obtain MRR, as shown in the formula:

[0168]

[0169] Where n is the number of nodes, rank i is the ranking of the first positive sample of the i-th node. The value range of MRR is between 0 and 1. The closer the value is to 1, the higher the prediction accuracy of the algorithm for the positive samples ranked in front, and the better the performance.

[0170] Then execute step six. In step six, iteratively repeat steps three to five until the maximum number of iterations is reached, and the average value of the area under the multiple ROC curves and the average value of the multiple average reciprocal ranking values ​​of multiple iterations are calculated as the final prediction result to complete the training of the link prediction model.

[0171] The goal of the AHGNN model is to capture effective structural information in static heterogeneous networks, including edge type information and neighborhood node information, and effective node information, including node type information, so as to complete the link prediction task. The pseudo code of the heterogeneous network link prediction algorithm based on the graph attention network is shown in Algorithm 1. The algorithm needs to input the meta-path T, the training set Train, the test set Test, the maximum number of iterations Epoch, and the size of each batch of training samples batchsize. During model training, the data of the training set is divided into multiple batches according to the batchsize, the model is iteratively trained, and the effect of the model is tested after each epoch training is completed.

[0172]

[0173]

[0174] Lines 1 to 3 of the algorithm describe the generation of training samples based on the random walk of meta-paths. First, the heterogeneous graph G = {V, E, φ, ψ} is divided into |T according to the type of edge ψ(e). e | layers, and then perform random walks on each layer according to the meta-path. For example, in layer G with edge type r r =(V,E r ), given a meta-path T: V1→V2→…V t →…V l , where l is the length of the meta-path, and n random walks are performed to generate a path set P r , then from P r Get the training samples {(v i ,v j ,r)}.

[0175] Lines 6 to 10 of the algorithm describe the generation of the overall node embedding module, which generates a shared embedding b i and the edge embedding u of the aggregated neighbors i,r Get the base embedding z of the target node i,r First, according to formula (2), the target node v i The edge embeddings of the k-order neighbors on the layer of type r are averaged and aggregated to obtain the edge embeddings of this type of edge In the implementation process, the length of the meta-path designed for the data set is 2, so the first-order and second-order neighbors of the target node can be obtained, that is, k = 2. From the formula, it can be seen that the parameters that need to be trained by back propagation are the mapping matrix Second, combine all types of edge embeddings u i,r Get node v i The edge feature matrix U i ; Then, according to formula (5), the self-attention mechanism is used to calculate U iThe coefficient α of the linear combination of vectors on edge type r i,r , from the formula, we can see that the parameters that need back propagation training are the weight matrix Finally, according to formula (6), the edge embedding matrix U i Perform weighted summation and integrate the shared embedding b i Get the base embedding z of the target node on each edge type i,r The overall node embedding module captures the basic information of nodes, including the shared information and neighborhood information of nodes.

[0176] Lines 11 to 15 of the algorithm correspond to the heterogeneous Skip-Gram module. According to the node sequence generated by random walk, skip-gram is performed on the node sequence to learn node embedding. First, based on the meta-path T, a path P with a length of l on the edge of type r and the context C of the target node is obtained by random walk; then, the next step transition probability p(v j |v i ,T), according to formula (11), we get the negative sampling optimization objective function E1, and the optimization objective function guides the basic embedding z i,r Finally, the basic embedding z of each edge type of the target node is concatenated according to formula (14) i,r Get node structure embedding As the input of the next module. This module has no parameters that need back-propagation training. The heterogeneous Skip-Gram module can capture the semantic relationship and structural features between various types of nodes in the heterogeneous graph.

[0177] Lines 16 to 20 of the algorithm describe the heterogeneous GAT module of AHGNN, which will learn the edge type information in the heterogeneous graph through the GAT layer. First, according to formula (12), the edge type is used to embed r ψ (e) and node structure embedding h i To calculate the attention score The parameters that need to be trained through back propagation in this formula are This module sets different numbers of GAT layers for sparse and dense networks. For sparse heterogeneous networks, three layers of GAT are set, that is, n = 3, and for dense heterogeneous networks, two layers of GAT are set, that is, n = 2. Secondly, in order to solve the problems of over-smoothing and gradient disappearance of GNN, the AHGNN model introduces residual connections and, according to formula (15), the original attention score is The algorithm uses multi-head attention to enhance the expressiveness of the model, so the set of parameters that need to be trained is updated to Where K is the number of attention heads. After learning this module, we can obtain the output vector o that integrates the structural information and edge type information.i .

[0178] Lines 21 to 22 of the algorithm correspond to the result prediction module, which uses formula (22) to predict whether there is an edge of type r between nodes based on the output vector of the target node. The parameters that need to be trained are In summary, the parameters that need to be trained in AHGNN are θ = {θ1, θ2, θ3, θ4}. Line 22 of the algorithm will calculate the Loss value, and the optimizer uses Adam to back-propagate and update θ.

[0179] Finally, step seven is executed. In step seven, the heterogeneous network data to be predicted is input into the trained link prediction model for link prediction. Specific embodiment 1

[0181] As shown in Table 2, the heterogeneous network dataset-Amazon product dataset includes product metadata and links between products. The present invention only uses product metadata of the electronic category, including product attributes and co-viewing and co-purchasing links between products. Product attributes include price, sales ranking, brand and category, including electronic products, the node type is product, and the edge type is the link between them with co-viewing and co-purchasing.

[0182] Table 2

[0183] Dataset Number of nodes Number of node types Number of edges Number of edge types Average Amazon 10099 1 148659 2 14.72

[0184] In heterogeneous networks of the same type as Amazon, the present invention can solve a variety of problems: optimize product recommendations, use users' co-viewing and co-purchasing data to recommend products that they may not have discovered but may be interested in, and increase product exposure and sales opportunities; identify potential new connections and predict new edges that may be formed in the future (such as new co-viewing or co-purchasing relationships) to help merchants identify market demand and user interests in advance; classify and cluster products, and identify which products are similar or can be classified into the same category by analyzing the connectivity between products, which can help improve product classification and display strategies; help formulate market strategies, and based on the link prediction results, carry out targeted marketing activities to improve marketing effectiveness.

[0185] When performing static heterogeneous network link prediction, the heterogeneous network data to be predicted is the heterogeneous graph of the current heterogeneous network. The trained link prediction model is used to perform link prediction. The current product nodes (product A, product B, etc.) and the known links between them (co-viewed and co-purchased edges) are known. Link prediction is performed for the current network structure, and the link prediction results include: predicting which products may form new co-viewed or co-purchased relationships, and assigning confidence scores to these potential links, identifying similar products or product groups that can be bundled for sale, and improving the effectiveness of product combinations. The present invention can optimize product recommendations and improve sales conversion rates, thereby gaining an advantage in a highly competitive market. Specific embodiment 2

[0187] As shown in Table 3, the heterogeneous network dataset - LastFM is an online music website. The LastFM dataset in the literature [1] is used to preprocess the dataset by filtering out users and tags with only one link. This dataset comes from the Last.fm7 online music system. Its users are connected to each other in a social network generated by the Last.fm "friend" relationship. Each user has a list of the most popular music artists, tags, that is, a tuple [user, tag, artist], and friend relationships in the dataset social network. Each artist has a Last.fm URL and an image URL. The node types are user, tag, and music artist, and the edge types are user-music artist, user-tag, and music artist-tag.

[0188] Table 3

[0189] Dataset Number of nodes Number of node types Number of edges Number of edge types Average LastFM 20612 3 141521 3 6.87

[0190] In a heterogeneous network of the same type as LastFM, the heterogeneous network data to be predicted is the heterogeneous graph of the current heterogeneous network, and link prediction is performed using the trained link prediction model. Based on the current heterogeneous network structure, the following link prediction tasks can be completed, namely, the prediction results include: predicting user-music artist edges, predicting which users may be interested in certain music artists that they have not yet heard of; predicting user-label edges, predicting which labels users may like or add to their preferences; predicting artist-label edges, predicting which labels certain music artists may be associated with in order to better describe their music style. The present invention can solve the following problems: user recommendation, generating a recommended music artist list for each user, helping them discover new music based on their historical behavior and marked preferences; tag recommendation, providing a recommended list of relevant tags for each user, encouraging them to use these tags to describe their favorite music, and enhancing the interaction between users and the platform; artist tag relationship, providing corresponding tag recommendations for each music artist, helping them attract more listeners and optimize their performance on the platform; potential edge identification, identifying edges that have not yet been established in the current network but have potential connections, such as possible relationships between users and music artists, users and tags, so that the recommendation algorithm can be further optimized; user behavior pattern identification, analyzing user behavior patterns based on prediction results, identifying potential preferences, and providing data support for marketing activities or recommendation system adjustments. Through these predictions and analyses, the online music system can more effectively meet user needs, improve user experience, and increase the discoverability and relevance of music. This can not only improve user satisfaction, but also improve the operating efficiency and user stickiness of the platform. Specific embodiment three

[0192] As shown in Table 4, the heterogeneous network dataset-PubMed is a biomedical literature library, using the PubMed dataset in the literature [2]. This dataset constructs a network consisting of genes, diseases, chemicals, and species from PubMed. The node types are gene, disease, chemical substance and species, and the edge types are: gene-disease edge, which indicates the correlation between genes and specific diseases, such as the relationship between gene mutations and related diseases; gene-chemical substance edge, which indicates the interaction between genes and chemical substances (such as drugs, compounds), which is usually used to describe the mechanism of action of drugs; gene-species edge, which indicates the conservation or expression of genes in different species, which helps to compare the functions of genes; disease-chemical substance edge, which indicates the relationship between a specific disease and the chemical substance (drug) used to treat the disease; disease-species edge, which indicates the manifestation and research of a certain disease in different species, such as the correlation between animal models and human diseases; chemical substance-species edge, which indicates the research and application of chemical substances in specific species, especially in pharmacology and toxicology; gene-gene edge, which indicates the interaction or regulatory relationship between genes, such as co-expressed genes or regulatory networks; disease-disease edge, which indicates the similarity or co-morbidity relationship between diseases, such as multiple diseases may have common biomarkers; chemical substance-chemical substance edge, which indicates the similarity or mutual relationship between different chemical substances, such as the structural similarity or functional association of compounds; species-species edge, which indicates the similarity between species, usually based on the similarities in their genomes, phenotypes or ecological functions.

[0193] Table 4

[0194] Dataset Number of nodes Number of node types Number of edges Number of edge types Average PubMed 63109 4 244986 10 3.88

[0195] In the heterogeneous network of the same type in PubMed, the heterogeneous network data to be predicted is the heterogeneous graph of the current heterogeneous network, and the trained link prediction model is used to perform link prediction; it can solve many important problems in the biomedical field: prediction of the relationship between genes and diseases, predicting the potential association between genes and diseases that have not yet been confirmed; prediction of drug-disease associations, predicting which drugs may be effective for specific diseases; prediction of gene function, speculating on the functions of unstudied genes, especially in disease research; prediction of disease comorbidity, identifying diseases that may occur at the same time to improve clinical decision-making; prediction of chemical interaction, predicting possible interactions between chemical substances for use in drug combination therapy; analysis of conservative relationships between species, exploring similar biological mechanisms and disease models in different species.

[0196] According to the known network structure, the prediction of missing edges can be completed, and the prediction results include: the missing connection between genes and diseases, a gene may be related to multiple unconfirmed diseases but not recorded; the connection between genes and drugs, a drug may act on genes that have not yet been studied; the connection between diseases and chemicals, known drugs may affect new diseases or symptoms; unknown connections between species and diseases, the manifestation of specific diseases in other species. According to the original network structure, through link prediction, new edge prediction results can be obtained, such as the association between gene prediction and specific diseases, and potential new indications for drugs. According to the prediction results, the present invention will obtain more abundant node relationships in the network, thereby enhancing the usability of the entire network, and can support new directions in biomedical research. By identifying new associations, it can provide guidance for future experiments, such as drug development for specific genes or research on related mechanisms for diseases. Specific embodiment 4

[0198] As shown in Table 5, the heterogeneous network dataset-YouTube dataset is a multi-way bidirectional network dataset consisting of "five types of interactions" between 15,088 YouTube users. The node type of the YouTube dataset is user, and the edge types include contacts, shared friends, shared subscriptions, shared subscribers, and shared favorite videos between users.

[0199] Table 5

[0200] Dataset Number of nodes Number of node types Number of edges Number of edge types Average Youtube 2000 1 1310544 5 655.27

[0201] In the YouTube data set, the heterogeneous network data to be predicted is the heterogeneous graph of the current heterogeneous network, and the trained link prediction model is used to perform link prediction; the present invention can solve multiple problems, especially in social network analysis and user behavior prediction: the present invention can predict user relationships and identify which users may become friends or contacts, thereby enhancing the connectivity of social networks; the present invention can predict shared subscriptions and preferences, predicting channels that users may subscribe to together or videos they like, so as to improve user experience and personalized content recommendations; the present invention can perform influence propagation analysis and predict which users may have the greatest influence on specific content, thereby helping to formulate marketing strategies or optimize information dissemination; the present invention can discover potential communities and identify potential communities or user groups based on the types of interactions between users, so as to better perform content recommendations or social promotion; the present invention can be used for content recommendation, and recommend related videos or channels to users based on the relationships and interaction types between users.

[0202] Based on the current network structure, missing edges can be predicted, and the prediction results include: possible unestablished connections between users (for example, two users may not have been contacts with each other, but according to other interaction types, they may establish connections); missing shared friend relationships (for example, two users may have common friends but have not yet connected to each other); channels that may be subscribed to together (for example, a user and another user may subscribe to a specific channel together); favorite videos that may be shared (for example, a user may share the same preferences with another user but it has not been recorded). The prediction results of the present invention in this type of scenario include the following aspects: newly established user relationships, by predicting possible connections, new friendships or interactive relationships between users can be discovered; recommended common subscription channels, predicting channels that new users may subscribe to together, which can be used for personalized recommendation systems; interest matching recommendations, through the mechanism of sharing favorites, recommending movies or channels that may be of interest to improve user experience; enhanced social networks, through link prediction, social networks will become closer and interactions between users will increase. Through the static heterogeneous network link prediction technology proposed by the present invention, the behavior patterns between users can be better understood, the quality of social interaction can be improved, and the accuracy of content recommendations can be enhanced, thereby improving the overall user experience.

[0203] The technical effect of the present invention is further verified through experiments.

[0204] In order to evaluate the accuracy of link prediction based on the AHGNN model, the experiment selected 6 of the most representative and latest heterogeneous network link prediction methods as baseline methods for comparative experiments. The specific methods are introduced as follows:

[0205] GCN: A deep learning model based on graph structure data. This method learns the representation of nodes by performing convolution operations on the graph structure. GCN aggregates the feature vector of each node with the feature vectors of its neighboring nodes to obtain a new representation of the node for link prediction.

[0206] GAT: A deep learning model based on graph attention mechanism. This method uses attention mechanism and multi-head attention mechanism to learn the relationship weights between nodes, weight the features of neighbor nodes, obtain node representation, and thus achieve link prediction tasks.

[0207] RGCN: A deep learning model for processing relational graph data. This method assigns a weight matrix to each relation type to learn the feature representation of that relation type. RGCN extends GCN to relational (multi-edge type) graphs, performs convolution operations on the graph structure to learn the representation of nodes, and updates the representation of nodes through weight matrices of different relation types to perform link prediction.

[0208] HetGNN: A deep learning model for processing heterogeneous graphs. This method sets different representations and features for each node and edge type, represents the heterogeneous graph as a multi-layer structure, and learns the representation of nodes by passing information across different layers to perform link prediction tasks.

[0209] HGT: A deep learning model for processing heterogeneous graphs based on the Transformer architecture. This method is based on the Transformer architecture and includes self-attention mechanism and multi-head attention mechanism. By passing information across Transformer blocks in different layers, the representation of nodes is learned to complete the link prediction task.

[0210] LHGNN: A method for link prediction on latent heterogeneous graphs, which maps the original heterogeneous graph into a low-dimensional latent space to obtain a representation of the latent heterogeneous graph. LHGNN obtains node-level and edge-level latent semantic information for link prediction without knowing the types of network nodes and edges.

[0211] Experiment 1 Baseline model comparison test: The AHGNN model proposed in the present invention and 6 baseline methods are compared on 4 data sets. In the present invention, the edges existing in the heterogeneous network are called positive samples, and the non-existent edges are called negative samples. The training set, validation set, and test set all contain positive samples and negative samples. The positive samples are divided proportionally after the edge sets of the heterogeneous network are shuffled, of which 81% are used for training, 9% are used for verification, and 10% are used for testing. In real scenarios, it is usually necessary to distinguish between positive and negative samples with similar characteristics, rather than random negative samples. Therefore, when performing negative sampling, the non-existent edges between the target node and its 2-hop neighbors are selected as negative samples. For the balance of training, the number of positive and negative samples in the training set is the same, and the number of positive and negative samples in the test set is also 1:1. 10 experiments were performed and the average value was taken as the final result. The experimental results are shown in Tables 6 and 7.

[0212] Table 6 AUC results of comparison between AHGNN method and method model (%)

[0213]

[0214] Table 7 MRR results of AHGNN model compared with baseline model (%)

[0215]

[0216] Experiment 1 analysis: From the results, we can see that the simple homogeneous GAT can match the best heterogeneous neural network in most cases. AHGNN, which inherits the heterogeneous GAT, outperforms all advanced heterogeneous neural network methods in link prediction on three datasets. R-GCN learns node representation by introducing a relational feature matrix and a multi-layer GCN structure, but in terms of experimental results, it is not as accurate as the simple GCN, because R-GCN learns node representation based on local neighbor information, which has the limitation of being difficult to capture global graph structure information. In tasks that require better capture of global information, traditional GCN performs better. HetGNN assigns GNN weight matrices to edge types and node types separately, but the number of different types of nodes and the number of different types of edges vary greatly. For those relationship types with few co-occurrences, it is difficult to learn accurate weights for them, so the effect is not as good as traditional GNN. HGT models heterogeneous attention for each edge and implicitly learns meta-paths, but when the relationship types are unbalanced, the individual information of large-category nodes will be ignored, so the optimal effect is not achieved. LHGNN can obtain node-level and edge-level heterogeneous information in the case of unknown network node and edge types, so it performs better in networks with unbalanced node and edge types. The performance of the AHGNN model proposed in this paper is better than all advanced heterogeneous network link prediction methods in terms of the evaluation index AUC on three data sets, and is always better than all baseline methods in terms of the evaluation index MRR, which shows that AHGNN can learn more effective feature vectors to represent the links to be predicted.

[0217] Experiment 2 Ablation experiment: In order to verify the effectiveness of each module in AHGNN, a non-overall node embedding (i.e. AHGNN -add ), no heterogeneous Skip-Gram (i.e. AHGNN -skip ), no heterogeneous GAT (i.e. AHGNN -HGAT Specifically, the sum of shared embedding and edge embedding in AHGNN is changed to only shared embedding as AHGNN. -add ; Remove Skip-Gram's negative sampling optimization function as AHGNN -skip ; Replace the heterogeneous GAT layer with a linear layer as AHGNN -HGAT The experimental results are shown in Table 8.

[0218] Table 8 Ablation experiment results (%)

[0219]

[0220] Experiment 2 analysis: The results show that both indicators of the complete AHGNN model perform best, which proves the effectiveness of generating overall node embedding, heterogeneous Skip-Gram, and heterogeneous GAT modules. Among them, the model without heterogeneous GAT has the worst overall performance, which shows that edge type information needs to be aggregated to improve node information in heterogeneous graphs. The effect of the model without overall node embedding and the heterogeneous Skip-Gram model is also lower than that of the complete AHGNN model, which proves that aggregating neighborhood node features is effective for link prediction tasks and that the heterogeneous Skip-Gram algorithm reduces some structural information loss.

[0221] Experiment 3 Parameter comparison experiment: In order to tune the parameters of the AHGNN method, the present invention selects the edge embedding vector dimension d u , the dimension of the edge type embedding vector in heterogeneous GAT is d r , the path length l of the Skip-gram algorithm s These three parameters are compared in experiments. Specifically, for d u Parameter comparison, fixed d r is 32, l s =10, and d u Set to 16, 32, 64, 128, 256, 512 respectively. r Parameter comparison, fixed d u 200, l s =10, and d r Set to 4, 8, 16, 32, and 64 respectively. s The experimental setup is similar to that of u =200, d r is 32, and l s Set to 4, 8, 16, 32, 64. The experimental results are as follows Figure 2 and Figure 3 shown.

[0222] Experiment 3 Analysis: Experimental results show that the edge embedding vector dimension d u When it is set to 128 on most datasets, the model achieves the best performance on multiple datasets. When it is set to 32 on LastFM, the model performs best in terms of MRR. r When the value is set to 16 on most datasets, the model achieves the best AUC indicator. r When the value is 32, the model AHGNN performs best on the AUC index of the Amazon dataset. r When the value is set to 32 on most datasets, the model achieves the best performance in terms of MRR. rWhen the value is 16, the model performs best in the MRR indicator on the LastFM dataset. For the path length l of the Skip-gram algorithm, s , we can see that when the path length is 32, the AUC and MRR indicators of the model perform best on most data sets. When it is set to 16 on Amazon, the AUC indicator of the model performs best. When it is set to 8 on Amazon, the MRR indicator of the model achieves the best effect.

[0223] Another embodiment of the present invention provides a static heterogeneous network link prediction system based on a graph attention network, the system comprising:

[0224] A data acquisition module is configured to acquire a heterogeneous network training data set and a test data set, wherein the data set includes a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, wherein V is a node set, E is an edge set, φ is a node type set, and ψ is an edge type set;

[0225] A data preprocessing module, configured to perform a random walk on a heterogeneous graph with an edge type of r using a preset meta-path to generate a path set, and obtain a sample batch from the path set;

[0226] The prediction model training module is configured to train the link prediction model based on the sample batches corresponding to the training data set, and performs the following steps for each training sample batch: first, the basic embedding of the target node is obtained by generating the shared embedding and the edge embedding of the aggregated neighbors; then, according to the node sequence generated by the random walk, the node sequence is skip-gram-performed to learn the node structure embedding, so as to obtain the semantic relationship and structural features between multiple types of nodes in the heterogeneous graph; then, the edge type information in the heterogeneous graph is learned through the graph attention network layer to obtain the output vector that integrates the structural information and the edge type information; then, according to the output vector of the target node, whether there is an edge of type r between the nodes is predicted to obtain the prediction probability; then, the loss value is calculated, and the optimizer uses Adam to perform back-propagation to update the parameters;

[0227] Input the sample batches corresponding to the test data set into the above-mentioned trained link prediction model for prediction, and obtain the prediction probability of each batch; calculate the area under the ROC curve and the average reciprocal ranking value of the prediction probability corresponding to the test data set;

[0228] Iterate and repeat the above training and testing steps until the maximum number of iterations is reached, calculate the average of multiple area values ​​under the ROC curves and the average of multiple average reciprocal ranking values ​​of multiple iterations as the final prediction result, and complete the training of the link prediction model;

[0229] The link prediction module is configured to input the heterogeneous network data to be predicted into the trained link prediction model to perform link prediction.

[0230] The function of the static heterogeneous network link prediction system based on graph attention network described in this embodiment can be explained by the aforementioned static heterogeneous network link prediction method based on graph attention network. Therefore, for the parts not described in detail in this embodiment, please refer to the above method embodiments and will not be repeated here.

[0231] Although the present invention has been described according to a limited number of embodiments, it will be apparent to those skilled in the art, with the benefit of the above description, that other embodiments are contemplated within the scope of the invention thus described. The disclosure of the present invention is intended to be illustrative rather than restrictive of the scope of the invention, which is defined by the appended claims.

[0232] The documents cited in the present invention are as follows:

[0233] [1]Cantador I, Brusilovsky P, Kuflik T.Second workshop on informationheterogeneity and fusion in recommender systems(HetRec2011)[C] / / Proceedings of the fifth ACM conference on Recommender systems.2011:387-388.

[0234] [2]Dong Y, Hu Z, Wang K, et al. Heterogeneous network representation learning [C] / / IJCAI.2020,20:4861-4867.

Claims

1. A static heterogeneous network link prediction method based on graph attention network, It is characterized in that The following steps are involved: Step 1: Obtain a heterogeneous network training data set and a test data set, where the data set includes a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, where V is a node set, E is an edge set, φ is a node type set, and ψ is an edge type set; Step 2: Use the preset meta-path to perform random walks on the heterogeneous graph with edge type r to generate a path set, and obtain a sample batch from the path set; Step 3: Train the link prediction model based on the sample batches corresponding to the training data set. Perform the following steps for each training sample batch: First, the base embedding of the target node is obtained by generating a shared embedding and aggregating the edge embeddings of neighbors; Then, based on the node sequence generated by random walk, skip-gram is performed on the node sequence to learn the node structure embedding to obtain the semantic relationship and structural features between multiple types of nodes in the heterogeneous graph; Then, the edge type information in the heterogeneous graph is learned through the graph attention network layer to obtain an output vector that combines structural information and edge type information; Then, based on the output vector of the target node, predict whether there is an edge of type r between the nodes and obtain the predicted probability; Then, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters; Step 4: Input the sample batches corresponding to the test data set into the link prediction model trained in step 3 for prediction, and obtain the prediction probability of each batch; Step 5: Calculate the area under the ROC curve and the average reciprocal ranking value of the predicted probability corresponding to the test data set; Step 6: Iterate and repeat steps 3 to 5 until the maximum number of iterations is reached, calculate the average of multiple area values ​​under the ROC curves of multiple iterations and the average of multiple average reciprocal ranking values ​​as the final prediction result, and complete the training of the link prediction model; Step 7: Input the heterogeneous network data to be predicted into the trained link prediction model for link prediction.

2. According to claim 1, a static heterogeneous network link prediction method based on graph attention network, It is characterized in that The step 2 of using a preset meta-path to perform a random walk on a heterogeneous graph with an edge type of r to generate a path set includes: dividing the heterogeneous graph G into multiple layers according to the edge type, performing a random walk on each layer according to the preset meta-path, and generating a path set P r .

3. According to claim 1, a static heterogeneous network link prediction method based on graph attention network, It is characterized in that The basic embedding of the target node obtained by generating shared embeddings and aggregating edge embeddings of neighbors in step 3 includes: First, use the mean aggregator to aggregate the target node v i The edge embeddings of the k-order neighbors on the layer of type r are averaged and aggregated to obtain the edge embedding of this edge type. Where σ is the activation function, is the mapping matrix; mean represents mean aggregation, N i,r is node v i The set of neighbors on the edge of type r; Represents node v i Neighbor node v on layer with edge type r j The k-1th order edge embedding of ; Then, combine the edge embedding u of all edge types i,r Get the target node v i The edge feature matrix U i ; Then, the self-attention mechanism is used to calculate the edge feature matrix U i coefficient a of the linear combination of vectors on edge type r i,r : In the formula, softmax is the activation function, w r and W r They are the trainable parameters on the edge of type r, and the superscript T represents the transpose of a vector or matrix; Then, based on the coefficient a i,r The edge embedding matrix U i Perform weighted summation and integrate the shared embedding b i Get the target node v i Basic embedding z on each edge type i,r : Where b i Represents node v i The shared embedding of ξ is randomly initialized by the model; r is a hyperparameter indicating the importance of edge embedding to the overall embedding; p Indicates a i,r The pth element of i,p Represents node v i is the edge embedding of edges of type p; R represents the total number of edge types.

4. According to claim 3, a static heterogeneous network link prediction method based on graph attention network, It is characterized in that In step 3, according to the node sequence generated by random walk, skip-gram is performed on the node sequence to learn the node structure embedding, including: First, calculate the negative sampling optimization objective function E 1 : Where σ is the sigmoid activation function; L is the number of negative samples corresponding to positive training samples; z i,r represents the basic embedding of node i on edge type r, z j,r represents the basic embedding of node j on edge type r; Represents the slave node v j The corresponding node set V t The noise distribution P defined above t (v) randomly select v k Function of v k is from node v j The corresponding node set V t The noise distribution P defined above t (v) randomly selected; Then, concatenate the target node v i The basic embedding z of each edge type i,r Get node structure embedding In the formula, || represents the concatenation operation.

5. According to claim 4, a static heterogeneous network link prediction method based on graph attention network, It is characterized in that The step 3 described in which the edge type information in the heterogeneous graph is learned through the graph attention network layer includes: First, the attention score is calculated using edge type embedding and node structure embedding In the formula, LeakyReLU represents the activation function; α represents the learnable mapping vector; r ψ(<i,j>) Indicates edge type characteristics; h j represents the structural embedding of node j; h k represents the structural embedding of node k; ψ(<i,j> ) represents the type of edge between node i and node j; W and W r is a learnable mapping matrix used to map node embeddings and edge type embeddings; N i Represents node v i The set of neighbors of ; Then, in the attention score Add residual connection to obtain comprehensive attention coefficient Where, the hyperparameter β∈[0,1] is the scaling factor; Represents node v i For node v j The attention score at the lth layer of the network; Then, according to the comprehensive attention coefficient Aggregate the layer l network: In the formula, Represents node v i In the structure embedding of the l-th layer network, W (l) Represents the trainable mapping matrix of the l-th layer network; Finally, the structure is embedded Perform L2 normalization to get the output embedding.

6. According to claim 5, a static heterogeneous network link prediction method based on graph attention network, It is characterized in that In step 3, based on the output vector of the target node, the following formula is used to predict whether there is an edge of type r between nodes: In the formula, sigmoid is the activation function; R r is a learnable matrix of edge type r; o u and v Represent the output embedding of node u and node v respectively; Prob r represents the probability that there is an edge of type r between node u and node v.

7. According to claim 6, a static heterogeneous network link prediction method based on graph attention network, It is characterized in that In step 3, the loss value is calculated according to the following formula: Where μ is a hyperparameter; L t is the set of target links to be predicted, p l is the probability that link l exists, y l ∈{0,1} is a label indicating whether the link is a positive or negative sample. 1 indicates that it is a positive sample and the link exists, and 0 indicates that it is a negative sample and the link does not exist.

8. A static heterogeneous network link prediction system based on graph attention network, It is characterized in that include: A data acquisition module is configured to acquire a heterogeneous network training data set and a test data set, wherein the data set includes a heterogeneous graph; the heterogeneous graph G = {V, E, φ, ψ}, wherein V is a node set, E is an edge set, φ is a node type set, and ψ is an edge type set; A data preprocessing module, configured to perform a random walk on a heterogeneous graph with an edge type of r using a preset meta-path to generate a path set, and obtain a sample batch from the path set; A prediction model training module is configured to train a link prediction model based on a sample batch corresponding to a training data set, and perform the following steps for each training sample batch: first, obtain a base embedding of a target node by generating a shared embedding and an edge embedding of aggregated neighbors; Then, according to the node sequence generated by random walk, skip-gram is performed on the node sequence to learn the node structure embedding to obtain the semantic relationship and structural features between multiple types of nodes in the heterogeneous graph; then, the edge type information in the heterogeneous graph is learned through the graph attention network layer to obtain an output vector that integrates the structural information and edge type information; then, according to the output vector of the target node, whether there is an edge of type r between the nodes is predicted to obtain the prediction probability; then, the loss value is calculated, and the optimizer uses Adam to perform backpropagation to update the parameters; Input the sample batches corresponding to the test data set into the above-mentioned trained link prediction model for prediction, and obtain the prediction probability of each batch; calculate the area under the ROC curve and the average reciprocal ranking value of the prediction probability corresponding to the test data set; Iterate and repeat the above training and testing steps until the maximum number of iterations is reached, calculate the average of multiple area values ​​under the ROC curves and the average of multiple average reciprocal ranking values ​​of multiple iterations as the final prediction result, and complete the training of the link prediction model; The link prediction module is configured to input the heterogeneous network data to be predicted into the trained link prediction model to perform link prediction.

Citation Information

Patent Citations

  • Link prediction method based on heterogeneous network representation learning

    CN111325326A

  • Patient food recommendation system based on graph neural network

    CN118098511A

  • Data processing method and apparatus, program product, computer device, and medium

    US20240177006A1

Cited By

  • Knowledge tracking and test question answering prediction method based on heterogeneous graph attention network

    CN119917815A

  • RFID warehouse logistics management method and system

    CN120069702A

  • RFID warehousing logistics management method and system

    CN120069702B