Link prediction method for heterogeneous networks based on meta-path type decomposition

Abstract

The present invention discloses a heterogeneous network link prediction method based on meta-path type decomposition, which belongs to the field of link prediction of complex networks, and includes the following steps: projecting different types of node features in heterogeneous networks into the same latent vector space to obtain a projection node feature network; mining the semantic information contained in the meta-path internal node type in the projection node feature network through intra-class aggregation and inter-class aggregation; performing meta-path aggregation on the semantic information, merging the semantic information contained in all meta-paths, and obtaining a vector representation of the target node; based on the vector representation of the target node, predicting downstream tasks through a multi-layer perceptron method. The present invention obtains the node information of the target node pair through feature projection, intra-class aggregation, inter-class aggregation, and meta-path aggregation, thereby achieving more accurate prediction.

Description

Technical Field

[0001] The present invention belongs to the field of link prediction of complex networks, and in particular relates to a heterogeneous network link prediction method based on meta-path type decomposition. Background Art

[0002] Find the network topology features that you have pre-set in the existing network structure, and assign scores to the structure you have set. Finally, use the scores of the node pairs at both ends of the target link as the standard for your prediction.

[0003] For example, Common neighbors checks the neighbor structure around a node and calculates the number of common neighbors of adjacent nodes. If there are many common neighbors between two nodes, it means that the relationship between them is relatively close, while if there are fewer common neighbors between them, it means that the relationship between them is not very close. This algorithm can effectively detect closely connected nodes in the network and use this as a basis for further link prediction in the network. The ResourceAllocation algorithm distinguishes the importance of the common neighbors of each node pair by evenly allocating resources to each neighbor of the target node pair, and uses this as a basis for calculating the possibility of the existence of the target link. The Katz algorithm, compared with the Common neighbor algorithm, performs calculations at the full graph level. It performs a more complex and more accurate evaluation from a semantic perspective by counting all the paths connecting the two target nodes and scoring different paths differently. When calculating, Katz assigns smaller weights to longer paths and larger weights to shorter paths to evaluate the closeness of the target node pairs from a full graph perspective.

[0004] However, the above methods have the following disadvantages: they rely on people's pre-observed statistical scoring experience, so they are not adaptable when facing networks with different topological characteristics, such as protein networks and other heterogeneous networks. Similarly, when facing the diversity of nodes and edges in heterogeneous networks, heuristic methods usually ignore the node characteristics of heterogeneous networks and can only predict heterogeneous networks through simple network topology structures. The loss of the characteristics of nodes and edges in heterogeneous networks is often fatal to heterogeneous networks. Summary of the invention

[0005] In order to solve the above technical problems, the present invention proposes a heterogeneous network link prediction method based on meta-path type decomposition to solve the problems existing in the prior art.

[0006] To achieve the above object, the present invention provides a heterogeneous network link prediction method based on meta-path type decomposition, comprising the following steps:

[0007] Project different types of node features in heterogeneous networks into the same latent vector space to obtain a projected node feature network;

[0008] Mining semantic information contained in the node types inside the meta-path in the projection node feature network through intra-class aggregation and inter-class aggregation;

[0009] Performing meta-path aggregation on the semantic information, merging the semantic information contained in all meta-paths, and obtaining a vector representation of the target node;

[0010] Based on the vector representation of the target node, the downstream task is predicted using the multi-layer perceptron method.

[0011] Optionally, the method of projecting into the same latent vector space includes: projecting the node features based on linear transformation, and the projection method expression is:

[0012]

[0013] In the formula, a represents the node type, Represents the original feature vector of the node, h' v Represents the vector after node projection, W a Represents the parameter weight matrix of the node.

[0014] Optionally, the intra-class aggregation method includes:

[0015] The set of nodes within the class is normalized by the softmax function to obtain the standardized important weights of the nodes within the class. The feature vector of the internal nodes of the final meta-path instance is calculated by weighted summation. The feature vector is calculated by the activation function to obtain the intra-class aggregation vector.

[0016] Optionally, the intra-class aggregation expression is:

[0017]

[0018]

[0019] In the formula, represents the attention parameter matrix of node v in node type t within meta-path instance p, V t p represents the node set of node type t on the meta-path instance p, e v represents the importance of node v in meta-path p for node type t, σ(·) represents the activation function, Represents the within-class aggregation vector.

[0020] Optionally, the inter-class aggregation method includes:

[0021] The softmax function is used to normalize the attention coefficients of the inter-class node types to obtain the standardized important weights of the inter-class nodes. The feature vectors of the inter-class nodes of the meta-path instances are calculated by weighted summation. The feature vectors of the inter-class nodes are calculated by the activation function to obtain the inter-class aggregation vector.

[0022] Optionally, the inter-class aggregation expression is:

[0023]

[0024]

[0025] In the formula, The attention parameter matrix representing the type of nodes in the meta-path p, T p Represents the node type set contained in the meta-path instance p, e t represents the importance of node type t to meta-path p, h' p Represents the between-class aggregation vector.

[0026] Optionally, the meta path aggregation expression is:

[0027]

[0028]

[0029]

[0030] In the formula, a pi Represents a meta-path instance p i The attention parameter matrix, h' pi Represents a meta-path instance p i The characteristic vector of Represents a meta-path instance p i The relative importance of the target node v, Represents a meta-path instance p j The relative importance of the target node v, h p represents the vector representation of the target node v in the meta-path p, h q Represents the vector representation of the target node v in the meta-path q, a p represents the attention of the meta-path p, a q represents the attention of meta-path q, P represents the set of meta-paths, A vector representation of the target node v.

[0031] Optionally, the predicted expression is:

[0032]

[0033] Puv =σ(w·z)

[0034] In the formula, σ(·) represents the sigmoid function, ω represents the weight matrix, Represents the vector representation of the target node v, P uv Represents the predicted label of the target link.

[0035] Compared with the prior art, the present invention has the following advantages and technical effects:

[0036] The heterogeneous network link prediction method based on meta-path type decomposition described in the present invention obtains node information of target node pairs through feature projection, intra-class aggregation, inter-class aggregation and meta-path aggregation, and can achieve more accurate prediction. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0038] Figure 1 Schematic diagram of a prediction method according to an embodiment of the present invention;

[0039] Figure 2 This is a diagram of the ablation experiment results of TDPM on DBLP according to an embodiment of the present invention;

[0040] Figure 3 This is a diagram of the ablation experiment results of TDPM on Amazon according to an embodiment of the present invention;

[0041] Figure 4 is a diagram of the ablation experiment results of TDPM on Last.fm according to an embodiment of the present invention;

[0042] Figure 5 Schematic diagram of experimental parameters of an embodiment of the present invention. DETAILED DESCRIPTION

[0043] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0044] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0045] like Figure 1 As shown, the present invention proposes a heterogeneous network link prediction method based on meta-path type decomposition, comprising the following steps:

[0046] Project different types of node features in heterogeneous networks into the same latent vector space to obtain a projected node feature network;

[0047] Mining semantic information contained in the node types inside the meta-path in the projection node feature network through intra-class aggregation and inter-class aggregation;

[0048] Performing meta-path aggregation on the semantic information, merging the semantic information contained in all meta-paths, and obtaining a vector representation of the target node;

[0049] Based on the vector representation of the target node, the downstream task is predicted using the multi-layer perceptron method.

[0050] The specific implementation methods are as follows:

[0051] 1. Feature Projection

[0052] The attributes of nodes in heterogeneous networks are usually heterogeneous, that is, different node types may have different feature vector dimensions. Even if the feature vectors of different node types happen to be of the same dimension, they may be in different latent feature spaces. For example, the bag-of-words vector of a text and the histogram vector of an image have different meanings even if their vector dimensions are equal. In order to handle feature vectors of different dimensions in a unified framework, it is necessary to first project different types of node features into the same latent vector space.

[0053] In this module, linear transformation is used to achieve the purpose of vector projection. For example, for a node v with node type a, the projection operation is as follows:

[0054]

[0055] Where a∈T v Indicates the node type, is the original feature vector of node v, h v '∈R d' is the vector after projection of node v, is the parameter weight matrix of the node.

[0056] Node feature projection solves the problem of heterogeneity in node attributes of the graph caused by different feature dimensions of node types. After applying this operation, the node features of all nodes after projection exist in the same feature dimension and latent vector space, which is conducive to the aggregation process of the next module of the model.

[0057] 2. Intra-class aggregation module

[0058] As a component of the meta-path, the node types that make up the meta-path naturally contain more detailed semantic meanings and features than the meta-path. The design purpose of the intra-class aggregation module and the inter-class aggregation module is to mine the semantic information contained in the node types within the meta-path in the heterogeneous network. Given a meta-path P = A1A2A3A2A1, which contains three specific node types A1A2A3, then for all instances of meta-path P p1, p2, ..., p n , the importance of different node entities contained in the same type of nodes in each meta-path instance can be calculated through the intra-class aggregation module. n In each instance, the node types A1A2A3 contained in P are aggregated within their types. From a semantic point of view, the meta-path is a specific sequence defined by the ordering of node types. Even if the selected meta-path is symmetrical, the contribution of each node entity within the same node type to the link prediction task should be different for the target node pair (target link). Therefore, in order to distinguish the contribution of each node entity within the same node type, the attention mechanism is used to aggregate each node type in each meta-path instance. The aggregation formula for nodes within a specific type is as follows:

[0059]

[0060]

[0061] in, represents the attention parameter matrix of node v in node type t within meta-path instance p, V t p represents the node set of node type t on the meta-path instance p, e v represents the importance of node v in meta-path p for node type t, and then uses the softmax function to calculate u∈V t p Normalize all nodes of , that is, normalize the importance of node v in all meta-path instances p. t p After the normalized important weights of , the feature vector of the internal node type t of the final meta-path instance p is calculated by weighted summation. Finally, the output of passes through an activation function σ(·) to obtain its final vector.

[0062] 3. Inter-class aggregation module

[0063] After the intra-class aggregation module aggregates the single node type inside the meta-path instance, the type vectors of each node type in the meta-path instance are obtained. In order to obtain a complete vector representation of the meta-path instance, the node type vectors obtained in the previous section need to be aggregated.

[0064] Based on semantic reasons, that is, different node types in the same meta-path contribute differently to the meta-path, different node types in the same meta-path instance are aggregated using an attention mechanism to distinguish the different contributions of node types to the meta-path instance. The aggregation formula of the inter-class aggregation module is as follows:

[0065]

[0066]

[0067] in, The attention parameter matrix representing the type of nodes in the meta-path p, T p Represents the node type set contained in the meta-path instance p, e t represents the importance of node type t to meta-path p. Next, we use the softmax function to represent u∈T p The attention coefficients of all node types are normalized, that is, the importance represented by each node type t in the meta-path p is normalized. After calculating all u∈T p After the normalized important weights of , the feature vector of the meta-path instance p is calculated by weighted summation. Finally, the output of is passed through an activation function σ(·) to obtain the final vector h' of the meta-path instance p p .

[0068] 4. Meta-path aggregation

[0069] After encoding the meta-path instance into a vector representation, the attention mechanism is used to perform weighted summation on the meta-path instances of P related to the target node pair. From the perspective of semantic representation, different meta-path instances will promote the feature representation of the target node pair to different degrees. Modeling can be performed by learning the standardized importance weight of each meta-path instance, and then performing weighted summation on all instances. The model can obtain a set of feature vectors {h} about each meta-path of the node v of one of the target node pairs. p1 ,h p2 ,...,h pn}.

[0070] After aggregating the feature data of each type of node in each meta-path, it is necessary to use aggregation between each meta-path instance to merge the semantic information contained in all meta-paths. Now for a node v that is one of the target nodes, there is a set of potential vectors for each meta-path of v: {hp1 ,h p2 ,...,h pn}. A simple method of meta-path aggregation is to take the element mean of these node vectors, but it is obviously unreasonable to treat various meta-path instances and meta-path aggregation methods with the same importance. This problem is solved by using the attention mechanism to assign different weights to different meta-paths. The meta-path aggregation formula applied is as follows:

[0071]

[0072]

[0073]

[0074] In the formula, a pi Represents a meta-path instance p i The attention parameter matrix, h' pi Represents a meta-path instance p i The characteristic vector of Represents a meta-path instance p i The relative importance of the target node v, Represents a meta-path instance p j The relative importance of the target node v, h p represents the vector representation of the target node v in the meta-path p, h q Represents the vector representation of the target node v in the meta-path q, a p represents the attention of the meta-path p, a q represents the attention of meta-path q, P represents the set of meta-paths, Represents the vector representation of the target node v. Where p i 、p j represents different meta-path instances, and P and q represent different meta-paths.

[0075] Similarly, similar operations are performed on the other node u in the target node pair that constitutes the target link, and the vector representation of the u node after aggregating the meta-path information can be obtained. In order to obtain the vector representation of the target link, the vectors of the two target nodes u and v are multiplied by matrix to obtain the representation of the target link in the vector dimension, and then the downstream task is predicted by a multi-layer perceptron:

[0076]

[0077] P uv =σ(w·z)

[0078] Among them, σ(·) is the sigmoid function, and ω is the weight matrix. The output P uvis the predicted label of the target link.

[0079] The following comparative experiments were conducted:

[0080] 1. Dataset Introduction

[0081] In this section, three commonly used public heterogeneous datasets: DBLP, Amazon, and Last.fm are used to evaluate the effectiveness of the meta-path type decomposition based link prediction method (TDPM) compared with baseline methods. Table 1 summarizes the simple statistics of these three datasets.

[0082] Table 1

[0083]

[0084] DBLP: This dataset is a computer science bibliography website that contains academic relationships in computer science. A widely used extracted subset of DBLP [79-80] is used, which consists of 4057 authors (A), 14328 papers (P), 7723 terms (T), and 20 locations (V). Each author has four types of labels, representing their different research fields (databases, data mining, artificial intelligence, and information retrieval). DBLP also includes 19,645 AP edges, 85,810 PT edges, and 14,328 PV edges. For the method using meta-paths, {APA, APTPA, APVPA} were selected in DBLP for experiments. On DBLP, author-paper links and co-author hyperlinks were predicted.

[0085] Amazon: This is an extracted movie review dataset. A data subset containing 1754 movies (M), 283 brands (B), 2476 users (U), and 95 movie categories (C) was extracted from the preprocessed subset of Amazon

[66] . The nodes form 34,042 MU-type edges, 734 MB-type edges, and 4,913 MC-type edges. On the Amazon dataset, for methods that require the use of meta-paths, experiments were conducted using the meta-path {UMU, UMBMU, UMCMU}. On Amazon, edges that predict UM are selected to simulate movie recommendations.

[0086] Last.fm: This dataset comes from a British online music website that includes the relationship between listeners and singers. The dataset used is a subset of the widely used recommendation dataset

[81] , which includes 1,892 users, 17,632 artists, and 1,088 artist tags. 92,834 UA edges, 23,252 AT edges, and 12,717 UU edges constitute the relationship between nodes. For methods that require the use of meta-paths, the meta-path set {UAU, UATAU, AUA, AUUA, ATA} is used for experiments. On Last.fm, the links between users and artists are predicted to simulate user preference recommendations. For semi-supervised learning on the above three datasets, node pairs with different link existences are randomly divided into training, validation, and test sets. The proportions are 70%, 10%, and 20%, respectively.

[0087] 2. Experimental Setup

[0088] For all random walk-based methods, such as DeepWalk, node2vec, Metapath2vec, and HERec, the input window size is set to 5, the walk length is set to 100, the walk for each node is set to 40, and the number of negative samples is set to 5. For GNN-based methods, including GAT, HAN, and MAGNN, the number of attention heads K is set to 8. For the THGNN method, since its text-based prior module uses a large amount of text information that may not exist in many data sets, the ablation method for removing the prior module provided in the embodiment is used as the baseline method for experimental comparison when comparing THGNN. For the method TDPM, TDPM is optimized by setting the learning rate to 0.005 and the number of iterations to 210. AUC and AP are used as evaluation indicators of model performance. Link prediction experiments were conducted in three data sets, and the performance of the method was compared with the baseline method. The AA node relationship on DBLP is constructed by calculating a similarity of more than 50%. For all the compared methods based on the above node embeddings, including DeepWalk, node2vec, Metapath2vec, HEREec, GAT, HAN, and MAGNN, the dot product of the target node pair embedding is applied to represent the possibility of link existence.

[0089]

[0090] Here, σ(·) represents the sigmoid function. Ten experiments were conducted on the same model, and the performance of each model was reported by calculating the average AUC and AP obtained.

[0091] Table 2

[0092]

[0093] 3 Results Analysis

[0094] From the experimental results in Table 2, it can be easily found that the methods of have achieved relatively good performance in the selected datasets compared with the selected baseline methods. At the same time, GAT, which was originally used for homogeneous networks and used the attention mechanism, achieved more ideal results than other traditional random walk-based network embedding methods, which to a certain extent shows the advantages of graph neural network methods over traditional graph embedding methods on heterogeneous networks. In the comparison with MAGNN, it can be found that the performance of the methods of generally exceeds MAGNN, which means that it is effective for more detailed semantic distinctions within the meta-path. In the comparison with THGNN, it can be found that the experimental performance of TDPM is relatively similar to that of THGNN. This is because the number of attention layers of the two methods is similar from a semantic perspective, which also shows that the semantic levels proposed by the two methods are meaningful. In order to further explore the impact of each module in the model on the overall performance of the model, that is, to answer RQ2, several different ablation models are designed to experiment with the effect of the aggregation module. The ablation model and the original model of are placed on the same three datasets for experimentation, and the hyperparameters are set the same as in Experiment 1 to better eliminate the impact of other objective conditions on the ablation experiment results. The ablation model settings are as follows:

[0095] The original model of the meta-path type decomposition-based link prediction method (TDPM) is used as the reference model; for the intra-class aggregation module, the following are designed: TDPMintra-mean switches to use the average pooling operation; TDPMintra-linear switches the linear aggregation operation to replace the designed intra-class aggregation module. For the inter-class aggregation module, the following are designed: TDPMinter-mean switches to use the average pooling encoder; TDPMinter-linear switches the linear combination method to replace the adopted inter-class aggregation module. In order to explore the effectiveness of the meta-path aggregation module, the following is designed: TDPMinstance directly aggregates the meta-path instances through linear combination.

[0096] From the results of the ablation experiment Figure 2 , Figure 3 and Figure 4From the above, the experimental results of TDPM are the best, which means that the intra-class aggregation module, inter-class aggregation module, and meta-path aggregation module designed in the embodiment are all helpful to improve the performance of the model, and can effectively aggregate the information of the target node on each semantic level in the surrounding meta-path. Relatively speaking, among the various ablation modules designed, the performance degradation caused by the changes made to the model by TDPMintra-linear is the smallest, which may mean that the difference in contribution of different nodes of the same node type to the node type in the same meta-path instance is smaller than the difference in contribution of different node types to the meta-path in the same meta-path. Similarly, compared with the difference in contribution of different meta-paths to the target node pair, the difference in contribution of different nodes of the same node type to the node type in the same meta-path instance is relatively much smaller. The performance fluctuation caused by the changes made by TDPMinstance to the model is the largest, which may also mean that the semantic difference between meta-path instances and between meta-paths is relatively greater. In order to answer RQ3, two parameter experiments were designed to study the sensitivity of parameters on the DBLP dataset. The evaluation indicators still use AUC and AP to measure the impact of learning rate on the performance of the model, while loss is used to evaluate whether the model has converged for the number of iterations.

[0097] The experiment selected the learning rate and the number of epochs for the experiment, and its impact on the model performance is as follows Figure 5 As shown in the figure, when the learning rate is adjusted from 0.0001 to 0.0002, the performance improves rapidly. Then, when the learning rate is between 0.0002 and 0.0005, the performance remains in a stable range. When the learning rate is further increased, the performance decreases. Therefore, a learning rate of 0.0003 is selected to balance the speed and performance of the model. Figure 5 As shown in the figure, in the first 70 iterations, the loss dropped very quickly, which shows that the model was effectively trained. After 130 iterations, the loss gradually stabilized. At the end of 230 iterations, the model was fully learned and the loss began to converge. In order to ensure the comparison and experimental effect of subsequent experiments, the number of iterations was set to 250 in the above experiment.

[0098] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. A heterogeneous network link prediction method based on meta-path type decomposition, characterized in that: The following steps are involved: Project different types of node features in heterogeneous networks into the same latent vector space to obtain a projected node feature network; Mining semantic information contained in the node types inside the meta-path in the projection node feature network through intra-class aggregation and inter-class aggregation; Performing meta-path aggregation on the semantic information, merging the semantic information contained in all meta-paths, and obtaining a vector representation of the target node; Based on the vector representation of the target node, the downstream task is predicted using the multi-layer perceptron method.

2. The heterogeneous network link prediction method based on meta-path type decomposition according to claim 1 is characterized in that: The method of projecting into the same latent vector space includes: projecting the node features based on linear transformation, and the projection method expression is: In the formula, a represents the node type, represents the original feature vector of the node, h v ' represents the vector after node projection, W a Represents the parameter weight matrix of the node.

3. The heterogeneous network link prediction method based on meta-path type decomposition according to claim 1 is characterized in that: The methods of in-class aggregation include: The set of nodes within the class is normalized by the softmax function to obtain the standardized important weights of the nodes within the class. The feature vector of the internal nodes of the final meta-path instance is calculated by weighted summation. The feature vector is calculated by the activation function to obtain the intra-class aggregation vector.

4. The heterogeneous network link prediction method based on meta-path type decomposition according to claim 3 is characterized in that: The in-class aggregate expression is: In the formula, represents the attention parameter matrix of node v in node type t within meta-path instance p, V t p represents the node set of node type t on the meta-path instance p, e v represents the importance of node v in meta-path p for node type t, σ(·) represents the activation function, Represents the within-class aggregation vector.

5. The heterogeneous network link prediction method based on meta-path type decomposition according to claim 1 is characterized in that: The inter-class aggregation method includes: The softmax function is used to normalize the attention coefficients of the inter-class node types to obtain the standardized important weights of the inter-class nodes. The feature vectors of the inter-class nodes of the meta-path instances are calculated by weighted summation. The feature vectors of the inter-class nodes are calculated by the activation function to obtain the inter-class aggregation vector.

6. The heterogeneous network link prediction method based on meta-path type decomposition according to claim 5 is characterized in that: The inter-class aggregation expression is: In the formula, The attention parameter matrix representing the type of nodes in the meta-path p, T p Represents the node type set contained in the meta-path instance p, e t represents the importance of node type t to meta-path p, h' p Represents the between-class aggregation vector.

7. The heterogeneous network link prediction method based on meta-path type decomposition according to claim 1 is characterized in that: The meta-path aggregation expression is: In the formula, a pi Represents a meta-path instance p i The attention parameter matrix, Represents a meta-path instance p i The characteristic vector of Represents a meta-path instance p i The relative importance of the target node v, Represents a meta-path instance p j The relative importance of the target node v, h p represents the vector representation of the target node v in the meta-path p, h q Represents the vector representation of the target node v in the meta-path q, a p represents the attention of the meta-path p, a q represents the attention of meta-path q, P represents the set of meta-paths, A vector representation of the target node v.

8. The heterogeneous network link prediction method based on meta-path type decomposition according to claim 1 is characterized in that: The predicted expression is: P uυ =σ(ω·z) In the formula, σ(·) represents the sigmoid function, ω represents the weight matrix, Represents the vector representation of the target node v, P uv Represents the predicted label of the target link.

Images