Graph neural network link prediction method based on Gaussian mutual information

Abstract

The invention relates to a graph neural network link prediction method based on Gaussian mutual information. The prediction method comprises the following steps: extracting hidden features of nodes by using a variational inference mechanism of a graph variational auto-encoder, and improving Gaussian distribution characteristics of the hidden features through an adaptive KL weight; separating the hidden features into causal features and hybrid features by means of an orthogonal projector; a Gaussian mutual information estimator is used for carrying out mutual information estimation on existence labels of the hybrid features and the edges, and the mutual information is used as a regularization item; extracting local subgraph representation containing distance features, and iterating node representation of each node in the subgraph by using a local feature encoder; and integrating the causal features and the local structure features through an edge predictor to carry out link probability prediction. According to the method, the AUC of link prediction can be obviously improved, accurate prediction of the potential relation is realized, the feature integrity can be effectively improved, redundant feature interference is reduced, and the method is suitable for link prediction tasks of complex graph structures such as social networks and knowledge graphs.

Description

Technical Field

[0001] This invention relates to the field of graph machine learning technology, and specifically to a graph neural network link prediction method based on Gaussian mutual information. Background Technology

[0002] Link prediction, a core task of graph machine learning, predicts potential edges using existing graph structures and node features. It is widely used in scenarios such as social network relationship inference, knowledge graph completion, and biological network analysis. With the explosive growth in the scale and increasing complexity of graph data, traditional link prediction methods (such as DeepWalk and LINE) struggle to capture the deep connections between high-order topological semantics and node features. While methods based on graph neural networks (GNNs) have become mainstream, they still have significant limitations in handling causal relationships, confounding factors, and long-distance dependencies in complex graph structures.

[0003] Existing GNN models learn node representations through neighborhood aggregation mechanisms. For example, in the paper "Semi-Supervised Classification with Graph Convolutional Networks," Kipf and Welling proposed a GCN model based on spectral graph theory. However, because they treat all neighborhood information equally, they do not explicitly distinguish between causal features and confounding features. Especially in social networks, users' shared interests may be the causal factor of connections, while geographical overlap may be a confounding factor. Traditional models struggle to separate the two, leading to prediction bias.

[0004] Mutual information, as a key indicator for measuring the dependency between variables, is widely used to quantify the correlation between multidimensional variables. Traditional link prediction often employs linear estimation or simple entropy approximation, which struggles to capture the nonlinear association between high-dimensional features and labels. Linear estimation of mutual information suffers from instability, while entropy estimation is computationally slow, making it unsuitable for link prediction tasks with a large number of edges. For example, the paper "Mutual Information Neural Estimation" discloses the MINE mutual information estimator, but this model struggles to quantify the association strength between multi-source features (such as topological features and node attributes) and link labels in complex graph structures, leading to feature selection bias and affecting prediction accuracy. Summary of the Invention

[0005] This invention provides a graph neural network link prediction method based on Gaussian mutual information to overcome the problem in the prior art where interference from confounding factors leads to biased feature selection and ultimately affects prediction accuracy.

[0006] To achieve the above objectives, the technical solution provided by this invention is: a graph neural network link prediction method based on Gaussian mutual information, comprising the following steps:

[0007] Step 1: Divide the graph data into training and test sets with negative sampling. Preprocess the training, test, and evaluation sets to extract local subgraphs and the representations of each node in the subgraphs.

[0008] Step 2: Extract the latent features of each node from the original features of the nodes using a graph variational autoencoder, and improve the Gaussian distribution characteristics of the latent features by using adaptive KL divergence.

[0009] Step 3: Use a bilinear orthogonal projector to separate the Gaussian latent features into causal features and promiscuous features;

[0010] Step 4: Use Gaussian mutual information to estimate the mutual information between the mixed features and the existence labels of the edges separated in Step 3 and use it as a regularization term;

[0011] Step 5: For the subgraph extracted in Step 1, use a local feature encoder to iterate the node representation of each node in the subgraph, where the local structural features of each subgraph are obtained through a pooling layer.

[0012] Step 6: Integrate causal features and local structural features through an edge predictor to predict link probabilities.

[0013] Furthermore, in step one above, the data preprocessing of the dataset includes the following steps:

[0014] 1.1 Using the adjacency matrix of the positively sampled edges in the training set in step one, extract all nodes within k hops for each target edge (src, dst) in the training set, test set, and evaluation set, and construct a local subgraph node set;

[0015] 1.2 Calculate the shortest path distance from each node to src and dst. If the distance exceeds k, set it to k+1 to form a two-dimensional distance feature [distsrc, distdst].

[0016] 1.3. Concatenate the original node features with the distance features to generate an extended feature vector [xnode,distsrc,distdst].

[0017] Furthermore, the specific steps of step two above are as follows:

[0018] 2.1 Extracting node features through a two-layer GATv2Conv network: Both GAT layers use the ELU function for non-linear activation. The first layer GATv2Conv (multi-head attention) maps the input features to the hidden_dim×heads dimension space; the second layer GATv2Conv (single-head attention) compresses the features to the output_dim dimension, and BatchNorm and Dropout are used to prevent overfitting.

[0019] 2.2. Generate the mean μ and logvariance logvar of the latent variables using fully connected networks μ_net and logvar_net;

[0020] 2.3. The reparameterization technique z = μ + ∈·exp(0.5·logvar) is used to generate the probabilistic latent representation z, where ∈ ~ N(0,I);

[0021] 2.4. Measuring the Gaussian loss of VAE using KL divergence.

[0022] Furthermore, the specific steps of step three above are as follows:

[0023] 3.1 The latent variable z is separated into causal feature c and confounding feature s through two-layer linear projection causal_proj and spurious_proj. The projection layer adopts orthogonal initialization and LayerNorm.

[0024] 3.2. Introduce the feature preservation system feature_preserver, which enhances the robustness of representation by fusing the original latent features and projected features through gradient gating.

[0025] Furthermore, the specific steps of step four above are as follows:

[0026] 4.1 Calculate the Gaussian kernel matrix K = exp(-||XX) for the joint feature X. T ||| 2 / (2σ 2 )) Calculate the Gaussian kernel matrix L for label y;

[0027] 4.2. Centering the kernel matrix using the central matrix H = I⁻¹ / n: K c =HKH,L c =HLH;

[0028] 4.3. Estimating mutual information using the Hilbert-Schmidt Independence Criterion (HSIC): HSIC = tr(K c L c ) / n 2 We use negative HSIC as the mutual information loss to quantify the dependency relationship between features and labels.

[0029] Furthermore, in step five above, the local feature encoder iterates over the features of each subgraph node by using two layers of GCNConv to extract subgraph features, combining BatchNorm and Dropout, and outputting the subgraph representation generated by attention pooling and global max pooling.

[0030] Compared with the prior art, the present invention has the following beneficial effects:

[0031] 1. This invention constructs a graph neural network architecture incorporating orthogonal projectors and Gaussian mutual information loss. In the graph encoder, it utilizes GATv2Conv combined with variational inference to extract probabilistic latent representations of nodes. Through two-level orthogonal constraints, latent features are separated into causal and confounding features. Furthermore, by centering the Gaussian kernel matrix, the mutual information between features and link labels is calculated, effectively addressing the problems of confounding interference and insufficient causal relationship capture in traditional link prediction. This mechanism can explicitly distinguish between causal and confounding features, significantly improving the AUC of link prediction and achieving accurate prediction of potential relationships.

[0032] 2. This invention employs a local feature fusion method, utilizing adjacency matrix exponentiation to extract k-hop neighborhood structures. It embeds the distance features from nodes to the target edge into the subgraph representation, combining a GCN encoder and attention pooling to generate local features containing topological semantics and distance information. This reduces the bottleneck of traditional GNN models in capturing long-distance dependencies. In sparse networks, it avoids feature loss due to missing edges, effectively improving feature integrity.

[0033] 3. The model architecture of this invention is compatible with various graph data scenarios, compressing the original feature dimension through hidden feature node representation. Simultaneously, this technique improves model performance with a lower parameter count, reduces computational resource consumption compared to traditional graph models, and reduces a significant amount of redundant feature interference through Gaussian mutual information-driven feature selection. It can be widely applied in fields such as social network relationship reasoning, knowledge graph completion, and protein interaction prediction. Attached Figure Description

[0034] Figure 1 This is a schematic diagram of the prediction model of the present invention;

[0035] Figure 2 This is a flowchart of the method of the present invention. Detailed Implementation

[0036] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments and accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments.

[0037] See Figure 1The overall model mainly includes four key parts: (1) a variational autoencoder based on GAT, used to generate latent features of Gaussian distribution; (2) an orthogonal projector composed of bilinear layers used to separate causal / non-causal features; (3) a mutual information estimator based on Gaussian mutual information calculation method used to constrain causal features; and (4) processing the subgraph of each edge to construct the local features of the model.

[0038] This invention provides a graph neural network link prediction method based on Gaussian mutual information. It utilizes the Gaussian latent features z output by a variational autoencoder to extract causal features c through orthogonal decomposition while maintaining the Gaussian distribution characteristics. Gaussian mutual information is used as a regularization term to inversely enhance the causal relationships of the causal features. Simultaneously, to improve the local structural performance of the entire model, a k-hop subgraph is added to construct local structural features. Finally, the causal features and local structural features are concatenated for link prediction.

[0039] Example

[0040] Specifically, in this embodiment, the object being processed is the Cora citation dataset. See also... Figure 2 This invention, for a specific object, employs a graph neural network link prediction method based on Gaussian mutual information, which specifically includes the following steps:

[0041] Step 1: Divide the graph data into training and test sets with negative sampling. Preprocess the training, test, and evaluation sets to extract local subgraphs and representations of each node in the subgraphs.

[0042] The specific steps of the preprocessing are as follows: 1.1 Using the adjacency matrix of the positively sampled edges in the training set, extract all nodes within k hops for each target edge (src, dst) in the training set, test set, and evaluation set. A node is defined as its neighboring nodes and itself, thus constructing a local subgraph node set. 1.2 Calculate the shortest path distance from each node to src and dst. If the distance exceeds k, set it to k+1, forming a two-dimensional distance feature [distsrc, distdst]. 1.3 Concatenate the original node features with the distance features to generate an extended feature vector [xnode, disistsrc, distdst].

[0043] Step 2: Extract the latent features of each node using a graph variational autoencoder, and enhance the Gaussian distribution characteristics of the Gaussian latent features through adaptive K-divergence.

[0044] The process of extracting latent features of nodes by the graph variational autoencoder includes the following steps:

[0045] 2.1 Extracting node features through a two-layer GATv2Conv network: Both GAT layers use the ELU function for non-linear activation. The first layer GATv2Conv (multi-head attention) maps the input features to the hidden_dim×heads dimension space; the second layer GATv2Conv (single-head attention) compresses the features to the output_dim dimension, and BatchNorm and Dropout are used to prevent overfitting.

[0046] For each node v in the graph, its feature vector x v The intermediate feature h is obtained by performing two layers of GAT processing. v :

[0047] h v =GAT(x v N v )

[0048] Where, N v It is x v The neighborhood of.

[0049] 2.2. Generate the mean μ and logvariance logvar of the latent variables using fully connected networks μ_net and logvar_net;

[0050] Variational inference layer assumes latent feature z∈R N×d The posterior distribution is a Gaussian distribution q(z|X,E)=N(μ,∑), and it is derived from the mean network μ_net and the log-variance network logσ. 2 _net generated parameters:

[0051] μ,logσ 2 =μ net (h v ),logσ 2 net (h v )

[0052] 2.3. The reparameterization technique z = μ + ∈ · exp(0.5 · log var) is used to generate the probabilistic latent representation z, where ∈ ~ N(0, I). This process forces z to follow a Gaussian distribution.

[0053] 2.4. Measure the loss of VAE using KL divergence to determine the posterior distribution and standard Gaussian prior p(z) = N(0, I d The difference between them. The KL divergence of a single sample is:

[0054]

[0055] Step 3: Using the Gaussian latent features extracted in Step 2, without altering the Gaussian properties, an orthogonal projector is used to separate the latent features into causal features and promiscuous features. A two-level orthogonal constraint is then used to ensure the independence of the causal and promiscuous features.

[0056] The specific steps of the orthogonal projector separating implicit functions include:

[0057] 3.1 The latent variable z is separated into causal feature c and confounding feature s through two-layer linear projection causal_proj and spurious_proj. The projection layer adopts orthogonal initialization and LayerNorm.

[0058] The latent feature z is passed through a causal projection layer Proj_c and a non-causal projection layer Prod_s to obtain causal features c and non-causal features s, respectively. The projection layer is a linear layer (including bias), which does not change the nature of the Gaussian distribution.

[0059] c = W c z+b c

[0060] If z is a Gaussian transform, then the linearly transformed c still follows a Gaussian distribution, with its mean and variance as follows:

[0061] E[c]=W c μ+b c

[0062]

[0063] 3.2. Introduce the feature preservation system feature_preserver, which enhances the robustness of representation by fusing the original latent features and projected features through gradient gating.

[0064] Step 4: Use Gaussian mutual information to estimate the mutual information between the separated hybrid features and the existence labels of the edges in Step 3, and use this as a regularization term:

[0065] The process of calculating the mutual information between promiscuous features and labels using the Gaussian mutual information calculation method includes the following steps:

[0066] 4.1 Calculate the Gaussian kernel matrix K = exp(-||XX) for the joint feature X. T ||| 2 / (2σ 2 )) Calculate the Gaussian kernel matrix L for label y.

[0067] Mutual information I(X;Y) is used to measure the dependency between cascaded features and edge labels Y. Since KL divergence forces z to follow a Gaussian distribution, and the subgraph features are generated through linear transformation and nonlinear activation, X approximately follows a multivariate Gaussian distribution, so mutual information is calculated using a Gaussian kernel.

[0068] 4.2. Centering the kernel matrix using the central matrix H = I⁻¹ / n: K c =HKH,L c =HLH.

[0069]

[0070] Wherein, the kernel matrix K X ∈R M×M and K Y ∈R M×M :

[0071]

[0072] 4.3. Estimating mutual information using the Hilbert-Schmidt Independence Criterion (HSIC): HSIC = tr(K c L c ) / n 2 We use negative HSIC as the mutual information loss to quantify the dependency relationship between features and labels.

[0073] Step 5: For the subgraph extracted in Step 1, use a local feature encoder to iterate the node representations of each node in the subgraph, where pooling layers are used to obtain the local structural features of each subgraph:

[0074] The local feature encoder iterates over the features of each subgraph node by using two layers of GCNConv to extract subgraph features, and combining BatchNorm and Dropout to output a subgraph representation generated by attention pooling (weighted softmax) and global max pooling.

[0075] Step 6: Integrate causal features and local structural features using an edge predictor to predict link probabilities.

[0076] For each edge (i,j), find the causal features c of its starting and ending points. i and c j Connecting them together, we obtain the causal feature representation c. ij And combine the subgraph feature g with the causal feature c ij Connecting them together yields the final feature representation f. ij :

[0077] f ij =[c i ;c j ;g]

[0078] The link prediction module processes the final feature representation through a multilayer perceptron with residuals and outputs the probability of edge existence.

[0079] Using the Cora citation dataset as the experimental subject, where 90% of the edges were set as the training set, the number of negatively sampled edges was seven times that of positively sampled edges. Finally, the AUC performance on the test set was 94.17%, which far exceeded the baseline GNN link prediction method.

[0080] The above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications made within the spirit and principles of the present invention are permitted.

[0081] Any modifications, equivalent substitutions, improvements, etc., made should be included within the scope of protection of this invention.

Claims

1. A link prediction method based on a graph neural network with Gaussian mutual information, characterized in that: Includes the following steps Step 1: Divide the graph data into training and test sets with negative sampling. Preprocess the training set, test set, and evaluation set to extract local subgraphs and the representations of each node in the subgraphs. Step 2: Extract the latent features of each node using a graph variational autoencoder, and enhance the Gaussian distribution characteristics of the Gaussian latent features using adaptive KL divergence. Step 3: Use a bilinear orthogonal projector to separate the Gaussian latent features into causal features and promiscuous features; Step 4: Use Gaussian mutual information to estimate the mutual information between the mixed features and the existence labels of the edges separated in Step 3 and use it as a regularization term; Step 5: For the subgraph extracted in Step 1, use a local feature encoder to iterate the node representation of each node in the subgraph, where the local structural features of each subgraph are obtained through a pooling layer. Step 6: Integrate causal features and local structural features through an edge predictor to predict link probabilities.

2. The link prediction method based on a graph neural network according to claim 1, characterized in that: In step one, preprocessing Includes the following steps: 1.

1. Using the original edges in the training set from step one to construct an adjacency matrix, extract all nodes within k hops for each target edge in the training set, test set, and evaluation set, and construct a local subgraph node set. 1.2 Calculate the shortest path distance from each node to src and dst. If the distance exceeds k, set it to k+1 to form a two-dimensional distance feature [distsrc, distdst]. 1.

3. Concatenate the original node features with the distance features to generate an extended feature vector.

3. The link prediction method based on a graph neural network according to claim 1, characterized in that: In step two, the process of extracting latent features of nodes by the graph variational autoencoder includes the following steps: 2.1 Extracting node features through a two-layer GATv2Conv network: Both GAT layers use the ELU function for non-linear activation. The first GATv2Conv layer maps the input features to the hidden_dim×heads dimension space; the second GATv2Conv layer compresses the features to the output_dim dimension, and BatchNorm and Dropout are used to prevent overfitting. 2.

2. Generate the mean μ and logvariance logvar of the latent variables using fully connected networks μ_net and logvar_net; 2.

3. The reparameterization technique z = μ + ∈·exp(0.5·logvar) is used to generate the probabilistic latent representation z, where ∈ ~ N(0,I); 2.

4. Measure the loss of VAE using KL divergence.

4. The link prediction method based on a graph neural network according to claim 1, characterized in that: In step three, the process of separating implicit functions by the orthogonal projector includes the following steps: 3.1 The latent variable z is separated into causal feature c and confounding feature s through two-layer linear projection causal_proj and spurious_proj. The projection layer adopts orthogonal initialization and LayerNorm. 3.

2. Introduce the feature preservation system feature_preserver, which enhances the robustness of representation by fusing the original latent features and projected features through gradient gating.

5. The graph neural network link prediction model and method based on Gaussian mutual information according to claim 1, characterized in that: In step four, the process of calculating the mutual information between promiscuous features and labels using the Gaussian mutual information calculation method. Includes the following steps: 4.1 Calculate the Gaussian kernel matrix K = exp(-||XX) for the joint feature X. T || 2 / (2σ 2 )) Calculate the Gaussian kernel matrix L for label y; 4.

2. Centering the kernel matrix using the central matrix H = I⁻¹ / n: K c =HKH,L c =HLH; 4.

3. Estimating mutual information using the Hilbert-Schmidt independence criterion: HSIC = tr(K c L c ) / n 2 We use negative HSIC as the mutual information loss to quantify the dependency relationship between features and labels.

6. The graph neural network link prediction method based on Gaussian mutual information according to claim 1, characterized in that: In step five, the local feature encoder iterates on the features of each subgraph node. It uses two layers of GCNConv to extract subgraph features, and combines BatchNorm and Dropout to output the subgraph representation generated by attention pooling and global max pooling.

Images