Robust heterogeneous graph neural network interpretation method with adaptive information constraint

Abstract

The invention discloses a robust heterogeneous graph neural network interpretation method with adaptive information constraint, and the method comprises the steps: firstly obtaining the relation between different types of entities through network data, enabling the relation to be represented by a node-edge-node mode, and carrying out the modeling of a graph; a graph representation of the graph is then obtained using a variational graph auto-encoder. And finally, capturing heterogeneous semantics in the graph representation by using a relationship-based interpretation generator, and generating an interpretation sub-graph for prediction. According to the method, robust graph representation is obtained by using variational inference, the generalization ability of the model is improved while the noise influence is reduced, and complex semantics in a heterogeneous graph can be captured.

Description

Technical Field

[0001] The present invention belongs to the field of graph neural network interpretability, and in particular to a robust heterogeneous graph neural network interpreter with adaptive information constraints. Background Art

[0002] Graph Neural Networks (GNNs), as a powerful tool for learning information from graph-structured data, have been applied in various real-world scenarios, such as social networks, citation networks, and recommender systems. GNNs integrate node features and graph structure information into a message passing framework, and have achieved remarkable performance in many tasks such as graph classification, node classification, and link prediction. Although GNNs are able to learn rich network information, their decision-making process is opaque and lacks interpretable explanations for human understanding. This opacity hinders their application in key areas related to fairness, privacy, and security. Therefore, studying the interpretability of GNNs can better understand their functions and promote improvements to beneficial results.

[0003] GNN explainers take the original graph and the model as input, aiming to identify the key subgraphs that have a significant impact on the prediction. Existing GNN explainers can be divided into ex post methods and built-in methods. Ex post methods apply explanation techniques or build explanation models based on trained models to measure the contributions of different components to explain the working mechanism or decision principle. Common ex post methods include perturbation-based methods and surrogate model-based methods. Built-in methods generate explanations during model training, where the generated graph serves as an explanation of the target prediction and is expected to meet specific task goals. Common built-in methods include prototype learning-based methods and graph generation-based methods. Since built-in methods are tailored for specific models and need to be trained separately for different scenarios, they lack generalizability, while ex post methods can be applied to most scenarios.

[0004] However, real-world data faces many challenges that limit the applicability of these explanation methods in practical applications. First, real-world graphs are usually heterogeneous and contain multiple types of nodes and edges, which increases the complexity of the structure. Second, real-world graph data is noisy. Specifically, the graph structure may contain edges that are added or deleted due to noise. At the same time, node features will also be disturbed by noise, making them unrealistic. Noise is a crucial issue for heterogeneous graphs because the inherent heterogeneity between node types causes noisy edges to carry erroneous heterogeneous relationship information and further propagate during message passing.

[0005] Existing work has proven that the presence of noise can seriously damage the performance of interpretables and increase the risk of generating incorrect explanations. However, no interpretable method has studied noise on heterogeneous graphs. At the same time, noise exacerbates the irregularity of the graph structure and changes the importance of nodes, which is not applicable to previous interpretable methods that require strict structural constraints (such as size, budget, connectivity). They will take noisy edges into account and exclude some correct edges under the influence of node importance. How to deal with the irregularity of the graph in the case of noise and adaptively explore key subgraphs also poses a great challenge to interpretable methods. Summary of the invention

[0006] In response to the above challenges, this paper proposes a robust heterogeneous graph neural network interpreter with adaptive information constraints, focusing on the ex post method and developing a GNN interpreter that can generate high-quality explanations in different real-world scenarios, improve the robustness of the interpretable, and enhance the interpretability in heterogeneous graphs.

[0007] The present invention comprises the following steps:

[0008] S1. Obtain the relationship between different types of entities through network data. For example, in the citation network, the entity types are papers, authors, and conferences, and the relationships in the network are "author-writing-paper", "paper-published-conference", etc. These relationships are represented by "node-edge-node" and modeled into a graph.

[0009] And the edge matrix E is composed of edges s ; The graph is composed of entity information The feature matrix X of .

[0010] S2. Use variational graph autoencoder to obtain graph Graph representation of .

[0011] Through variational inference, the statistics of latent variables can be effectively inferred from the latent space rather than the observation space, thus obtaining robust graph information. Given the graph observed in step S1 The variational graph autoencoder assumes that the data is generated through two steps of independent and identical distribution. First, an initial Gaussian distribution is used to generate the prior distribution p(Z) of the latent variable Z; second, the graph The joint distribution of and variable Z is Where θ represents the variational graph autoencoder parameter that needs to be optimized, and θ is optimized to maximize the observed graph The log-likelihood of According to the Jünsen inequality, The derivation is as follows:

[0012]

[0013] in, It is used to approximate the true posterior distribution The posterior distribution of represents expectation, λ represents requirement Parameters estimated in ; Evidence lower bound It is expressed as follows:

[0014]

[0015] in, denotes the distribution observed by Z and KL(·||·) denotes the KL divergence.

[0016] Use a variational graph autoencoder composed of a graph convolutional neural network to estimate the parameters in the above evidence lower bound, the sampled latent variable Z and the reconstructed adjacency matrix The calculation process is as follows:

[0017] Z (l) =GCN(Z (l-1) , A), Z (0) =X

[0018] μ=GCN μ (Z (l) , A), logσ=GCN σ (Z (l) , A)

[0019]

[0020] Among them, GCN is a graph convolutional neural network, A is a graph The adjacency matrix, Z (l) is the variable of the lth layer, that is, the graph representation of this layer, l is the number of layers of the model, μ and σ are Gaussian distribution parameters, and Sigmoid is the activation function; the adjacency matrix obtained by reconstruction Compute the loss and train the variational graph autoencoder.

[0021] Since the goal of variational inference is to make the posterior distribution To capture saliency map information while mitigating the impact of noise, no separate heterogeneous semantics learning module is designed. Instead, the sampled latent variables Z are fed into a relation-based explanation generator where complex semantics on heterogeneous graphs are learned.

[0022] S3. Use a relation-based explanation generator to capture the heterogeneous semantics in the graph representation and generate an explanation subgraph for the prediction.

[0023] Assume that the explanation subgraph is a Gilbert random graph where the edges are conditionally independent and the edge matrix Es Each element e in ij is a binary variable indicating whether the edge is included in the subgraph. i With node v j When there is an edge (i, j) between them, e ij =1, otherwise e ij = 0, the Gibert random graph variable is decomposed into:

[0024]

[0025] Among them, e ij is a Bernoulli distribution Bern(θ ij ) binary variable, p(e ij ) represents the probability of the existence of edge (i, j); the binary variable e ij is discrete, and the reparameterization method is applied to relax it into a continuous variable between 0 and 1, as follows:

[0026]

[0027] Among them, τ is the temperature coefficient of smooth optimization, α ij is the parameter, ∈ conforms to the uniform distribution Uniform(0,1).

[0028] For each edge type Assign an edge type embedding And use edge type embedding and node embedding to calculate α ij :

[0029]

[0030] Among them, a and W represent learnable parameters, z i The graph represents the i-th value in Z, represents the neighbors of node i, W r is a learnable edge-type weight matrix and ReLU is the activation function.

[0031] All α ij Composition probability matrix M p , from M p Generating Explanation Subgraphs by Sampling

[0032]

[0033] A s =M p ⊙A;X s =M p ⊙X

[0034] A sTo interpret the adjacency matrix of the subgraph, X s To explain the characteristic matrix of the subgraph, ⊙ represents element-wise product.

[0035] Furthermore, the variational graph autoencoder and the explanation generator constitute a heterogeneous graph explanation network model, and the loss function of the network model is constructed as follows:

[0036] The predictor is composed of a neural network f, explaining the subgraph Get the prediction result through the predictor The extended self-information constraint is used to optimize the explanation generator composed of the neural network α, and the loss function is as follows:

[0037]

[0038] Represents the prediction result obtained by predictor f The distribution of Represents the explanation generator α Distribution of

[0039] The total loss function is as follows:

[0040]

[0041] Beneficial effects of the present invention:

[0042] This paper proposes a robust heterogeneous graph neural network interpreter with adaptive information constraints. Variational inference is used to obtain a robust graph representation, which reduces the impact of noise and improves the generalization ability of the model. The neural network is used to parameterize the interpretation subgraph generation process, and the weights of heterogeneous relations are integrated in the process of generating the interpretation subgraph, so that it can capture the complex semantics in the heterogeneous graph. The most sufficient and necessary information is obtained through self-information constraints. In addition, our method can explain predictions at the node, edge, and graph levels. BRIEF DESCRIPTION OF THE DRAWINGS

[0043] Figure 1 This is a network structure diagram of the heterogeneous graph neural network interpreter of the present invention. DETAILED DESCRIPTION

[0044] Example 1

[0045] In this embodiment, a robust heterogeneous graph neural network interpreter with adaptive information constraints is disclosed, the process is as follows Figure 1 As shown, including:

[0046] S1: Obtain the relationships between different types of entities in the real world and form heterogeneous relational data, which are then modeled into heterogeneous graphs.

[0047] In the academic network, there are many types of nodes, such as authors, conferences, journals, etc. Among them, the heterogeneous relationships included are "author-conference", "author-journal", etc. This embodiment models the heterogeneous relationships into a heterogeneous graph Contains multiple types of nodes and edge ε, where X represents the graph composed of entity information The feature matrix of .

[0048] S2: Obtaining high-quality graph representations using variational graph autoencoders.

[0049] For heterogeneous graphs This example uses a graph convolutional neural network (GCN) to obtain the representation of the node:

[0050] Z (l) =GCN(Z (l-1) ,A),Z 0 =X

[0051] Among them, l is the number of neural network layers, Z (l) is the variable of the lth layer, that is, the graph representation of this layer, A is the adjacency matrix, and X is the feature matrix. Since the optimization process of the variational graph autoencoder requires the expectation and variance of the data, this embodiment uses GCN to estimate the expectation μ and variance σ of the data:

[0052] μ=GCN μ (Z (l) ,A),logσ=GCN σ (Z (l) ,A)

[0053] Subsequently, in order to make the inferred data as close as possible to the original graph structure, it is necessary to obtain the reconstructed graph adjacency matrix:

[0054]

[0055] Sigmoid is the activation function.

[0056] Subsequently, this embodiment uses the following objective to optimize the variational graph autoencoder:

[0057]

[0058] in, denotes the distribution observed by Z and KL(·||·) denotes the KL divergence.

[0059] S3: Use a relation-based explanation generator to capture heterogeneous semantics and generate explanations for predictions.

[0060] This embodiment uses the Gilbert random graph variable decomposition method to decompose the heterogeneous graph into a probability multiplication form:

[0061]

[0062] Among them, e ij is a Bernoulli distribution Bern(θ ij ) binary variable, p(e ij ) represents the probability of the existence of edge (i, j); the binary variable e ij is discrete, and the reparameterization method is applied to relax it into a continuous variable between 0 and 1, as follows:

[0063]

[0064] Among them, τ is the temperature coefficient of smooth optimization, α ij is the parameter, ∈ conforms to the uniform distribution Uniform(0,1).

[0065] This embodiment calculates α by fusing edge types. ij , for each edge type Assign an edge type embedding Utilize both edge type embedding and node embedding to calculate:

[0066]

[0067] Among them, a and W represent learnable parameters, z i The graph represents the i-th value in Z, represents the neighbors of node i, W r is a learnable edge-type weight matrix, and ReLU is the activation function;

[0068] All α ij Composition probability matrix M p , from M p Generating Explanation Subgraphs by Sampling

[0069]

[0070] A s =M p ⊙A;X s =M p ⊙X

[0071] A s To interpret the adjacency matrix of the subgraph, X s To explain the characteristic matrix of the subgraph, ⊙ represents element-wise product.

[0072] The predictor is composed of a neural network f, explaining the subgraph Get the prediction result through the predictor The extended self-information constraint is used to optimize the explanation generator composed of the neural network α, and the loss function is as follows:

[0073]

[0074] Represents the prediction result obtained by predictor f The distribution of Represents the explanation generator α Distribution of

[0075] The total loss function is as follows:

[0076]

[0077] This embodiment conducts experiments on three real-world datasets: DBLP, ACM, and Freebase. The target nodes are divided into training set, validation set, and test set in a ratio of 24:6:70.

[0078] This embodiment uses the node classification task to evaluate the generation quality of the explanation subgraph, and compares the prediction results of the original graph with the prediction results of the explanation subgraph. In the adopted dataset, the target node type of DBLP is the author, the target node type of ACM is the paper, and the target node type of Freebase is the movie.

[0079] This embodiment uses the mean absolute error (MAE) and the root mean square error (RMSE) to evaluate the performance of the method disclosed in this embodiment. i , the prediction result of the original image is The prediction result of the explanation subgraph is The decision function is The calculation method is as follows:

[0080]

[0081]

[0082] The detection results of the method disclosed in this embodiment under different noise ratios are compared with those of PGExplainer, GNNExplainer, PGM-Explainer, V-InfoR and PGE-Relation. The results on the DBLP dataset are shown in Table 1, the results on the ACM dataset are shown in Table 2, and the results on the Freebase dataset are shown in Table 3. The method disclosed in this embodiment outperforms other baselines in most detection results and achieves the best performance on the DBLP and Freebase datasets. Since the edge types in the DBLP dataset are similarly distributed, the dataset exhibits higher heterogeneity, which enhances the ability of the module designed in this embodiment to capture heterogeneous information. At the same time, as a plug-and-play module, this embodiment can be easily extended to other parameterized explanation methods to generate explanations for heterogeneous graphs. On the small and medium-scale datasets DBLP and ACM, the explanation methods based on original features, such as GNNExplainer, are more susceptible to noise, which indicates that the original features are more susceptible in smaller graphs. Since this embodiment generates a potential variational graph representation, it can better mitigate the impact of noise, which is why the interpreter V-InfoR based on the potential representation performs well in many scenarios. Under the guidance of adaptive information constraints, this embodiment can adaptively select redundant edges and noise edges while excluding redundant edges, thereby providing the best explanation for the prediction model.

[0083] Table 1

[0084]

[0085] Table 2

[0086]

[0087]

[0088] Table 3

[0089]

[0090] The robust heterogeneous graph neural network interpreter with adaptive information constraints disclosed in this embodiment adopts a variational graph autoencoder to obtain a robust graph representation and integrates rich relational information to capture the complexity of different node and edge types, thereby generating accurate heterogeneous graph interpretations.

Claims

1. A robust heterogeneous graph neural network interpretation method with adaptive information constraints, characterized in that: The following steps are involved: S1. Obtain the relationship between different types of entities through network data, represent the relationship with node-edge-node, and model it into a graph S2. Use variational graph autoencoder to obtain graph Graph representation of ; S3. Use a relation-based explanation generator to capture the heterogeneous semantics in the graph representation and generate an explanation subgraph for the prediction.

2. A robust heterogeneous graph neural network interpretation method with adaptive information constraints according to claim 1, characterized in that: The step 1 also includes: forming an edge matrix E from the edges s ; The graph is composed of entity information The feature matrix X of .

3. A robust heterogeneous graph neural network interpretation method with adaptive information constraints according to claim 2, characterized in that: The variational graph autoencoder is used to obtain the graph The specific implementation process is as follows: Use an initialized Gaussian distribution to generate the prior distribution p(z) of the latent variable Z; Figure The joint distribution of and variable Z is Where θ represents the variational graph autoencoder parameter that needs to be optimized, and θ is optimized to maximize the observed graph The log-likelihood of According to the Jünsen inequality, The derivation is as follows: in, It is used to approximate the true posterior distribution The posterior distribution of represents expectation, λ represents requirement Parameters estimated in ; Evidence lower bound It is expressed as follows: in, represents the distribution observed by Z, KL(·||·) represents the KL divergence; Use a variational graph autoencoder composed of a graph convolutional neural network to estimate the parameters in the above evidence lower bound, the sampled latent variable Z and the reconstructed adjacency matrix The calculation process is as follows: Z (l) =GCN(Z (l-1) ,A),Z (0) =S μ=GCN μ (FROM (l) ,A),logσ=GCN σ (FROM (l) ,AND) Among them, GCN is a graph convolutional neural network, A is a graph The adjacency matrix, Z (l) is the variable of the lth layer, that is, the graph representation of this layer, μ and σ are Gaussian distribution parameters, and Sigmoid is the activation function; The adjacency matrix obtained by reconstruction Compute the loss and train the variational graph autoencoder.

4. A robust heterogeneous graph neural network interpretation method with adaptive information constraints according to claim 3, characterized in that: The specific implementation process of step S3 is as follows: Assume that the explanation subgraph is a Gilbert random graph where the edges are conditionally independent and the edge matrix E s Each element e in ij is a binary variable indicating whether the edge is included in the subgraph. i With node v j When there is an edge (i, j) between them, e ij =1, otherwise e ij =0, the Gilbert random graph variable is decomposed into: Among them, e ij is a Bernoulli distribution Bern(θ ij ) binary variable, p(e ij ) represents the probability of the existence of edge (i, j); the binary variable e ij is discrete, and the reparameterization method is applied to relax it into a continuous variable between 0 and 1, as follows: Among them, τ is the temperature coefficient of smooth optimization, α ij is the parameter, ∈ conforms to the uniform distribution Uniform(0,1); For each edge type Assign an edge type embedding And use edge type embedding and node embedding to calculate α ij : Among them, a and W represent learnable parameters, z i The graph represents the i-th value in Z, represents the neighbors of node i, W r is a learnable edge-type weight matrix, and ReLU is the activation function; All α ij Composition probability matrix M p , from M p Generating Explanation Subgraphs by Sampling A s =M p ⊙A;X s =M p ⊙X A s To interpret the adjacency matrix of the subgraph, X s To explain the characteristic matrix of the subgraph, ⊙ represents element-wise product.

5. A robust heterogeneous graph neural network interpretation method with adaptive information constraints according to claim 4, characterized in that: The variational graph autoencoder and the explanation generator constitute a heterogeneous graph explanation network model, and the loss function of the network model is constructed as follows: The predictor is composed of a neural network f, explaining the subgraph Get the prediction result through the predictor The extended self-information constraint is used to optimize the explanation generator composed of the neural network α, and the loss function is as follows: Represents the prediction result obtained by predictor f The distribution of Represents the explanation generator α Distribution of The total loss function is as follows:

Images