Network motif-based explainable graph neural network method

Abstract

The application belongs to the field of explainability of graph neural networks, and discloses a graph neural network explainability method based on network motifs, which can identify low-order structure motifs that frequently appear in the graph neural network, prove that the motifs have a very important influence on the actual prediction result through counterfactual reasoning, thereby playing a key role in the explainability of the graph neural network, and can provide better explanation services. The application discusses the influence of model hyperparameters and explanation size on model performance, and shows better performance than other baselines on synthetic and real datasets. The method created by the application is model-agnostic, and therefore is theoretically applicable to all graph neural networks.

Description

Technical Field

[0001] This invention belongs to the field of interpretability of graph neural networks, specifically a method for achieving interpretability of graph neural networks using network motif technology and counterfactual reasoning optimization methods. Background Technology

[0002] Graph neural networks (Graph Neural Networks) demonstrate superior performance on various types of graph data, such as social networks, citation networks, and molecular networks. Graph Neural Networks can also effectively capture node features and graph topology, which is beneficial for graph / node classification and link prediction. Understanding and interpreting the prediction principles of Graph Neural Networks is key to unlocking their security, reliability, and widespread application. Unlike Euclidean data, graph data can have any size and complex topology, so traditional deep learning methods cannot directly utilize graph data.

[0003] Current methods for interpretability of graph neural networks can be categorized into two types: general model-agnostic methods (Rex Ying, Dylan Bourgeois, Jiaxuan You, Marinka Zitnik, and Jure Leskovec. 2019. GNN Explainer: A Tool for Post-hoc Explanation of Graph Neural Networks.) and instance-level methods (Dongsheng Luo, Wei Cheng, Dongkuan Xu, Wenchao Yu, Bo Zong, Haifeng Chen, and Xiang Zhang. 2020. Parameterized Explainer for Graph Neural Network. and Wanyu Lin, Hao Lan, and Baochun Li. 2021. Generative Causal Explanations for Graph Neural Networks, Vol. 139.). Model-agnostic methods provide an interpretable explanation for a given instance by training a soft mask; instance-level methods generate subgraphs by identifying important edge or node features. However, most of them ignore the unique high-order structural features in graph data.

[0004] The graph neural network interpretability method based on network models employs a network motif-aware subgraph generation module and a counterfactual optimization layer to provide high-quality counterfactual interpretations for graph neural networks, while also focusing on the higher-order features of graph data. Motifs possess characteristics of real-world significance, frequent occurrence, and low-order structure. This method generates a learnable mask matrix from the input graph, constructs a motif adjacency matrix to adjust subgraph generation, and continuously updates the learnable mask matrix using counterfactual reasoning optimization to obtain interpretable subgraphs, thus achieving interpretability for the graph neural network. Summary of the Invention

[0005] This invention primarily addresses the issue of high-quality counterfactual interpretability for graph neural networks (GNNs). It transforms the opacity of GNNs to users into transparency by leveraging the real-world significance, high frequency of occurrence, and low-order structure of network motifs. This enhances the predictive power of interpretable GNNs and improves the security and reliability of deep model deployments. Furthermore, it addresses the higher-order characteristics of graph data. Since there are no fixed node orders or reference points in the graph, a graph neural network interpretability method based on network motifs is proposed. This method employs a model-aware subgraph generation module and a counterfactual optimization layer. The goal of the interpretation task is to find the most important subgraph related to the prediction from a given input graph, and then learn a graph classifier to predict the estimated label of the input graph.

[0006] The technical solution of this invention:

[0007] An interpretable graph neural network method based on network motifs, comprising the following steps:

[0008] Step 100: Generate an interpretable subgraph G predicted by GNNs based on the instance adjacency matrix A of the input graph G. S By providing the adjacency matrix W of the motif M Constructing a motif perception matrix to guide subgraph G S generate;

[0009] Step 101: Initialize and construct a random mask matrix M with the same shape as the graph instance adjacency matrix A according to an N×N normal distribution. Then, calculate the modal adjacency matrix W using the random mask matrix M according to step 102. M =W(M);

[0010] Step 102, use the modal adjacency matrix W M Interpretable subgraphs G used as motif-aware matrices to guide the generation of GNN predictions S The formula for calculating the adjacency matrix of a motif containing node (i,j) is: Among them, (W) M ) i,jThis represents the sum of the weights of the module containing nodes i and j; 1(·) is a truth indicator function, which takes the value 1 if the statement in 1 is true, and 0 otherwise; υ is the entire node set, and χυ is a selection function used to select a subset of υ;

[0011] Step 103: Use the Sigmoid activation function to normalize the random mask matrix M, mapping the variables to the range of 0 and 1;

[0012] Step 104 yields the interpretable subgraph G. S The generation formula is as follows: Combine the random mask matrix M from step 101 and the modal adjacency matrix W from step 102. M Multiplying the hyperparameter δ of the adjustment phantom on the masking effect by a matrix yields M′, i.e., M′=M·W M • δ; The adjacency matrix W of the phantom is controlled by changing the hyperparameter δ. M For the generated interpretable subgraph G S The impact;

[0013] Step 105: Calculate the interpretable subgraph G from step 104. S Adjacency matrix A s Multiply the input graph instance adjacency matrix A with M′ obtained in step 104 to obtain the subgraph adjacency matrix A. s ,Right now Based on the subgraph adjacency matrix A s Calculate the counterfactual adjacency matrix A with the graph instance adjacency matrix A. cf A cf =AA s ;

[0014] Step 106, use the subgraph adjacency matrix A obtained in step 105. s The set of elements constitutes a generating subgraph, and returns a generating interpretable subgraph G. s The reverse fact adjacency matrix A obtained in step 105 is... cf The set of numbers forms a counterfactual graph, and the counterfactual graph G is returned. cf ;

[0015] Step 200, counterfactual optimization; Step 201 effectively limits the size of the interpretation, preventing its infinite expansion to satisfy counterfactuality; simultaneously, it controls the difference between the counterfactual diagram and the original diagram, ensuring that the variation in the counterfactual interpretation is minimized. Calculate the maximum counterfactual reasoning condition L. cf and optimization model;

[0016] Step 201, obtain the counterfactual graph G based on step 106. cf The probability of c occurring is predicted in the middle. Predict the probability of occurrence of c in the input graph G The absolute value of the difference between the two graphs represents the predicted discrepancy between the two graphs.

[0017] Step 202, use the distance function D(·) to reduce the counterfactual graph G. cf The difference between the input graph G and the input graph G, i.e., D(G) cf ,G); Fact graph G cf The difference from the original graph G can be divided into two modes: the difference in explained size D(·). size And explain the sparsity difference D(·) sparsity ; Apply L1 norm to calculate fact graph G cf Explain the size L1(G) cf The difference D(·) between the original graph G and the interpreted size L1(G) size That is, D(G) cf ,G) size =L1(G cf ) / L1(G), to explain the sparsity differences

[0018] Step 203, calculate the maximum counterfactual inference condition: First, calculate the prediction gap diff(G) between the counterfactual graph and the original graph calculated in step 201. cf Then, in step 202, the difference D(G) between the counterfactual graph and the original graph is calculated. cf Finally, the hyperparameter λ is used to adjust the balance between the calculation results of step 201 and step 202, i.e., argmaxL cf =λDiff(P ψ (c|G cf ),P ψ (c|G))+D(G,G cf );

[0019] Step 204: Calculate the counterfactual optimization model. Based on the ratio of true to counterfactual explanations according to the hyperparameter α, the prediction of the explanatory subgraph is... The hyperparameter λ in step 202 and the maximum counterfactual reasoning condition in step 203 are used to calculate the optimized model as follows:

[0020]

[0021] The beneficial effects of this invention are as follows: The interpretability method of this invention achieves interpretability of graph neural networks based on the motif adjacency matrix and counterfactual optimization method, transforming the predictive behavior of graph neural networks into causal relationships that are understandable to humans. The first part involves obtaining the motif adjacency matrix. First, a learnable random mask matrix with the same shape as the input graph's adjacency matrix is ​​obtained. Then, the sum of node weights contained within the selection function is determined based on the selection function, resulting in a motif adjacency matrix generation function. Subgraph generation rules are obtained based on the combined effect of the motif adjacency matrix, the random mask matrix, and the hyperparameters that influence the motif on the mask. By changing the hyperparameter values, the influence of the motif adjacency matrix on subgraph generation can be controlled. Finally, the subgraph and counterfactual graph are obtained based on the input graph's adjacency matrix and the subgraph generation rules. The second part involves counterfactual reasoning. Through counterfactual reasoning, the possibility of generating "false relationships" is reduced using negation, resulting in "causal relationships." In graph neural network interpretability, when the graph neural network remains unchanged, the predictions of the subgraph and the original graph are the same; however, when the input undergoes minimal change, the predictions will change accordingly. This invention establishes a method for using network motifs in graph neural networks (Graph Neural Networks). This method identifies frequently occurring low-order structural motifs within the Graph Neural Network and demonstrates, through counterfactual reasoning, that these motifs have a significant impact on actual prediction results. This plays a crucial role in the interpretability of Graph Neural Networks and can provide better interpretation services. This invention discusses the impact of model hyperparameters and interpretation size on model performance, demonstrating superior performance compared to other baselines on both synthetic and real datasets. The method established in this invention is model-agnostic and therefore theoretically applicable to all Graph Neural Networks. Attached Figure Description

[0022] Figure 1 This is a flowchart illustrating the implementation of the graph neural network interpretability method based on network motifs provided by the present invention.

[0023] Figure 2 This is the flowchart for implementing step 100;

[0024] Figure 3 This is the flowchart for implementing step 200. Detailed Implementation

[0025] To make the technical problems solved by this invention, the technical solutions adopted, and the technical effects achieved clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative of the invention and not intended to limit it. Furthermore, it should be noted that, for ease of description, only the parts relevant to the invention are shown in the accompanying drawings, not all of them.

[0026] Figure 1This is a flowchart illustrating the implementation of the graph neural network interpretability method based on network motifs provided by this invention. For example... Figure 1 As shown, the graph neural network interpretability method based on network motifs provided in this embodiment of the invention includes:

[0027] Step 100: Generate an interpretable subgraph G predicted by GNNs based on the instance adjacency matrix A of the input graph G. S By providing the adjacency matrix W of the motif M Constructing a motif perception matrix to guide subgraph G S generate.

[0028] like Figure 2 As shown, step 100 includes the following steps 101 to 106:

[0029] Step 101: Initialize and construct a random mask matrix M with the same shape as the graph instance adjacency matrix A according to an N×N normal distribution. Then, calculate the modal adjacency matrix W using the random mask matrix M according to step 102. M =W(M);

[0030] In this step, the random mask matrix M is a learnable mask matrix that can be updated and learned according to the optimization model.

[0031] Step 102, use the modal adjacency matrix W M The generation of subgraph G is guided by the motif perception matrix. S , (W M ) i,j This represents the sum of the weights of the motifs containing nodes i and j. 1(·) is a truth indicator function; it takes the value 1 if the statement in 1 is true. υ is the entire node set, and χυ is a selection function used to select a subset of υ. The adjacency matrix of the motif containing node (i,j) is calculated as follows:

[0032] Step 103: Use the Sigmoid activation function to normalize the random mask matrix M, mapping the variables to the range of 0 and 1.

[0033] Step 104 yields the interpretable subgraph G. S The generation formula is as follows: Combine the random mask matrix M from step 103 and the modal adjacency matrix W from step 102. M Multiplying the hyperparameter δ of the adjustment phantom on the masking effect by a matrix yields M′, i.e., M′=M·W M • δ; The adjacency matrix W of the phantom is controlled by changing the hyperparameter δ. M The impact on the generated subgraph.

[0034] In this embodiment, the subgraph generation formula can be expressed as M′=M·W M·δ, where δ represents the degree of influence of the phantom perception matrix on the subgraph interpretation. By adjusting its size, the subgraph generation formula can be adjusted.

[0035] Step 105, calculate subgraph G from step 104. S Adjacency matrix A s Multiply the adjacency matrix A of the input graph instance by M′ obtained in step 104 to obtain the subgraph adjacency matrix A. s ,Right now Based on the subgraph adjacency matrix A s Calculate the counterfactual adjacency matrix A with the graph instance adjacency matrix A. cf A cf =AA s ;

[0036] In this embodiment, the formula corresponding to the subgraph adjacency matrix is ​​expressed as A. s =A·M′, which can also yield the counterfactual graph matrix A. cf =A-A·M′.

[0037] Step 106, use the subgraph adjacency matrix A obtained in step 105. s The set of elements constitutes a generating subgraph, and the generating subgraph G is returned. s The counterfactual adjacency matrix A obtained in step 105 is... cf The set of numbers forms a counterfactual graph, and the counterfactual graph G is returned. cf ;

[0038] In this embodiment, G s ←{Α s}

[0039] like Figure 3 As shown, counterfactual optimization yields the conditions and optimization model for counterfactual reasoning.

[0040] Step 200: Limit the size of the interpretation to prevent its infinite expansion, thus satisfying counterfactuality and obtaining the maximum counterfactual reasoning condition and optimized model. Step 200 includes the following steps 201 to 204:

[0041] Step 201, from step 106, we obtain the counterfactual graph G. cf According to the counterfactual diagram G cf The probability of c occurring is predicted in the middle. Predict the probability of occurrence of c in the input graph G The absolute value of the difference between the two graphs represents the predicted discrepancy between the two graphs.

[0042] In this embodiment, the difference function can be represented as Diff(G) cf ,G)=|P ψ (c|Gcf )-P ψ (c|G)|

[0043] Step 202, use the distance function D(·) to reduce the counterfactual graph G. cf The difference between the input graph G and the input graph G, i.e., D(G) cf ,G); Fact graph G cf The difference from the input graph G is specifically divided into two modes: the difference in interpretation size D(·). size And to explain the sparsity difference D(·) sparsity Calculate the fact graph G using the L1 norm. cf Explain the size L1(G) cf The difference D(·) between the input graph G and the explanatory size L1(G) is explained. size That is, D(G) cf ,G) size =L1(G cf ) / L1(G), to explain the sparsity differences

[0044] Step 203, using hyperparameter adjustment: the predicted gap between the counterfactual graph and the input graph G in step 202, and the gap between the counterfactual graph and the original graph in step 203. Finally, the maximum counterfactual inference condition is obtained.

[0045] Step 203, calculate the maximum counterfactual inference condition: First, calculate the prediction gap diff(G) between the counterfactual graph and the original graph calculated in step 201. cf Then, in step 202, the difference D(G) between the counterfactual graph and the original graph is calculated. cf Finally, the hyperparameter λ is used to adjust the balance between the calculation results of step 201 and step 202, i.e., argmaxL cf =λDiff(P ψ (c|G cf ),P ψ (c|G))+D(G,G cf );

[0046] Step 204: Calculate the counterfactual optimization model. Based on the ratio of true to counterfactual explanations according to the hyperparameter α, the prediction of the explanatory subgraph is... The hyperparameter λ in step 202 and the maximum counterfactual reasoning condition in step 203 are used to calculate the optimized model as follows:

[0047]

[0048] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications to the technical solutions described in the foregoing embodiments, or equivalent substitutions for some or all of the technical features, do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. An interpretable graph neural network method based on network motifs, characterized in that, The steps are as follows: Step 100: Generate interpretable subgraphs predicted by GNNs based on the instance adjacency matrix A of the input graph G. By providing the adjacency matrix of the motif Constructing a motif perception matrix to guide subgraphs generate; Step 101, according to A random mask matrix M with the same shape as the graph instance adjacency matrix A is constructed using a normal distribution initialization. Then, the modal adjacency matrix is ​​calculated using the random mask matrix M according to step 102. ; Step 102, use the module adjacency matrix Interpretable subgraphs used as motif-aware matrices to guide the generation of GNN predictions. The formula for calculating the adjacency matrix of a motif containing node (i,j) is: ;in, This represents the sum of the weights of the module containing nodes i and j; It is a truth indicator function; if the statement in step 1 is true, it takes the value 1; otherwise, it takes the value 0. It is the entire set of nodes. It is a selection function used to select... A subset of; Step 103: Use the Sigmoid activation function to normalize the random mask matrix M, mapping the variables to the range of 0 and 1; Step 104, obtain the interpretable subgraph The generation formula is as follows: Combine the random mask matrix M from step 101 and the modal adjacency matrix from step 102... Adjusting the hyperparameters of the phantom for masking effect Matrix multiplication yields ,Right now By changing hyperparameters Control phasor adjacency matrix For the generated interpretable subgraph The impact; Step 105: Calculate the interpretable subgraph from step 104. adjacency matrix The input graph instance adjacency matrix A is compared with the one obtained in step 104. Multiplication yields the adjacency matrix of the subgraph ,Right now According to the subgraph adjacency matrix Calculate the counterfactual adjacency matrix using the graph instance adjacency matrix A. ,Right now ; Step 106, use the subgraph adjacency matrix obtained in step 105. The set constitutes a generating subgraph, and returns a generating interpretable subgraph. The counterfactual adjacency matrix obtained in step 105 The set constitutes a counterfactual graph, and returns the counterfactual graph. ; Step 200, Counterfactual Optimization; Calculate the largest counterfactual reasoning condition. and optimization model; Step 201: Obtain the counterfactual diagram based on step 106. The probability of c occurring is predicted in the middle. Predict the probability of occurrence of c in the input graph G The absolute value of the difference between the two graphs represents the predicted discrepancy between the two graphs. ; Step 202, using the distance function To reduce counterfactual diagrams The difference between the input graph G and the input graph G, i.e. Fact chart The differences from the original graph G can be divided into two modes: differences in explanatory size. And explaining the differences in sparsity ;application Norm calculation fact graph Explain size Size of the original graph G Differences ,Right now Explaining sparsity differences ; Step 203, calculate the maximum counterfactual reasoning condition: First, calculate the prediction gap between the counterfactual graph and the original graph calculated in step 201. Then, step 202 calculates the difference between the counterfactual graph and the original graph. Finally, hyperparameters were used. Adjust the balance between the calculation results of step 201 and step 202, that is... ; Step 204, calculate the counterfactual optimization model based on the hyperparameters. Balancing the proportion of factual and counterfactual interpretations, the prediction of the explanatory subgraph is... In step 202, hyperparameters The largest counterfactual reasoning condition in step 203 is used to calculate the optimized model as follows: 。

Images