Graph neural network explanation method based on generative adversarial network

Abstract

The application provides a graph neural network explanation method based on a generative adversarial network, wherein the method obtains node representation of graph data by using a GCN model first, then obtains edge representation vectors by combining a negative sampling model and graph structure to process original graph data, and finally obtains potential subgraphs with explainability to prediction results of the GCN model by using an MLP model and the GCN model to explain and classify the edge representation vectors. The method provided by the application only needs 14.7s to train graph data with 1000 nodes, and the training efficiency is about 140 times higher than that of the existing training method (35min). Furthermore, the explanation method provided by the application can explain graph data with 5000 nodes at the same time. Through the above method, the graph neural network explanation method based on the generative adversarial network can batch explain graph data.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of Internet security, and particularly relates to a graph neural network explanation method based on a generative adversarial network. BACKGROUND

[0002] Deep learning is a kind of machine learning, and machine learning is a necessary path to realize artificial intelligence. The most commonly used models in deep learning include CNN, RNN, etc. These models have achieved excellent results in both CV and NLP fields. The above-mentioned models mainly process dimensional structures, such as the processing object of CNN is a picture, and the features of the picture can be extracted by using the CNN model. Since the picture structure is translation invariant, its internal structure does not change with the position, so CNN can realize parameter sharing. The processing object of the RNN model is sequence information, and the RNN model can capture the features of the sequence by means of various gate operations. However, the data existing in reality is not all structured and dimensional structures. When the above-mentioned models process irregular data structures, they cannot play a good role.

[0003] The storage, query and data structure of the graph database are very different from those of the structured data. The graph data structure directly stores the dependency relationship between nodes, while the relational database and other types of non-relational databases represent the relationship between data in a non-direct way. The graph database stores the association between data as part of the data, and labels, directions and attributes can be added on the association. The query for the relationship in other databases must be specifically operated at runtime, which is the reason why the graph database has a huge performance advantage in relation query compared with other types of databases. By explaining the structured data, important information hidden in the structured data can be analyzed, thereby promoting the development of artificial intelligence. However, the existing explanation method for the graph model of the graph data is limited to chemical molecular structure with only 10-20 nodes, and cannot explain the large data set in the Internet field.

[0004] Therefore, it is necessary to design an improved graph neural network explanation method based on a generative adversarial network to solve the above problems. SUMMARY

[0005] The present application relates to the technical field of Internet security, and particularly relates to a graph neural network explanation method based on a generative adversarial network.

[0006] To achieve the above-mentioned purposes, the application provides a graph neural network explanation method based on a generative adversarial network.

[0007] Preferably, the graph neural network explanation method comprises the following steps:

[0008] S1, data preprocessing: the GCN model is used to extract node features of graph data, all node subgraphs are spliced for parallel training of all nodes, the negative sampling model is used to process and screen the spliced node subgraphs and node representation vectors, and edge representation vectors are obtained, and the collection of all edge representation vectors is called Edge_embeddings;

[0009] S2, learning and explanation of edge representation vectors by using an MLP model: the MLP model is used to calculate the edge representation vectors of Edge_embeddings obtained in step S1, and the corresponding graph data subgraphs are saved or deleted according to the weight values output by the MLP model, and all saved graph data subgraphs constitute Sampled graph;

[0010] S3, correctness evaluation of the explanation method: the Sampled graph obtained in step S2 and the original graph data are input into the GCN model respectively to obtain prediction results Ys and actual results Y0, and the correctness of the potential subgraph can be evaluated according to the prediction results Ys and the actual results Y0.

[0011] Preferably, in step S1, when the negative sampling model is used for data processing, the edges existing in the original graph data are recorded as "1 edges", and the edges not existing are recorded as "0 edges"; preferably, the method for generating "0 edges" by the negative sampling model is that, taking the number M of "1 edges" as a reference, 2XM edges in "0 edges" are randomly sampled as negative samples.

[0012] Preferably, in step S1, the generation method of the node subgraph is that, taking a node as the center, a node group within a range of 3-5 degrees is diffused outward to form the node subgraph; preferably, the maximum number of nodes is 5000.

[0013] Preferably, in step S1, after saving or deleting the subgraph of the corresponding graph data, re-parameterization processing is further required to avoid omissions in the learning process and to process binary variables that are not in the range of 0-1 into continuous variables in the range of 0-1.

[0014] Preferably, in step S2, the determination method for saving or deleting the graph data is that when the weight of the edge of the graph data is greater than 0.5, the edge corresponding to the graph data is saved, otherwise, the edge corresponding to the graph data is deleted; preferably, the value range of the weight is 0-1.

[0015] Preferably, in step S3, the correctness evaluation of the potential subgraph is performed by the following method: when the difference between the prediction result Ys and the actual result Y0 is less than 0.01, the potential subgraph is correct.

[0016] Preferably, if the difference between the prediction result Ys and the actual result Y0 is greater than 0.01, return to step S1 to re-perform the training process.

[0017] Preferably, when the difference between the prediction result Ys and the actual result Y0 is less than 0.01 and the difference is fixed, the parameters of the MLP model are determined.

[0018] Preferably, in step S2, the MLP model can also summarize the weight of the corresponding edge according to the embedding of the neighbor node, and then make an explanation for most of the abnormal data.

[0019] The beneficial effects of the present application are:

[0020] 1、The graph neural network explanation method based on the generative adversarial network provided by the present application first obtains the node representation of the graph data through the GCN model, then combines the negative sampling model and the graph structure to process the original graph data to obtain the representation vector of the edge, then trains the representation vector of the edge by means of the MLP model and the GCN model, and finally finds the potential subgraph that has explanatory power for the prediction of the GCN, and at the same time, the above process can also determine the parameters to be determined in the MLP model, so that the trained method can be directly used to explain the graph data subsequently.

[0021] 2、The graph neural network explanation method based on the generative adversarial network provided by the application, by introducing a negative sampling model to sample the 0 edge to be explained, finally generates a negative sample, effectively reduces the complexity of subsequent training of the node subgraph by using the MLP model and the GCN model, ensures the balance of the sample, and provides a basic condition for batch explanation of graph data; in addition, the introduction of the negative sample model can also effectively reduce the size of Edge_embeddings, compared with the memory utilization rate of the device when the same graph data is explained by using the existing graph data explanation method, the memory utilization rate can be improved by more than 4 times; by using the MLP model to batch train multiple node subgraphs, only the comparison process of Ys and Y0 needs to be updated once in the training process, therefore, the training efficiency of the node subgraph can be greatly improved, and in the application, the graph data of 1000 nodes is trained only for 14.7s, compared with 35min of the existing training method, the training efficiency is improved by about 140 times; furthermore, the explanation method provided by the application can explain the graph data of 5000 nodes at the same time. Through the above-mentioned mode, the graph neural network explanation method based on the generative adversarial network which can batch explain graph data is provided. BRIEF DESCRIPTION OF DRAWINGS

[0022] Fig. 1 The specific steps of the graph neural network explanation method based on the generative adversarial network of the application;

[0023] Fig. 2 The flowchart of the graph neural network explanation method based on the generative adversarial network of the application;

[0024] Fig. 3 The comparison of the graph data before and after explanation in embodiment 1 of the application. DETAILED DESCRIPTION

[0025] In order to make the purpose, technical scheme and advantages of the application more clear, the application is described in detail below with reference to the drawings and specific embodiments.

[0026] Here, it also needs to be explained that, in order to avoid the unnecessary details from blurring the application, only the structures and / or processing steps closely related to the scheme of the application are shown in the drawings, and other details not closely related to the application are omitted.

[0027] In addition, it also needs to be explained that the term "comprises", "includes" or any other variant thereof is intended to cover non-exclusive inclusion, so that the process, method, article or device including a series of elements not only includes those elements, but also includes other elements not explicitly listed or inherent to such process, method, article or device.

[0028] Please refer to Figs. 1-2As shown, the graph neural network explanation method based on the generative adversarial network provided by the application first uses a GCN model (graph convolutional neural network) to process the graph data to be explained to obtain the node representation of the graph data, then combines a negative sampling model and the graph structure of the node subgraph to perform data processing on the spliced subgraph data, obtains the representation vector of the edge, then trains the representation vector of the edge by means of an MLP model (multi-layer neural network) and the GCN model, and finally finds the latent subgraph that has an explanation for the prediction result of the GCN.

[0029] The graph neural network explanation method based on the generative adversarial network provided by the application specifically includes the following steps:

[0030] S1, original graph data preprocessing: first, the node representation is learned by using the GCN model, in this process, the node features of the graph data can be extracted, that is, the node features Z of the graph model to be explained can be obtained, then the spliced subgraph data is processed by combining a negative sampling model and the graph structure of the node subgraph, the edges existing in G0 are recorded as "1 edges", and the edges not existing are recorded as "0 edges", the node subgraph is obtained in the processing process, the above process can realize the training of all node subgraphs, and the set of representation vectors of the edges of all node subgraphs obtained by training is called Edge_embeddings; wherein the negative sampling model is: taking the number M of "1 edges" as a reference, 2XM edges in the "0 edges" are randomly sampled as negative samples, which not only ensures the balance of the training samples, but also improves the memory utilization. The method for generating the node subgraph is: taking the node as the center, the nodes within the range of 3-5 degrees are diffused to form a graph, which is the subgraph of the current node;

[0031] S2, learning and explanation of the representation vector of the edge by using the MLP model: the representation vector of Edge_embeddings obtained in step S1 is calculated by using the MLP model, and the weight value is output, the range of the weight is 0-1, in this process, the edges with a weight greater than 0.5 are saved, the edges with a weight greater than 0.5 are represented by Gs, if the output weight value is less than 0.5, the edge is deleted, this process can realize the decomposition of the original graph G0, and the latent target subgraph Gs is obtained, in order to train all nodes synchronously, all subgraphs Gs to be explained are collected together to form a Sampled graph.

[0032] In particular, the above process also needs to be re-parameterized, on the one hand, in order to prevent the 1 edges from being directly output in the training process, on the other hand, due to the discreteness of the graph data, the weight of the edge is only 0 and 1, therefore, the weight of the edge needs to be processed from a binary variable to a continuous variable in the range of 0-1 through the re-parameterization process; in the training process, the weight of the edge is divided more and more, which achieves the purpose of finding the latent subgraph Gs;

[0033] S3, explanation method correctness evaluation: input the Sampled graph obtained in step S2 into the GCN model to obtain a prediction result Ys, compare Ys with Y0 obtained by inputting G0 into the GCN model, and use the comparison as a supervision signal to update the MLP model, and repeat the process to continuously train, when the difference between the two is less than 0.01 and the difference is fixed, it is considered that the Gs at this time is a potential subgraph, and the Gs at this time is an explanatory subgraph that makes an important contribution to the prediction result of the GCN model; if the difference between Ys and Y0 is greater than 0.01, steps S1 to S3 need to be implemented again until the difference between Ys and Y0 is less than 0.01, and at the same time, the difference between Ys and Y0 is fixed, which can uniquely determine the parameters of the MLP model, that is, the training of the entire explanation method can be completed. In this way, the parameters of the MLP model are fixed, and the above method can be used for the explanation of the graph data.

[0034] The application will be further described below in combination with specific embodiments:

[0035] Embodiment 1

[0036] The graph neural network explanation method based on the generative adversarial network is used to explain the graph neural network, and the specific process includes the following steps:

[0037] S1, original graph data preprocessing: first, the node representation is learned by using the GCN model, in this process, the node features of the graph data can be extracted, that is, the node features Z of the graph model to be explained can be obtained, then the spliced subgraph data is processed in combination with the negative sampling model and the graph structure of the node subgraph, the edges existing in G0 are recorded as "1 edges", and the edges not existing are recorded as "0 edges", the processing process obtains the node subgraph, the above process can realize the training of all node subgraphs, and the set of edges of all node subgraphs obtained by training is called Edge_embeddings; wherein the negative sampling model is: taking the number M of 1 edges as a reference, 2XM edges in 0 edges are randomly sampled as negative samples. In this embodiment, the number of nodes trained is 5000, and the generation method of the node subgraph is: taking the node as the center, the nodes within the range of 3 degrees are diffused to form a graph, which is the subgraph of the current node.

[0038] S2, learning and explanation of the edge representation vector by using the MLP model: the weight of the edge of the node subgraph obtained in step S1 can be calculated and output by using the MLP model, and the weight range is 0-1, in this process, the edge with a weight greater than 0.5 is saved, and the edge with a weight greater than 0.5 is represented by Gs, Gs is the potential target subgraph that the model hopes to find, and the set of all Gs constitutes Sampledgraph.

[0039] In particular, the above process also needs to be re-parameterized, on the one hand, in order to prevent the model from not learning to directly output all 1 edges; on the other hand, due to the discreteness of the graph data, the weight of the edge is only 0 and 1, therefore, the weight of the edge needs to be processed from a binary variable to a continuous variable in the range of 0~1 through the re-parameterization process. In the training process, the weight of the edge is more and more open, which achieves the purpose of finding the important subgraph Gs.

[0040] S3, correctness evaluation of the explanation method: input the Sampledgraph obtained in step S2 into the GCN model to obtain the prediction result Ys, compare Ys with Y0 obtained by inputting G0 into the GCN model, and use the comparison as a supervision signal to update the MLP model, repeat the process to continuously train, when the difference between the two is less than 0.01 and the difference is fixed, it is considered that the corresponding Gs is the latent subgraph at this time, at the same time, the parameters of the MLP model can be uniquely determined at this time, that is, the training of the entire model can be completed; during the above training process, the MLP model can also summarize the weight of the corresponding edge according to the embedding of the neighbor node, and can explain most of the abnormal data, based on the above reasons, further ensure that the prediction results of Gs and G0 remain consistent, so that the finally found Gs can more approximate the effect of the original graph; the comparison of the graph data before and after processing in the embodiment is shown in FIG. Fig. 3 The whole graph is G0, and the graph connected by important edges is Gs. The time required for training 5000 nodes in the embodiment is 243.04s.

[0041] In addition, the training of 1000 node graph data is also carried out, and the results show that the training of 1000 nodes only needs 14.7s, compared with 35min of the prior art, the training efficiency is improved by about 140 times; in addition, the use of the negative sampling model can optimize the Edge_embeddings to 7.4G, and the above process cannot run on a 32G device before optimization, and the optimization process effectively improves the memory utilization.

[0042] In summary, the graph neural network explanation method based on the generative adversarial network proposed in the embodiment, by first using the negative sampling model to process and screen the node subgraph to be explained, obtaining the representation vector Edge_embeddings of the edge to be trained, then training the representation vector of the edge by means of the MLP model and the GCN model, and finally finding the latent subgraph which has explanation for the prediction of the GCN, at the same time, the above process can also determine the parameters to be determined in the MLP model, therefore, the trained method can be directly used for explanation of the graph data in the subsequent process.

[0043] The above examples are only used to illustrate the technical solutions of the present application but not limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or equivalent replaced without departing from the spirit and scope of the technical solutions of the present application.

Claims

1. A graph neural network interpretation method based on generative adversarial networks, characterized by: The graph neural network interpretation method first obtains node features of the graph data through the GCN model to be interpreted. Then, it combines the negative sampling model and the graph structure of the node subgraph to process and screen the subgraph data to obtain edge representation vectors. The MLP model is then used to interpret and classify the edge representation vectors, and ultimately finds potential subgraphs that can explain the prediction results of the GCN model. The graph neural network interpretation method comprises the following steps: S1. Preprocessing of original graph data: First, use the GCN model to extract node features of the graph data. In order to train all node subgraphs in parallel, all node subgraphs are spliced ​​together. Then, the negative sampling model and the graph structure of the node subgraph are used to process and screen the spliced ​​subgraph data to obtain the edge representation vectors. The set of all edge representation vectors is called edge_embeddings. S2. Use the MLP model to interpret and classify the edge representation vectors: Use the MLP model to calculate the edge representation vectors of edge_embeddings obtained in step S1, and save or delete the corresponding subgraphs of the graph data based on the weight values ​​output by the MLP model. All saved subgraphs of the graph data constitute a set called Sampled graph. S3. Correctness evaluation of the interpretation method: The set Sampled graph and the original graph data obtained in step S2 are respectively input into the GCN model to obtain the predicted result Ys and the actual result Y0. The correctness of the potential subgraph can be evaluated based on the predicted result Ys and the actual result Y0: Ys is compared with Y0 obtained by inputting the original graph G0 into the GCN model, and the MLP model is updated with the comparison result of the predicted result Ys and the actual result Y0. This process is repeated for continuous training. When the difference between the two is less than 0.01 and the difference remains unchanged, the corresponding potential target subgraph Gs is considered to be a potential subgraph, and the parameters of the MLP model are determined.

2. The graph neural network interpretation method based on generative adversarial network according to claim 1 is characterized in that In step S1, when the negative sampling model is used to process data, the edges existing in the original graph data are recorded as "1 edges" and the edges not existing in the original graph data are recorded as "0 edges"; The negative sampling model generates "0 edges" by taking the number M of "1 edges" as a reference and randomly sampling 2×M edges from "0 edges" as negative samples.

3. The graph neural network interpretation method based on generative adversarial network according to claim 1 is characterized in that In step S1, the node subgraph is generated by taking the node as the center and expanding outwards from the node within a range of 3 to 5 degrees to obtain the node subgraph; The maximum number of nodes is 5000.

4. The graph neural network interpretation method based on generative adversarial network according to claim 1 is characterized in that In step S2, after the subgraph of the corresponding graph data is saved or deleted, it needs to undergo reparameterization to avoid omissions in the learning process and to process binary variables that are not in the range of 0~1 into continuous variables in the range of 0~1.

5. The graph neural network interpretation method based on generative adversarial network according to claim 1 is characterized in that In step S2, the method for determining whether to save or delete the subgraph of the corresponding graph data is: when the weight of the edge of the graph data is greater than 0.5, the edge corresponding to the graph data is saved; otherwise, the edge corresponding to the graph data is deleted.

6. The graph neural network interpretation method based on generative adversarial network according to claim 1 is characterized in that In step S3, the correctness evaluation of the potential subgraph is performed in the following manner: when the difference between the predicted result Ys and the actual result Y0 is less than 0.01, the potential subgraph interpretation is correct.

7. The graph neural network interpretation method based on generative adversarial network according to claim 6 is characterized in that If the difference between the predicted result Ys and the actual result Y0 is greater than 0.01, return to step S1 and repeat the training process.

Images