Output control program, output control method, and information processing device
The output control program improves the explanation of graph machine learning model inference results by generating and comparing explanations across similar graph data, addressing the 'black box' problem and enhancing understanding of inference processes.
Patent Information
- Application Number
- PCT/JP2023/018493
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-17
- Publication Date
- 2024-11-21
AI Technical Summary
Existing graph machine learning models face difficulties in explaining inference results, particularly in understanding why specific graph elements contribute to the inference, leading to a 'black box' problem, which XAI technologies partially address but do not fully resolve.
An output control program that generates first and second inference results using a trained graph machine learning model, selects similar graph data, and outputs explanations for both, leveraging existing XAI techniques to provide insights into the inference processes by associating nodes and edges across target and similar graph data.
Enhances understanding of inference reasons by comparing explanations between target and similar graph data, facilitating better comprehension of graph machine learning model decisions.
Smart Images

Figure JP2023018493_21112024_PF_FP_ABST
Abstract
Description
Output control program, output control method, and information processing device
[0001] The present invention relates to an output control program, an output control method, and an information processing device.
[0002] Conventionally, in machine learning for graph data, a technique has been known in which graph data is converted into vector data and then machine learning techniques such as decision trees or random forests are used to process the vector data. Furthermore, currently, convolutional graph neural networks (GNNs) are widely used as graph machine learning models that handle graph data as is (Z. Wu, S. Pan, F. Chen, G. Long, C. Zhang, and S.Y. Philip, “A comprehensive survey on graph neural networks,” IEEE Trans. NeuralNetw. Learn. Syst., 2020. https: / / doi.org / 10.1109 / TNNLS.2020.2978386).
[0003] AI (Artificial Intelligence) has a complex internal structure, and it is difficult to explain its inference results. This problem is sometimes called the black box problem. Therefore, XAI (eXplainable AI) is being developed as a technology that makes it possible to explain to humans the process that leads to such AI inference results.
[0004] XAI for graph machine learning has also been developed (Non-Patent Documents 1 and 2, etc.). XAI for graph machine learning presents, as an explanation, graph elements that contributed to the inference result from the input target graph data. The presented graph elements are nodes, edges, or subgraphs.
[0005] Alexandre Duval and Fragkiskos D Malliaros, “GraphSVX: Shapley value explanations for graph neural networks,” arXiv preprint arXiv:2104.10482, 2021. Ying, R., Bourgeois, D., You, J., Zitnik, M., & Leskovec, J. GNNExplainer: Generating Explanations for Graph Neural Networks.
[0006] XAI for graph machine learning only shows the graph elements that contributed to an inference, but it can be difficult to understand the reasons for the inference.
[0007] In one aspect, an object of the present invention is to facilitate understanding of the reasoning behind the inference results of a graph machine learning model.
[0008] In one aspect, the output control program may cause a computer to execute the following processes: the process obtains a first inference result inferred using a trained model based on input target graph data, and a first graph feature that is intermediate data generated by the trained model corresponding to the target graph data; the process selects, from a plurality of graph data different from the target graph data, one or more similar graph data, in which a second inference result that matches or is similar to the first inference result is inferred using the trained model, and a second graph feature that is similar to the first graph feature is generated by the trained model; and the process outputs first information regarding an explanation of the first inference result and second information regarding an explanation of the second inference result.
[0009] In one aspect, the present invention can facilitate understanding of the reasoning behind the inference results of a graph machine learning model.
[0010] 1 is a diagram showing an example of graph data. FIG. 2 is a diagram showing an example of a compound structure represented by graph data. FIG. 3 is a diagram showing an example of an explanation of an inference result in graph machine learning. FIG. 4 is an output example of information regarding the explanation of an inference result in a comparative example. FIG. 5 is a first example of output of an explanation of an inference result in an embodiment. FIG. 6 is a second example of output of an explanation of an inference result in an embodiment. FIG. 7 is a block diagram showing an example of a hardware (HW) configuration of a computer that realizes an information processing device according to an embodiment. FIG. 8 is a block diagram showing an example of a functional configuration in a training phase of an information processing device according to an embodiment. FIG. 9 is a block diagram showing an example of a functional configuration in an inference phase and advance preparation of an information processing device according to an embodiment. FIG. 10 is a diagram showing an example of a database that stores a plurality of candidate graph data. FIG. 11 is a diagram showing an example of a graph machine learning model. FIG. 12 is a diagram showing an example of target graph data input to a graph machine learning model. FIG. 13 is a diagram showing an example of node features. FIG. 14 is a diagram showing an example of graph features. FIG. 15 is a flowchart showing an example of operation in a training phase by an information processing device according to an embodiment. FIG. 16 is a flowchart showing a first example of processing in the inference phase and advance preparation of an information processing device according to an embodiment.
[0011] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the embodiment described below is merely an example, and is not intended to exclude various modifications or application of techniques not explicitly stated below. For example, this embodiment can be implemented with various modifications within the scope of its spirit. In the drawings used in the following description, parts with the same reference numerals represent the same or similar parts unless otherwise specified.
[0012] [A] Example of Output for Explaining Inference Results in Graph Machine Learning Fig. 1 is a diagram showing an example of graph data 1. The graph data 1 is a data format capable of expressing connections between people and things. For example, the graph data 1 displays SNS (Social Networking Service), chemical compounds, communication history, transaction history, etc.
[0013] Graph data 1 includes nodes 2-1 to 2-5 (collectively referred to as nodes 2) and edges 3-1 to 3-5 (collectively referred to as edges 3). The number of nodes 2 and edges 3 included in the graph data is not limited to the case shown in FIG. 1. Node 2 indicates the subject of a connection relationship. Edge 3 indicates a connection relationship. Node 2 may have a node label as attribute information. Edge 3 may have an edge label as attribute information.
[0014] FIG. 2 is a diagram showing an example of a compound structure represented by graph data 1 (denoted as 1a in the diagram). In FIG. 2, nodes 2 represent atoms. Edges 3 represent bonds between atoms. Each node 2 has an atomic type, such as carbon, oxygen, or hydrogen, as a node label. Each edge 3 has a bond type, such as a single bond or a double bond, as an edge label.
[0015] FIG. 3 is a diagram showing an example of an explanation of inference results in graph machine learning. Graph data has complex relationships and interdependencies between objects. Therefore, conventionally, graph data is converted into vector data, and machine learning is performed on the vector data using techniques such as decision trees or random forests. Currently, graph machine learning techniques such as convolutional GNNs are used, which extend deep learning approaches to graph data and allow machine learning of graph data as is.
[0016] According to the graph machine learning technology, an inference result can be obtained by inputting target graph data 11 into a trained graph machine learning model. The inference result may be graph classification, node classification, link prediction, etc.
[0017] In graph classification, a graph machine learning model classifies the presence or absence of a characteristic from the target graph data 11. In one example, graph classification predicts the presence or absence of toxicity for an unknown compound.
[0018] In node classification, when the node label of some nodes 2-x in the target graph data 11 is unknown, the graph machine learning model predicts the node label of node 2-x from information on other nodes 2 and edges 3.
[0019] In link prediction (i.e., edge classification), when some edges 3-x in the target graph data 11 are unknown, the graph machine learning model predicts the relationship indicated by edge 3-x from information on other nodes 2 and edges 3.
[0020] Note that, for details of the graph machine learning model, existing techniques such as the convolutional GNN described in the aforementioned paper by Z. Wu et al. can be used, and therefore detailed explanations will be omitted.
[0021] XAI for graph machine learning is a technology that explains the process leading to inference results in graph machine learning in a way that is understandable to humans.It presents the graph elements that contributed to the inference results as an explanation.
[0022] In one example, the description of the process leading to the inference result may include a graphical description 40a characterized by, for example, changing the color of nodes 41a-1 to 41a-3 (sometimes collectively referred to as nodes 41a) that contributed to the inference result among the nodes 2 of the target graph data 11. The description of the process leading to the inference result may include a graphical description 40b characterized by, for example, changing the color of edges 41b-1 to 41b-3 (sometimes collectively referred to as edges 41b) that contributed to the inference result among the edges 3 of the target graph data 11. The description of the process leading to the inference result may be subgraphs 42a, 42b of the target graph data 11. The subgraphs 42a, 42b may be one node 2 or edge 3, may include multiple nodes 2 or multiple edges 3, or may include multiple nodes 2 and multiple edges 3.
[0023] As XAI for graph machine learning, GraphSVX listed in Non-Patent Document 1 and GNNExplainer listed in Non-Patent Document 2 can be used.
[0024] It may be difficult to understand the reason for the inference from the explanation 40a or explanation 40b presented as in Fig. 3. According to the explanation 40a or explanation 40b, it is possible to know the node 41a or edge that contributed to the inference, but it may be difficult to understand why the node or edge contributes to the inference. In other words, it is not easy for a human to understand the inference rules adopted by the graph machine learning model.
[0025] Domain knowledge about the data, i.e., knowledge of the specialized field to which the data belongs, can help understand the inference results, but AI does not necessarily make inferences that are in line with domain knowledge.
[0026] FIG. 4 is an example of output information related to the explanation of the inference result in a comparative example. In the comparative example shown in FIG. 4, in addition to an explanation 40 of the inference result for the target graph data, explanations 50a to 50d (sometimes collectively referred to as explanation 50) of the inference result for similar graph data whose inference results match or are similar to those of the target graph data are output. The explanations 50a to 50c include nodes 51a to 51c that contributed to the inference result. The explanation 50d includes multiple nodes 51d-1 and 51d-2 that contributed to the inference result. In the example of FIG. 4, four graph data are displayed as similar graph data, but this is not limited to this case. Furthermore, edges or subgraphs may be displayed as explanations 50a to 50d instead of nodes 51a to 51d-2 (sometimes collectively referred to as node 51).
[0027] To obtain the output example of FIG. 4 , each of a plurality of candidate graph data that are candidates for similar graph data is input into a trained graph machine learning model to obtain an inference result. The plurality of candidate graph data are different from the target graph data 11. Furthermore, the target graph data 11 is input into the trained graph machine learning model to obtain an inference result. Then, a similarity between the inference result for the target graph data 11 and the inference results for each of the candidate graph data is calculated. In one example, the similarity between the inference results may be the reciprocal of the absolute value of the difference between the inference results.
[0028] Candidate graph data that shows an inference result that matches or has a predetermined degree of similarity or higher with the inference result for the target graph data is selected as similar graph data. Then, an explanation 40 including nodes 41 that contributed to the inference is output using an existing XAI for graph machine learning for the target graph data 11. Similarly, an explanation 50 including nodes 51 that contributed to the inference is output using an existing XAI for graph machine learning for the selected similar graph data.
[0029] As in the comparative example shown in FIG. 4, explanations of other data that lead to the same inference result can also be helpful, but even if the inference result is the same, the reason for the inference is not necessarily the same.
[0030] 5 is a first example of an output of an explanation of an inference result in one embodiment. Similar graph data that matches or is similar to the inference result based on the target graph data 11 and has similar graph features is selected from the candidate graph data. The graph features are intermediate data that are generated by a trained model corresponding to the target graph data 11. The graph features are data that allow for the definition of similarity, such as Euclidean distance or cosine similarity, between graph data. Furthermore, the graph features correspond to the entire graph data. Graph features will be described later.
[0031] Then, an explanation 40 including nodes 41 that contributed to the inference is output using an existing XAI for graph machine learning for the target graph data 11. Similarly, an explanation 60 including nodes 61 that contributed to the inference is output using an existing XAI for graph machine learning for the selected similar graph data. The explanation 60 in Figure 5 corresponds to the explanation 50b in Figure 4.
[0032] If the inference results are not only identical or similar but also have similar graph features, the likelihood that the inference reasons are similar can be increased compared to when the inference results are simply identical or similar. Therefore, users such as analysts can easily understand the inference results by focusing on the commonalities between explanation 40 and explanation 60.
[0033] In one example, the candidate graph data from which similar graph data is obtained may be a graph dataset used to train a graph machine learning model. However, the candidate graph data is not limited to this, and may be multiple graph data that can be input to the target graph machine learning model.
[0034] 5 shows a case where one node 41 that contributed to the inference for the target graph data 11 is output, but multiple nodes 41 (i.e., elements) that contributed to the inference may also be output. For similar graph data, multiple nodes 61 (i.e., elements) that contributed to the inference may also be output. While FIG. 5 shows a case where an explanation is given for one similar graph data, the present invention is not limited to this case.
[0035] 6 shows a second example of an output of an explanation of an inference result in one embodiment. In addition to the first example shown in FIG. 5, association information 70-1 and 70-2 (sometimes collectively referred to as association information 70) are output.
[0036] The association information 70 associates at least one node included in the target graph data 11 with at least one node in the similar graph data. The association between nodes is made according to the similarity between a first node feature and a second node feature. Note that the association information 70 may also associate at least one edge included in the target graph data 11 with at least one edge in the similar graph data. The association information 70 is an example of information that associates at least one element included in the target graph data 11 with at least one element in the similar graph data.
[0037] The first node feature is an example of a first element feature, which is intermediate data generated by a trained model corresponding to each element included in the target graph data 11. The second node feature is an example of a second element feature, which is intermediate data generated by a trained model corresponding to each element included in the similar graph data. When there are multiple similar graph data, a second node feature is obtained corresponding to each node for each similar graph data. Node features such as the first node feature and the second node feature are data that allow similarity, such as Euclidean distance or cosine similarity, to be defined between nodes. Furthermore, node features correspond to nodes. Node features will be described later.
[0038] The association information 70 may associate some or all of the nodes included in the target graph data 11 with some or all of the nodes included in the similar graph data. In particular, the association information 70-1 and 70-2 may be provided by selecting one or more nodes 41a-1, 41a-2 that contributed to the inference in the target graph data and one or more nodes 61a-1, 61a-2 that contributed to the inference in the similar graph data.
[0039] [B] Hardware Configuration Example The functions of the information processing device 100 according to an embodiment may be implemented by one computer or two or more computers. Furthermore, at least some of the functions of the information processing device 100 may be implemented using hardware (HW) resources and network (NW) resources provided by a cloud environment.
[0040] 7 is a block diagram showing an example of the hardware (HW) configuration of a computer that realizes the functions of the information processing device 100 according to one embodiment. When multiple computers are used as HW resources that realize the functions of the information processing device 100, each computer may have the HW configuration shown in FIG.
[0041] As shown in FIG. 7, the information processing device 100 may include, as its HW configuration, a processor 100a, a memory 100b, a storage unit 100c, an IF (Interface) unit 100d, an IO (Input / Output) unit 100e, and a reading unit 100f, for example.
[0042] The processor 100a is an example of an arithmetic processing device that performs various controls and calculations. The processor 100a may be connected to each block in the information processing device 100 via a system bus 100i so that they can communicate with each other. The processor 100a may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.
[0043] Examples of the processor 100a include integrated circuits (ICs) such as a CPU, MPU, GPU, APU, DSP, ASIC, and FPGA. Note that the processor 100a may be a combination of two or more of these integrated circuits. CPU is an abbreviation for Central Processing Unit, MPU is an abbreviation for Micro Processing Unit, GPU is an abbreviation for Graphics Processing Unit, and APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.
[0044] The memory 100b is an example of HW that stores various types of data, programs, and other information. The memory 100b may be, for example, a volatile memory such as a dynamic random access memory (DRAM) or a non-volatile memory such as a persistent memory (PM).
[0045] The storage unit 100c is an example of HW that stores various types of data, programs, and other information. Examples of the storage unit 100c include various storage devices such as a magnetic disk device such as an HDD, a semiconductor drive device such as an SSD, and a nonvolatile memory. Examples of nonvolatile memory include a flash memory, a storage class memory (SCM), and a read-only memory (ROM).
[0046] The storage unit 100c may store a program 100g (output control program) that realizes all or part of the various functions of the information processing device 100. The program 100g may include an OS (Operating System) and the like in addition to the output control program.
[0047] For example, the processor 100a can realize the function of a control unit (control unit 110 in Figures 8 and 9 described below) of the information processing device 100 by expanding and executing a program 100g stored in the storage unit 100c in the memory 100b.
[0048] The information processing device 100 may function as a computer and execute an output control program to perform each process of performance information collection.
[0049] The IF unit 100d is an example of a communication IF that controls connections and communications between the information processing device 100 and various networks, including networks between other devices.
[0050] For example, the IF unit 100d may include an adapter that complies with a LAN (Local Area Network) such as Ethernet (registered trademark) or optical communication such as FC (Fibre Channel). The adapter may be compatible with either or both of wireless and wired communication methods.
[0051] The program 100g may be downloaded from a network to the information processing device 100 via the communication IF and stored in the storage unit 100c.
[0052] The IO unit 100e may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, and a touch panel. Examples of output devices include a monitor, a projector, and a printer. The IO unit 100e may also include a touch panel that combines an input device and an output device.
[0053] The reading unit 100f is an example of a reader that reads data and program information recorded on the recording medium 100h. The reading unit 100f may include a connection terminal or device to which the recording medium 100h can be connected or inserted. Examples of the reading unit 100f include an adapter compliant with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 100h may store a program 100g, and the reading unit 100f may read the program 100g from the recording medium 100h and store it in the memory unit 100c.
[0054] Examples of the recording medium 100h include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, compact discs (CDs), digital versatile discs (DVDs), Blu-ray discs, and holographic versatile discs (HVDs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.
[0055] The above-described HW configuration of the information processing device 100 is an example. Therefore, the HW in the information processing device 100 may be increased or decreased (for example, adding or deleting any block), divided, integrated in any combination, or the HW may be added or deleted as needed.
[0056] [C] Functional Configuration Example [C-1] Training Phase Fig. 8 is a block diagram showing an example of the functional configuration of the information processing device 100 according to an embodiment in the training phase. The information processing device 100 has a control unit 110. In the information processing device 100, training graph data 4 is input to the graph machine learning model 20, and the graph machine learning model 20 is trained.
[0057] The control unit 110 has a candidate graph data acquisition unit 111. The candidate graph data acquisition unit 111 acquires candidate graph data 5. The candidate graph data 5 may be a part or all of the training graph data 4 used in training. However, the candidate graph data 5 is not necessarily limited to the training graph data 4.
[0058] The graph machine learning model 20 may be a GNN. In one example, the graph machine learning model 20 may be a convolutional GNN. The graph machine learning model 20 may be a learning model for node classification or a learning model for graph classification. In one example, the graph machine learning model 20 can be realized by a convolutional GNN described in the literature by Z. Wu et al. Therefore, a detailed description thereof will be omitted.
[0059] 9 is a block diagram showing an example of a functional configuration of the information processing device 100 according to an embodiment in the training phase. The control unit 110 includes a first acquisition unit 112, a second acquisition unit 113, a selection unit 114, a first explanation information calculation unit 116, a first image output unit 117, a second explanation information calculation unit 118, a second image output unit 119, an additional information calculation unit 120, and an additional image output unit 121.
[0060] The first acquisition unit 112 acquires a first inference result 6, a first graph feature 7, and a first node feature 8 that are inferred using the trained graph machine learning model 20 based on the target graph data 11 input to the trained graph machine learning model 20.
[0061] The first inference result 6 may be a graph classification, a node classification, etc. The first graph feature 7 is intermediate data generated by the trained graph machine learning model 20 in correspondence with the target graph data 11. The first node feature 8 is intermediate data generated by the trained graph machine learning model 20 in correspondence with each node 2 included in the target graph data 11.
[0062] The second acquisition unit 113 acquires inference results 81, graph features 82, and node features 83 inferred using the trained graph machine learning model 20 based on multiple candidate graph data 5 input to the trained graph machine learning model 20.
[0063] 10 is a diagram showing an example of a database 130 that stores a plurality of candidate graph data. The database 130 may include a plurality of databases 131 and 132. The second acquisition unit 113 stores an inference result 81 and graph features 82 in the database 131 for each graph ID that identifies the candidate graph data 5. The second acquisition unit 113 stores a node feature 83 in the database 132 for each graph ID of the candidate graph data 5 and each node ID of the node 2.
[0064] 9 , the selection unit 114 receives the first inference result 6, the first graph feature 7, and the first node feature 8 from the first acquisition unit 112. The selection unit 114 selects one or more similar graph data 84 from among the multiple candidate graph data 5 stored in the database 130.
[0065] Similar graph data 84 is graph data that, when input into graph machine learning model 20, results in a second inference result 85 that is identical to or similar to first inference result 6 as the inference result, and also results in a second graph feature 86 that is similar to first graph feature 7 as intermediate generated data.
[0066] A match between the first inference result 6 and the second inference result 85 may be a match between the classification labels of the graph classification or the classification labels of the node classification. The similarity between the first inference result 6 and the second inference result may be the reciprocal of the absolute value of the difference between the first inference result 6 and the second inference result. The similarity between the first graph feature 7 and the second graph feature 86 and the similarity between the first node feature 8 and the second node feature 87 may be Euclidean distance or cosine similarity.
[0067] The selection unit 114 may include a similarity calculation unit 115. The similarity calculation unit 115 may calculate the respective similarities. The selection unit 114 may select, as the similar graph data 84, a plurality of graph data in which the first inference result 6 and the second inference result 85 match and in which the similarity with the first graph feature 7 is equal to or greater than a predetermined value. On the other hand, the selection unit 114 may select, as the similar graph data 84, a predetermined number of graph data, for example, one graph data in which the first inference result 6 and the second inference result 85 match and in descending order of similarity with the first graph feature 7.
[0068] The first explanation information calculation unit 116 calculates first information 88 relating to the explanation of the first inference result 6. The second explanation information calculation unit 118 calculates second information 89 relating to the explanation of the second inference result 85.
[0069] The first information 88 may include information on at least one node or edge that contributed to the first inference result 6 and that was selected from the nodes 2 or edges 3 included in the target graph data 11. The second information 89 may include information on at least one node or edge that contributed to the second inference result 85 and that was selected from the nodes 2 or edges 3 included in the similar graph data 84.
[0070] The first explanation information calculation unit 116 and the second explanation information calculation unit 118 use an existing XAI for graph machine learning (i.e., graph XAI) to calculate the first information 88 and the second information 89. For example, GraphSVX shown in Non-Patent Document 1 may be used as the graph XAI.
[0071] GraphSVX, described in Non-Patent Document 1, calculates the nodes that contributed to the inference result. GraphSVX extends and applies the Shapley value in game theory to graph data. A surrogate model is constructed using a perturbed dataset, and the fair contribution of the node to the inference result is calculated as the Shapley value. GraphSVX may ultimately provide the Shapley value for each node 2 as an explanation.
[0072] XAI for graph machine learning such as GraphSVX can use existing technology, so detailed explanation will be omitted.
[0073] The first information 88 may include first image information that identifies at least one node that contributed to the first inference result 6 in the target graph data 11. The second information 89 may include second image information that identifies at least one node that contributed to the second inference result 85 in each of one or more similar graph data 84. The image information that identifies the contributing nodes (i.e., the first image information, the second image information) may be, for example, a drawing of the entire target graph data with the contributing nodes indicated by a color scheme or the like.
[0074] In one example, the first image output unit 117 outputs first image information that identifies at least one node that contributed to the first inference result 6 in the target graph data 11. The second image output unit 119 outputs second image information that identifies at least one node that contributed to the second inference result 85 in each of one or more similar graph data 84.
[0075] 6 , the additional information calculation unit 120 calculates association information 70 that associates a node 41 included in the target graph data 11 with a node 61 included in the similar graph data 84, in accordance with the similarity between the first node feature 8 and the second node feature 87. The additional image output unit 121 outputs the association information 70.
[0076] Fig. 11 is a diagram showing an example of a graph machine learning model 20. The graph machine learning model 20 in Fig. 11 illustrates a case of a convolutional GNN (ConvGNN).
[0077] The graph machine learning model 20 includes a first graph convolution layer (Gconv) 21, a pooling layer (Pooling) 22, a second graph convolution layer (Gconv) 23, a readout layer (Read Out) 24, an MLP layer (MLP: Multilayer Perceptron) 25, and a softmax layer (Softmax) 26.
[0078] The first graph convolution layer 21 convolves the graph structure of the input target graph data 11. The target graph data 11 may broadly include node 2 connection relationship information 31 and node initial features 32. The pooling layer 22 is provided following the first graph convolution layer 21, and the output data of the first graph convolution layer 21 and the node 2 connection relationship information 31 are input as input data 12 to the pooling layer 22. The pooling layer 22 pools the graph data into subgraphs.
[0079] The second graph convolution layer 23 is provided following the pooling layer 22, and receives the output data 13 of the pooling layer 22. The second graph convolution layer 23 convolves the graph structure of the subgraph. While the example of Fig. 11 shows a case where two graph convolution layers and one pooling layer are provided, the numbers of graph convolution layers and pooling layers may be changed as appropriate.
[0080] The readout layer 24 is provided subsequent to the second graph convolution layer 23. The readout layer 24 receives the output data 14 from the second graph convolution layer 23. The readout layer 24 reads vector data generated by adding together values of multiple subgraphs.
[0081] The MLP layer 25 is a multilayer perceptron provided following the readout layer 24. The MLP layer 25 receives vector data, which is the output data 15 of the readout layer 24. The MLP layer 25 has multiple weighting coefficients adjusted using the backpropagation method during the training phase. The output data 16 from the final layer of the MLP layer 25 is also vector data. The softmax layer 26 converts the output data 16 of the MLP layer 25 into a sum probability in a specified dimension and outputs an inference result 17.
[0082] FIG. 12 is a diagram showing an example of target graph data 11 input to the graph machine learning model 20. The target graph data 11 roughly includes connection relationship information 31 of node 2 and initial node features 32. In the connection relationship information of node 2 in FIG. 12, 1 is assigned when there is a connection relationship between nodes, and 0 is assigned when there is no connection relationship. As the initial node features 32, node labels A1 to Aa are assigned to each of nodes N1 to Nn. In one example, the node labels A1 to Aa are real values.
[0083] 13 is a diagram showing an example of node features. The node features are a first node feature 8 and a second node feature 87. The node features are acquired based on the input data 12 of the pooling layer 22. The input data 12 includes connection relationship information 31 of node 2 and intermediate features 33 of the node. The intermediate features 33 of the node may be output data of the first graph convolutional layer 21. The connection relationship information 31 of node 2 may be connection relationship information included in the input data to the graph machine learning model 20.
[0084] The intermediate feature 33 of the nodes has real values of intermediate features such as B1, B2, ..., Bb for each of the nodes N1 to Nn. The intermediate feature 33-1 for the node N1 is the node feature for the node N1, the intermediate feature 33-2 for the node N2 is the node feature for the node N2, and the intermediate feature 33-n for the node Nn is the node feature for the node Nn.
[0085] Each intermediate feature 33-1 to 33-n is vector data having 1 to b elements. Therefore, each intermediate feature 33-1 to 33-n is intermediate data generated by the graph machine learning model 20 in the inference phase based on graph data (first requirement). Each intermediate feature 33-1 to 33-n is data corresponding to each node (second requirement). Furthermore, each intermediate feature 33-1 to 33-n can be used to calculate similarity between nodes of different graph data (third requirement). Each intermediate feature 33-1 to 33-n satisfies the first to third requirements for node features and can therefore be used as a node feature. However, as long as the first to third requirements for node features are satisfied, the first node feature 8 and the second node feature 87 are not limited to each intermediate feature 33-1 to 33-n.
[0086] 14 is a diagram showing an example of graph features. The graph features are a first graph feature 7 and a second graph feature 86. The graph features may be obtained based on input data to the MLP layer 25, i.e., output data 15 of the readout layer 24, or may be obtained based on output data 16 of the MLP layer 25. The output data 15 and output data 16 are vector data. The output data 16 of the MLP layer 25 is an example of data obtained by transforming the output data 15 of the readout layer 24.
[0087] The output data 16 is vector data having 1 to c elements. The output data 16 is intermediate data generated by the graph machine learning model 20 in the inference phase based on the graph data (first requirement). The output data 16 is data corresponding to the graph data (second requirement). Furthermore, the output data 16 can be used to calculate similarity between different graph data (third requirement). The output data 16 satisfies the first to third requirements for the graph feature, and therefore can be used as a graph feature. Similarly, the output data 15 also satisfies the first to third requirements for the graph feature, and therefore can be used as a graph feature. However, as long as the first to third requirements for the graph feature are satisfied, the first graph feature 7 and the second graph feature 86 are not limited to the output data 15 and 16.
[0088] [D] Operational Examples [D-1] Training Phase FIG. 15 is a flowchart showing an operational example in the training phase by the information processing device according to one embodiment.
[0089] As illustrated in FIG. 15, the control unit 110 of the information processing device 100 trains the graph machine learning model 20 using the training graph data 4 (step S11).
[0090] The candidate graph data acquisition unit 111 acquires candidate graph data 5 (step S12). The candidate graph data 5 may be a part or all of the training graph data 4 used in training. However, the candidate graph data 5 is not necessarily limited to the training graph data 4.
[0091] [D-2] Inference Phase and Advance Preparation FIG. 16 is a flowchart showing a first example of processing in the inference phase and advance preparation of the information processing device according to an embodiment.
[0092] The second acquisition unit 113 inputs each candidate graph data 5 into the trained graph machine learning model 20 (step S21). The second acquisition unit 113 also acquires inference results 81, which are inference results based on each candidate graph data 5 by the trained graph machine learning model 20, graph features 82, and node features 83, and creates a database 130 (i.e., makes a database).
[0093] The first acquisition unit 112 inputs the target graph data 11 to the trained graph machine learning model 20. The first acquisition unit 112 acquires a first inference result 6, a first graph feature 7, and a first node feature 8, which are inference results by the trained graph machine learning model 20 based on the target graph data 11 (step S22).
[0094] The selection unit 114 selects one or more similar graph data 84 from among the multiple candidate graph data 5 stored in the database 130 (step S23). The similar graph data 84 is graph data in which a second inference result 85 that matches or is similar to the first inference result 6 is inferred using the graph machine learning model 20, and a second graph feature 86 that is similar to the first graph feature 7 is intermediately generated by the graph machine learning model 20.
[0095] The selection unit 114 selects a pair of nodes corresponding to each other as similar nodes based on the similarity between node 2 of the target graph data 11 and node 2 of the similar graph data 84 (step S24).
[0096] The first explanation information calculation unit 116 calculates first information 88 relating to the explanation of the first inference result 6 for the target graph data 11 (step S25).
[0097] The second explanation information calculation unit 118 calculates second information 89 relating to the explanation of the second inference result 85 for the similar graph data 84 (step S26).
[0098] The first image output unit 117 outputs the first information 88 as an image that identifies at least one node that contributed to the first inference result 6 in the target graph data 11 (step S27). The second image output unit 119 outputs the second information 89 as an image that identifies at least one node that contributed to the second inference result 85 in the similar graph data 84 (step S28).
[0099] 6 , the additional information calculation unit 120 calculates association information 70 that associates a node 41 included in the target graph data 11 with a node 61 included in the similar graph data 84, in accordance with the similarity between the first node feature 8 and the second node feature 87. The additional image output unit 121 outputs the association information 70 (step S29).
[0100] 17 is a flowchart illustrating a second example of the processing in the inference phase and advance preparation of the information processing device according to an embodiment. In the operation example illustrated in FIG. 17, output of the association information 70 is omitted.
[0101] The second acquisition unit 113 inputs each candidate graph data 5 to the trained graph machine learning model 20 (step S31). The second acquisition unit 113 also acquires inference results 81 and graph features 82, which are inference results based on each candidate graph data 5 by the trained graph machine learning model 20, and creates a database 130. At this time, the second acquisition unit 113 can omit acquiring the node features 83 and can omit generating the database 132.
[0102] The first acquisition unit 112 inputs the target graph data 11 to the trained graph machine learning model 20. The first acquisition unit 112 acquires a first inference result 6, which is an inference result by the trained graph machine learning model 20 based on the target graph data 11, and a first graph feature 7 (step S32). The first acquisition unit 112 may omit acquiring the first node feature 8.
[0103] The processing in steps S33 to S37 is the same as the processing in steps S23 and S25 to S28 in Fig. 15. Therefore, repeated explanation will be omitted.
[0104] The process of outputting the first image information and the second image information corresponding to the first information 88 and the second information 89 can take various forms. For example, the second explanation information calculation unit 118 may calculate the second information 89 for each of the multiple similar graph data 84. At this time, the second explanation information calculation unit 118 may display, together with each of the second information 89, the similarity of the inference results and the similarity of the graph features between the target graph data 11 and each of the similar graph data 84. Furthermore, the second image output unit 119 may display explanations of the inference results for the multiple similar graph data 84 side by side, based on the similarity of the inference results or the similarity of the graph features between the target graph data 11 and each of the similar graph data 84.
[0105] [E] Effects of an Embodiment According to a technique according to an embodiment, the information processing device 100 acquires a first inference result 6 inferred using the graph machine learning model 20 based on input target graph data 11, and a first graph feature 7 intermediately generated by the graph machine learning model 20 corresponding to the target graph data 11. The information processing device 100 selects similar graph data 84 from a plurality of candidate graph data 5 acquired in advance. The similar graph data 84 is graph data from which a second inference result 85 that matches or is similar to the first inference result 6 is inferred in the graph machine learning model 20, and from which a second graph feature 86 similar to the first graph feature 7 is intermediately generated. The information processing device 100 outputs first information 88 related to an explanation of the first inference result 6 and second information 89 related to an explanation of the second inference result 85.
[0106] As a result, in addition to an explanation of the inference result for the target graph data 11, an explanation of the inference result for the similar graph data 84 that is likely to have a similar reason for inference is also output. Therefore, compared to when only the explanation of the inference result for the target graph data 11 is presented, the analyst can better understand the reason for the inference regarding the inference result of the graph machine learning model 20.
[0107] Furthermore, the information processing device 100 outputs an explanation of the reason for inference for similar data that is more likely to have a similar reason for inference, compared to when outputting an explanation of the reason for inference for similar graph data that only has the same or similar inference result as the target graph data 11. Therefore, the analyst can better understand the reason for inference regarding the inference result of the graph machine learning model 20.
[0108] The first information 88 includes information on at least one node 2 that contributed to the first inference result 6 and that was selected from the nodes 2 included in the target graph data 11. The second information 89 includes information on at least one node 2 that contributed to the second inference result 85 and that was selected from the nodes 2 included in the similar graph data 84.
[0109] This makes it possible to output an explanation of the inference results for both the target graph data 11 and the similar graph data 84 in a format suitable for the graph data.
[0110] The first information 88 includes first image information that identifies at least one node 2 that contributed to the first inference result 6 in the target graph data 11. The second information 89 includes second image information that identifies at least one node 2 that contributed to the second inference result 85 in each of the one or more similar graph data 84.
[0111] This makes it possible to display an explanation of the inference results for both the target graph data 11 and the similar graph data 84 using images in a format suited to the graph data.
[0112] The graph machine learning model 20 includes a first graph convolution layer 21 that convolves the graph structure of graph data, and a pooling layer 22 that follows the first graph convolution layer 21 and pools the graph data into subgraphs. The graph machine learning model 20 also includes a second graph convolution layer 23 that convolves the graph structure of the subgraphs, and a readout layer 24 that follows the second graph convolution layer and reads vector data generated by adding together values of multiple subgraphs. The process of acquiring the first graph feature 7 includes a process of acquiring the first graph feature based on the vector data or data obtained by transforming the vector data.
[0113] In the inference phase using the graph machine learning model 20, the first graph feature 7 can be obtained based on vector data, which is intermediately generated data, thereby reducing the burden of executing new processing separately from the inference processing.
[0114] The information processing device 100 acquires each first element feature (e.g., first node feature 8) which is intermediate data generated by the graph machine learning model 20 in correspondence with each node 2 included in the target graph data 11. The information processing device 100 acquires each second element feature (e.g., second node feature 87) which is intermediate data generated by the graph machine learning model 20 in correspondence with each node 2 included in the similar graph data 84. The information processing device 100 further outputs association information 70 which associates elements included in the target graph data 11 with elements included in the similar graph data 84 in accordance with the degree of similarity between the acquired first element feature and second element feature.
[0115] Therefore, the association of elements between the target graph data 11 and the similar graph data 84 is clearly shown, making it easier to understand the reason for the inference results of the graph machine learning model 20.
[0116] Potential benefits in business situations include the ability to provide more understandable explanations of inference results in network fields such as SNS, academic fields such as chemical compound research, computer communications fields such as communication history, and business fields such as transaction history.
[0117] [F] Others The disclosed technology is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present embodiment. The configurations and processes of the present embodiment can be selected or combined as needed.
[0118] 1, 1a Graph data 2, 2-1 to 2-5, 2-x, 41, 41a-1 to 41a-3, 51a to 51d-2, 61, 61a-1, 61a-2 Node 3, 3-1 to 3-5, 3-x, 41b-1 to 41b-3 Edge 4 Training graph data 5 Candidate graph data 6 First inference result 7 First graph feature 8 First node feature 11 Target graph data 12, 13, 14, 15, 16 Output data 17 Inference result 20 Graph machine learning model 21 First graph convolution layer 22 Pooling layer 23 Second rough convolution layer 24 Readout layer 25 MLP layer 26 Softmax layer 31 Connection relationship information 32 Node initial feature 33, 33-1 to 33-n Intermediate feature 40, 40-1, 40a, 40b, 60, 60-1 Explanation 41a-1 to 41a-3 Contributed nodes 41b-1 to 41b-3 Contributed edges 70 Association information 81 Inference result 82 Graph feature 83 Node feature 84 Similar graph data 85 Second inference result 86 Second graph feature 87 Second node feature 88 First information 89 Second information 100 Information processing device 100a Processor 100b Memory 100c Storage unit 100d IF unit 100e IO unit 100f Reading unit 100h Recording medium 100i Bus 100g Program 110 Control unit 112 First acquisition unit 113 Second acquisition unit 114 Selection unit 115 Similarity calculation unit 116 First explanation information calculation unit 117 First image output unit 118 Second explanation information calculation unit 119 Second image output unit 120 Additional information calculation unit 121 Additional image output unit 130, 131, 132 Database
Claims
1. An output control program that causes a computer to execute the following processes: acquire a first inference result inferred using a trained model based on input target graph data, and a first graph feature that is intermediate data generated by the trained model corresponding to the target graph data; select one or more similar graph data from a plurality of graph data different from the target graph data, in which a second inference result that matches or is similar to the first inference result is inferred using the trained model, and a second graph feature that is similar to the first graph feature is intermediately generated by the trained model; and output first information regarding an explanation of the first inference result and second information regarding an explanation of the second inference result.
2. The output control program of claim 1, wherein the first information includes information of at least one node selected from among the nodes included in the target graph data and contributing to the first inference result, and the second information includes information of at least one node selected from among the nodes included in the similar graph data and contributing to the second inference result.
3. The output control program of claim 2, wherein the first information includes first image information that identifies at least one node in the target graph data that contributed to the first inference result, and the second information includes second image information that identifies at least one node in each of the one or more similar graph data that contributed to the second inference result.
4. The output control program according to any one of claims 1 to 3, wherein the trained model comprises a first graph convolution layer that convolves a graph structure of the graph data, a pooling layer subsequent to the first graph convolution layer that pools the graph data into subgraphs, a second graph convolution layer that convolves the graph structure of the subgraphs, and a readout layer subsequent to the second graph convolution layer that reads out vector data generated by adding together values of a plurality of the subgraphs, and wherein the process of acquiring the first graph feature includes a process of acquiring the first graph feature based on the vector data or data obtained by transforming the vector data.
5. The output control program according to claim 1, further comprising: acquiring first element features, which are intermediate data generated by the trained model in correspondence with each element included in the target graph data; acquiring second element features, which are intermediate data generated by the trained model in correspondence with each element included in the similar graph data; and outputting association information that associates at least one element included in the target graph data with at least one element included in the similar graph data in accordance with a similarity between the acquired first element features and the acquired second element features.
6. The output control program according to claim 5, wherein the trained model comprises a first graph convolution layer that convolves a graph structure of the graph data, a pooling layer subsequent to the first graph convolution layer that pools the graph data into subgraphs, a second graph convolution layer that convolves the graph structure of the subgraphs, and a readout layer subsequent to the second graph convolution layer that reads out vector data generated by adding up values of a plurality of the subgraphs, wherein the process of acquiring the first graph feature comprises a process of acquiring the first graph feature based on the vector data or data obtained by transforming the vector data, and the process of acquiring the second node feature comprises a process of acquiring the second graph feature based on data generated by the first graph convolution layer.
7. An output control method in which a computer executes the following processes: acquiring a first inference result inferred using a trained model based on input target graph data and a first graph feature which is intermediate data generated by the trained model corresponding to the target graph data; selecting one or more similar graph data from a plurality of graph data different from the target graph data, in which a second inference result which matches or is similar to the first inference result is inferred using the trained model and a second graph feature which is similar to the first graph feature is intermediately generated by the trained model; and outputting first information regarding an explanation of the first inference result and second information regarding an explanation of the second inference result.
8. The output control method according to claim 7, wherein the first information includes information of at least one node selected from among the nodes included in the target graph data and contributing to the first inference result, and the second information includes information of at least one node selected from among the nodes included in the similar graph data and contributing to the second inference result.
9. The output control method of claim 8, wherein the first information includes first image information that identifies at least one node in the target graph data that contributed to the first inference result, and the second information includes second image information that identifies at least one node in each of the one or more similar graph data that contributed to the second inference result.
10. The output control method according to any one of claims 7 to 9, wherein the trained model comprises a first graph convolution layer that convolves a graph structure of the graph data, a pooling layer that is provided following the first graph convolution layer and that pools the graph data into subgraphs, a second graph convolution layer that convolves the graph structure of the subgraphs, and a readout layer that is provided after the second graph convolution layer and that reads out vector data generated by adding together values of a plurality of the subgraphs, and wherein the process of obtaining the first graph feature includes a process of obtaining the first graph feature based on the vector data or data obtained by transforming the vector data.
11. The output control method according to claim 7, further comprising the steps of: acquiring first element features, which are intermediate data generated by the trained model corresponding to each element included in the target graph data; acquiring second element features, which are intermediate data generated by the trained model corresponding to each element included in the similar graph data; and outputting association information that associates at least one element included in the target graph data with at least one element included in the similar graph data according to a similarity between the acquired first element features and second element features.
12. The output control method according to claim 11, wherein the trained model comprises a first graph convolution layer that convolves a graph structure of the graph data, a pooling layer that is provided subsequent to the first graph convolution layer and that pools the graph data into a subgraph, a second graph convolution layer that convolves the graph structure of the subgraph, and a readout layer that is provided subsequent to the second graph convolution layer and that reads out vector data generated by adding up values of a plurality of the subgraphs, wherein the process of acquiring the first graph feature comprises a process of acquiring the first graph feature based on the vector data or data obtained by transforming the vector data, and the process of acquiring the second node feature comprises a process of acquiring the second graph feature based on data generated by the first graph convolution layer.
13. An information processing device having a processor that acquires a first inference result inferred using a trained model based on input target graph data and a first graph feature which is intermediate data generated by the trained model corresponding to the target graph data, selects one or more similar graph data from a plurality of graph data different from the target graph data, in which a second inference result that matches or is similar to the first inference result is inferred using the trained model and a second graph feature similar to the first graph feature is intermediately generated by the trained model, and outputs first information regarding an explanation of the first inference result and second information regarding an explanation of the second inference result.
14. An information processing device as described in claim 13, wherein the first information includes information of at least one node selected from among nodes included in the target graph data and contributing to the first inference result, and the second information includes information of at least one node selected from among nodes included in the similar graph data and contributing to the second inference result.
15. An information processing device as described in claim 14, wherein the first information includes first image information that identifies at least one node in the target graph data that contributed to the first inference result, and the second information includes second image information that identifies at least one node in each of the one or more similar graph data that contributed to the second inference result.
16. The information processing device described in any one of claims 13 to 15, wherein the trained model comprises a first graph convolution layer that convolves a graph structure of the graph data, a pooling layer that is provided following the first graph convolution layer and pools the graph data into subgraphs, a second graph convolution layer that convolves the graph structure of the subgraphs, and a readout layer that is provided after the second graph convolution layer and reads out vector data generated by adding together values of a plurality of the subgraphs, wherein the processor, in the process of acquiring the first graph feature, acquires the first graph feature based on the vector data or data obtained by transforming the vector data.
17. The information processing device according to claim 13, further comprising: the processor acquires respective first element features which are intermediate data generated by the trained model corresponding to each element included in the target graph data; acquires respective second element features which are intermediate data generated by the trained model corresponding to each element included in the similar graph data; and further outputs association information which associates at least one element included in the target graph data with at least one element included in the similar graph data in accordance with a similarity between each of the acquired first element features and second element features.
18. The information processing device according to claim 17, wherein the trained model comprises a first graph convolution layer that convolves a graph structure of the graph data, a pooling layer subsequent to the first graph convolution layer that pools the graph data into subgraphs, a second graph convolution layer that convolves the graph structure of the subgraphs, and a readout layer subsequent to the second graph convolution layer that reads out vector data generated by adding together values of a plurality of the subgraphs, wherein the processor, in the process of acquiring the first graph feature, acquires the first graph feature based on the vector data or data obtained by transforming the vector data, and in the process of acquiring the second node feature, acquires the second graph feature based on data generated by the first graph convolution layer.
Citation Information
Patent Citations
Determination result presentation program, determination result presentation method and determination result presentation device
JP2022070561A
Explaining graph-based predictions using network motif analysis
JP2022117452A
Method for providing explainable artificial intelligence
US20220164687A1
