Data selection program, information processing device, and data selection method

Abstract

The present invention involves: determining, for each of a plurality of sets of graph structure data, a weight in data selection on the basis of the number of adjacent nodes common to a combination of two nodes included in the graph structure data; and selecting, from among the plurality of sets of graph structure data on the basis of the determined weight and information indicating whether or not there is a link between nodes for each of one or more combinations of two nodes, training data to be used for training a neural network for predicting whether or not there is a link between nodes included in graph structure data input as input data.

Description

Data selection program, information processing device and data selection method

[0001] The present invention relates to a data selection program, an information processing device, and a data selection method.

[0002] In the field of graph artificial intelligence (AI), there is link prediction, which is a technique for predicting the presence or absence of a link between a certain node pair included in graph data.

[0003] FIG. 1 is a diagram illustrating link prediction.

[0004] When a node group such as that shown by symbol A1 exists, a link prediction such as that shown by symbol A2 is performed. For example, when the link shown by symbol A21 does not exist, it is predicted that the link shown by symbol A22 exists.

[0005] FIG. 2 is a diagram illustrating link prediction using a Graph Convolutional Network (GCN).

[0006] As shown by symbol B1, a distributed representation x is assigned to each node. As shown by symbol B2, a latent variable h for each node is calculated by GCN convolution processing. Then, as shown by symbol B3, the presence or absence of a link is predicted using the inner product of the latent variables h and h as the feature of the link (i, j).

[0007] In learning the GCN weight matrix (in other words, error backpropagation), when a link (i, j) present in the input graph is treated as a positive example, the GCN weight matrix W is updated so that hi·hj approaches 1. On the other hand, when a link (i', j') not present in the input graph is treated as a negative example, the GCN weight matrix W is updated so that hi'·hj' approaches 0.

[0008] The GCN part can be replaced not only with the original GCN but also with other GCN-based methods (e.g., GraphSAGE, and Graph Attention Network (GAT)).

[0009] Link prediction may utilize external knowledge (i.e., information about links that are known to exist or not exist). Node embeddings of the external knowledge are computed and used for link prediction. The embedding vectors are given as distributed representations of the nodes in the inputs of link prediction.

[0010] Thomas N. Kipf, MaxWelling, “Variational Graph Auto-Encoders”, 2016.11.21

[0011] However, since most node embedding methods calculate the distributed representation of each node based on the transition probability between nodes, in situations where sufficient external knowledge is not given, the external knowledge graph may be disconnected and divided into many parts, and node embedding may not be calculated properly.

[0012] One aspect of the present invention aims to improve the accuracy of verification and discovery of inter-node relationships.

[0013] In one aspect, the data selection program determines a weight for data selection for each of a plurality of graph structure data based on the number of adjacent nodes common to two nodes for a pair of two nodes included in the graph structure data, and selects training data from the plurality of graph structure data to be used for training a neural network that predicts the presence or absence of a link between nodes included in the graph structure data input as input data based on the determined weight and information indicating the presence or absence of a link between the nodes for each of one or more pairs of two nodes.

[0014] In one aspect, it is possible to verify and discover inter-node relationships with greater accuracy.

[0015] 7 is a diagram for explaining link prediction; FIG. 8 is a diagram for explaining link prediction by GCN; FIG. 9 is a diagram for explaining a process of using external knowledge to select positive examples and negative examples when learning a GCN in an embodiment; FIG. 10 is a diagram for explaining data representation of a graph; FIG. 11 is a diagram showing characteristics of a node pair; FIG. 12 is a block diagram schematically showing an example of the hardware configuration of an information processing device in an embodiment; FIG. 13 is a flowchart for explaining link prediction processing in an embodiment; and FIG. 14 is a flowchart for explaining details of the positive example selection processing and negative example selection processing shown in FIG.

[0016] [A] Embodiments An embodiment will be described below with reference to the drawings. However, the embodiments shown below are merely examples, and are not intended to exclude various modifications or application of techniques not explicitly stated in the embodiments. In other words, this embodiment can be implemented with various modifications within the scope of its purpose. Furthermore, each figure does not intend to include only the components shown in the figure, but may include other components, etc.

[0017] [A-1] Configuration Example FIG. 3 is a diagram for explaining the process of using external knowledge to select positive examples and negative examples during GCN training in this embodiment.

[0018] 3, for link prediction, external knowledge (in other words, information on links that are known to exist or not exist) indicated by reference symbol C2 may be used for an input graph indicated by reference symbol C1. In the external knowledge indicated by reference symbol C2, a dashed line indicates the presence of a link, and a dashed line indicates the absence of a link.

[0019] The node embedding of the external knowledge is calculated and used for link prediction. For node embedding, DeepWalk, node2vec, or TransE may be used. The embedding vector is given as a distributed representation of the node input for link prediction.

[0020] Link prediction may be used, for example, to verify and predict gene networks, recommend social networking service (SNS) users, and recommend products.

[0021] In verifying and predicting gene networks, nodes correspond to genes and links correspond to regulatory relationships for gene expression, and the regulatory relationships for gene expression are predicted. Gene expression refers to the production of proteins and other products from genes. Elucidating the mechanisms of gene expression in relation to various biological phenomena can lead to the development of drugs and other medicines.

[0022] In the recommendation of SNS users, nodes are associated with users, links with follow relationships, etc., and friend / follow relationships are predicted. Then, other users who may be of interest to a certain user are predicted and recommended.

[0023] In product recommendation, nodes correspond to users and products, and links correspond to follow relationships between users and purchasing relationships between users and products, and products that users are likely to purchase are predicted and recommended.

[0024] FIG. 4 is a diagram illustrating the data representation of a graph.

[0025] In FIG. 4, a matrix format called an adjacency matrix, indicated by reference symbol D2, is expressed for an input graph, indicated by reference symbol D1.

[0026] In a matrix format called an adjacency matrix, elements between adjacent vertices (in other words, between vertices with a link) are expressed as 1, and elements between non-adjacent vertices are 0.

[0027] In the matrix format shown by symbol D2 in Figure 4, for example, for node 1, the elements of nodes 2 and 3 are set to 1, indicating that the node is set, and for node 2, the elements of nodes 1, 5, and 6 are set to 1, indicating that the node is set.

[0028] Many node embedding methods calculate the distributed representation of each node based on the transition probability between nodes. In situations where there is insufficient external knowledge, the external knowledge graph is disconnected and divided into many parts. When the external knowledge graph is disconnected and divided into many parts, many of the transition probabilities between nodes become 0, which makes it difficult to calculate node embedding properly and may prevent knowledge from being reflected in link prediction.

[0029] Therefore, in this embodiment, external knowledge is used to select positive examples and negative examples during GCN learning.

[0030] Links that are known not to exist are not selected as positive examples. In the example shown in Figure 3, link (2, 6) is included in the input graph but is not selected as a positive example.

[0031] On the other hand, edges that are known to exist are not selected as negative examples. In the example shown in Figure 3, the link (4, 9) is not selected as a negative example because its existence is clear from external knowledge.

[0032] Constraints based on external knowledge will enable more accurate selection of positive and negative examples, which is expected to improve prediction accuracy.

[0033] Furthermore, link prediction may take into account hard positives and hard negatives. A hard positive is a node pair that has a link despite the fact that it appears structurally to be unlinked, and a hard negative is a node pair that does not have a link despite the fact that it appears structurally to be linked.

[0034] In GCN, latent variables are calculated so that the distance between the features of adjacent nodes (in other words, links) is close. The more adjacent nodes a node pair shares, the closer the distance between the node pair will be (in other words, the easier it will be to predict if there is a link). Therefore, node pairs that share many adjacent nodes but have no link, or node pairs that share few adjacent nodes but have a link, are more likely to result in incorrect predictions. Therefore, by selecting node pairs that are more likely to result in incorrect predictions as training data, prediction accuracy can be improved.

[0035] Therefore, the weighting of the training data sampling is such that if there is no link between a node pair that shares many adjacent nodes, it is more likely to be selected as a negative example, and if there is a link between a node pair that shares few or no adjacent nodes, it is more likely to be selected as a positive example.

[0036] FIG. 5 is a diagram showing the characteristics of a node pair.

[0037] 5, in an example of a node pair with a possible link, denoted by reference symbol E1, nodes A and B share three adjacent nodes. On the other hand, in an example of a node pair with an unlikely link, denoted by reference symbol E2, nodes A and B do not share any adjacent nodes.

[0038] FIG. 6 is a block diagram schematically illustrating an example of the hardware configuration of the information processing device 1 according to the embodiment.

[0039] As shown in FIG. 6, the information processing device 1 is an example of a computer, and includes a CPU 11 , a memory 12 , a display control device 13 , a storage device 14 , an input interface (IF) 15 , an external recording medium processing device 16 , and a communication IF 17 .

[0040] The memory 12 is an example of a storage unit, and may be, for example, a read-only memory (ROM) or a random access memory (RAM). A program such as a basic input / output system (BIOS) may be written to the ROM of the memory 12. The software program in the memory 12 may be read and executed by the CPU 11 as appropriate. The RAM of the memory 12 may be used as a temporary storage memory or a working memory.

[0041] The display control device 13 is connected to and controls the display device 131. The display device 131 is a liquid crystal display, an organic light-emitting diode (OLED) display, a cathode ray tube (CRT), an electronic paper display, or the like, and displays various information to an operator, etc. The display device 131 may be combined with an input device, such as a touch panel. The display device 131 displays various information to a user of the information processing device 1.

[0042] The storage device 14 is a storage device with high IO performance, and may be, for example, a dynamic random access memory (DRAM), a solid state drive (SSD), a storage class memory (SCM), or a hard disk drive (HDD).

[0043] The input IF 15 may be connected to input devices such as a mouse 151 and a keyboard 152, and may control the input devices such as the mouse 151 and the keyboard 152. The mouse 151 and the keyboard 152 are examples of input devices, and an operator performs various input operations via these input devices.

[0044] The external recording medium processing device 16 is configured to allow a recording medium 160 to be attached thereto. The external recording medium processing device 16 is configured to be able to read information recorded on the recording medium 160 when the recording medium 160 is attached thereto. In this example, the recording medium 160 is portable. For example, the recording medium 160 is a non-transitory recording medium such as a flexible disk, an optical disk, a magnetic disk, a magneto-optical disk, or a semiconductor memory.

[0045] The communication IF 17 is an interface that enables communication with an external device.

[0046] The CPU 11 is an example of a processor, and is a processing device that performs various controls and calculations. The CPU 11 realizes various functions by executing an operating system (OS) and programs loaded into the memory 12. The CPU 11 may be a multiprocessor including multiple CPUs, a multi-core processor having multiple CPU cores, or a configuration having multiple multi-core processors.

[0047] The device for controlling the overall operation of the information processing device 1 is not limited to the CPU 11, and may be, for example, any one of an MPU, DSP, ASIC, PLD, and FPGA. The device for controlling the overall operation of the information processing device 1 may also be a combination of two or more of the CPU, MPU, DSP, ASIC, PLD, and FPGA. Note that MPU is an abbreviation for Micro Processing Unit, DSP is an abbreviation for Digital Signal Processor, and ASIC is an abbreviation for Application Specific Integrated Circuit. Furthermore, PLD is an abbreviation for Programmable Logic Device, and FPGA is an abbreviation for Field Programmable Gate Array.

[0048] The information processing device 1 executes GCN learning for link prediction according to the following steps (1) to (5). The input graph is represented by G(v, e), and the knowledge graph (with links) is represented by K. + (v + ,e + ) and the knowledge graph (without links) is K - (v - ,e - ), where v represents a node and e represents an edge (in other words, a link).

[0049] (1) The information processing device 1 acquires an input graph G and assigns a distributed representation to each node.

[0050] (2) The information processing device 1 generates a knowledge graph K + ,K - is obtained.

[0051] (3) The information processing device 1 inputs G into the GCN and calculates the latent variable h of each node.

[0052] (4) The information processing device 1 samples positive and negative examples. e is extracted by weighted random sampling. In the case of a positive example, ei∈K + The number of neighboring nodes shared by both ends of i Then e i The weight of (1+1 / n i) In the case of negative examples, e i ∈K - The number of neighboring nodes shared by both ends of i Then e i The weight of n i When extracting positive examples, e∈K - , when extracting negative examples, e∈K + If extracted, resampling is performed without selecting it as a positive or negative example.

[0053] (5) The information processing device 1 updates the weight matrix of the GCN using backpropagation of errors from the positive and negative examples.

[0054] [A-2] Operation Example The link prediction process in this embodiment will be described with reference to the flowchart (steps S1 to S17) shown in FIG.

[0055] The information processing device 1 acquires an input graph G and assigns a distributed representation to each node (step S1).

[0056] The information processing device 1 generates a knowledge graph K + ,K ー is acquired (step S2).

[0057] The information processing device 1 inputs G into the GCN and calculates the latent variable h of each node (step S3).

[0058] The information processing device 1 executes a process of selecting positive examples and generates a list L of positive examples. + (Step S4). The positive example selection process in Step S4 will be described in detail later with reference to FIG.

[0059] The information processing device 1 executes a negative example selection process and generates a negative example list L ー (Step S5). Details of the negative example selection process in step S5 will be described later with reference to FIG.

[0060] The information processing device 1 receives the positive example L + , negative example L ー Then, the weight matrix of the GCN is updated by backpropagation (step S6), and the link prediction process is completed.

[0061] Next, the positive example selection process (step S4) shown in FIG. 7 will be described in detail with reference to the flowchart (steps S41 to S44) shown in FIG.

[0062] The information processing device 1 assigns the following weighting to each edge and extracts edge e (step S41): i ∈K + The number of neighboring nodes shared by both ends of i Then e i The weight of (1+1 / n i )

[0063] The information processing device 1 is ー It is determined whether the following holds true (step S42).

[0064] e∈K ー If this is true (see the Yes route in step S42), the process returns to step S41.

[0065] e∈K ー does not hold (see the No route in step S42), the information processing device 1 adds the extracted edge e to the list L of positive examples. + (step S43).

[0066] The information processing device 1 is + It is determined whether the length of is less than N (step S44).

[0067] L + If the length of is less than N (see the Yes route in step S44), the process returns to step S41.

[0068] L + If the length of is not less than N (see the No route in step S44), the positive example selection process ends.

[0069] Next, the negative example selection process (step S5) shown in FIG. 7 will be described in detail with reference to the flowchart (steps S51 to S54) shown in FIG.

[0070] The information processing device 1 assigns the following weighting to each edge and extracts edge e (step S51): i ∈K ー The number of neighboring nodes shared by both ends ofi Then e i The weight of n i Let's say.

[0071] The information processing device 1 is ＋ It is determined whether the following holds true (step S52).

[0072] e∈K ＋ If this is true (see the Yes route in step S52), the process returns to step S51.

[0073] e∈K ＋ does not hold (see the No route in step S52), the information processing device 1 adds the extracted edge e to the list L of positive examples. ー (step S53).

[0074] The information processing device 1 is ー It is determined whether the length of is less than N (step S54).

[0075] L ー If the length of is less than N (see the Yes route in step S54), the process returns to step S51.

[0076] L ー If the length of is not less than N (see the No route in step S54), the positive example selection process ends.

[0077] [B] Effects According to the data selection program, the information processing device, and the data selection method in the above-described embodiment, the following effects can be achieved, for example.

[0078] The information processing device 1 determines a weight for data selection for each of a plurality of graph structure data based on the number of adjacent nodes common to a pair of two nodes included in the graph structure data. Based on the determined weight and information indicating the presence or absence of a link between the nodes for each of one or more pairs of two nodes, the information processing device 1 selects, from the plurality of graph structure data, training data to be used for training a neural network that predicts the presence or absence of a link between nodes included in graph structure data input as input data.

[0079] This makes it possible to improve the accuracy of verification and discovery of inter-node relationships.

[0080] The information processing device 1 selects two nodes as positive examples when there is a link between two nodes that share fewer than a predetermined number of adjacent nodes or no nodes at all. The information processing device 1 selects two nodes as negative examples when there is a link between two nodes that share a predetermined number of adjacent nodes or more. The information processing device 1 updates the weight matrix of the neural network by backpropagation from the nodes selected as positive examples and the negative examples.

[0081] This will allow us to restrict the positive and negative examples selected using external knowledge, which will lead to more accurate selection and improved accuracy.In addition, by selecting links that are prone to misprediction as training data using external knowledge, we can expect to improve accuracy.

[0082] [C] Others The disclosed technology is not limited to the above-described embodiment, and various modifications can be made without departing from the spirit of the present embodiment. The configurations and processes of the present embodiment can be selected or combined as needed.

[0083] 1: Information processing device 11: CPU 12: Memory 13: Display control device 14: Storage device 15: Input IF 16: External recording medium processing device 17: Communication IF 131: Display device 151: Mouse 152: Keyboard 160: Recording medium

Claims

1. For each of the multiple graph structure data, the weights for data selection are determined based on the number of neighboring nodes common to the two nodes included in the graph structure data. Based on the determined weights and information indicating the presence or absence of links between nodes for each pair of one or more nodes, training data is selected from the plurality of graph structure data to be used to train a neural network that predicts the presence or absence of links between nodes included in the graph structure data input as input data. A data selection program that causes a computer to perform a process.

2. If a link exists between two nodes that share an adjacent node with fewer than a predetermined number of nodes or none of the nodes that do, the two nodes are selected as positive examples. If a link exists between the two nodes that share a predetermined number or more adjacent nodes, the two nodes are selected as negative examples. The weight matrix of the neural network is updated by backpropagation using the nodes selected as the positive and negative examples. A data selection program according to claim 1, which causes the computer to perform the processing.

3. For each of the multiple graph structure data, the weights for data selection are determined based on the number of neighboring nodes common to the two nodes included in the graph structure data. Based on the determined weights and information indicating the presence or absence of links between nodes for each pair of one or more nodes, training data is selected from the plurality of graph structure data to be used to train a neural network that predicts the presence or absence of links between nodes included in the graph structure data input as input data. An information processing device equipped with a processor.

4. For each of the multiple graph structure data, the weights for data selection are determined based on the number of neighboring nodes common to the two nodes included in the graph structure data. Based on the determined weights and information indicating the presence or absence of links between nodes for each pair of one or more nodes, training data is selected from the plurality of graph structure data to be used to train a neural network that predicts the presence or absence of links between nodes included in the graph structure data input as input data. A method of data selection in which a computer performs processing.