Information processing program, information processing method, and information processing device

Abstract

To provide an information processing program, an information processing method and an information processing apparatus capable of generating natural sentences in which weights are reflected, based on graph node data in which relations between a plurality of nodes, each of which is given a weight, are defined as edges.SOLUTION: The information processing program defines relations between a plurality of first nodes, each of which is given a weight, as a plurality of second nodes based on first graph node data 11a defining relations between the plurality of first nodes as edges; and causes a computer to perform a process of generating a word based on a first graph vector which generates second graph node data 26 in which a weight given to each of the plurality of first nodes is defined as an edge and which is obtained from the plurality of second nodes of the second graph node data 26 and a second graph vector obtained from the plurality of second nodes of the second graph node data 26 and the edges for which the weights are defined.SELECTED DRAWING: Figure 8

Description

[Technical Field]

[0001] The present invention relates to an information processing program, an information processing method, and an information processing device. [Background technology]

[0002] An AI (Artificial Intelligence) task is known that generates a plurality of words, such as natural sentences, related to the content of graph node data based on graph node data in which relationships between a plurality of nodes are defined as edges. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] International Publication No. 2019 / 146084 Brochure Summary of the Invention [Problem to be solved by the invention]

[0004] The AI task described above does not assume that weighted graph node data, in which weights are assigned to each of multiple nodes, will be processed.

[0005] For example, when natural language sentences are generated based on weighted graph node data in the AI task described above, the weights of each node are ignored or deleted, and the weights of each node may not be reflected in the natural language sentences.

[0006] In one aspect, one of the objectives of the present invention is to generate natural language sentences that reflect weights based on graph node data that defines the relationships between multiple nodes, each of which has a weight assigned, as edges. [Means for solving the problem]

[0007] In one aspect, an information processing program may cause a computer to execute the following processes. The processes may include, based on first graph node data in which relationships between a plurality of first nodes, each of which has a weight assigned thereto, are defined as edges, generating second graph node data in which relationships between the plurality of first nodes are defined as a plurality of second nodes and weights assigned to each of the plurality of first nodes are defined as edges. The processes may also include performing word generation based on first graph vectors obtained from a plurality of second nodes in the second graph node data and second graph vectors obtained from a plurality of second nodes in the second graph node data and the edges to which the weights are assigned. [Effects of the Invention]

[0008] In one aspect, the present invention can generate natural sentences that reflect weights based on graph node data that defines the relationships between multiple nodes, each of which has a weight assigned, as edges. [Brief explanation of the drawings]

[0009] [Figure 1] FIG. 2 is a block diagram illustrating an example of the hardware (HW) configuration of a computer that realizes the functions of a generating device according to an embodiment. [Figure 2] FIG. 2 is a block diagram illustrating an example of a software configuration of a generating device according to an embodiment. [Figure 3] FIG. 1 is a diagram for explaining an example of the relationship between a relationship graph and natural language sentences. [Figure 4] 10 is a flowchart illustrating an example of the operation of machine learning processing by a generating device according to an embodiment. [Figure 5] FIG. 10 is a diagram for briefly explaining an example of the operation of machine learning processing by the generating device. [Figure 6] 10 is a flowchart illustrating an example of the operation of an inference process performed by a generating device according to an embodiment. [Figure 7] FIG. 10 is a diagram for briefly explaining an example of the operation of an inference process performed by the generating device. [Figure 8]FIG. 10 is a diagram illustrating an example of a graph conversion process. [Figure 9] FIG. 10 is a diagram for explaining another example of the graph conversion process. [Figure 10] FIG. 10 is a diagram illustrating an example of a vector generation process. [Figure 11] FIG. 10 is a diagram illustrating an example of a weight-considered vector generation process. [Figure 12] 10A and 10B are diagrams illustrating an example of a word generation process performed by a word generation unit. [Figure 13] FIG. 10 is a diagram showing an example of a processing result of word generation based on a relationship graph. DETAILED DESCRIPTION OF THE INVENTION

[0010] Hereinafter, embodiments of the present invention will be described with reference to the drawings. However, the embodiments described below are merely examples, and are not intended to exclude various modifications or application of techniques not explicitly described below. For example, the present embodiment can be implemented with various modifications within the scope of the spirit thereof. In the drawings used in the following description, parts with the same reference numerals represent the same or similar parts unless otherwise specified.

[0011] [A] Configuration example An example of the configuration of a generating device 1 (see FIG. 2) according to an embodiment will be described below.

[0012] [A-1] Hardware configuration example An apparatus for realizing the generating apparatus 1 according to an embodiment may be a virtual server (VM) or a physical server. The functions of the generating apparatus 1 may be realized by one computer or two or more computers. Furthermore, at least some of the functions of the generating apparatus 1 may be realized using HW resources and network (NW) resources provided by a cloud environment.

[0013] 1 is a block diagram showing an example of the HW configuration of a computer 10 that realizes the functions of a generation device 1 according to an embodiment. When multiple computers are used as HW resources that realize the functions of the generation device 1, each computer may have the HW configuration shown in FIG.

[0014] As shown in FIG. 1, the computer 10 may include, as its HW configuration, a processor 10a, a memory 10b, a storage unit 10c, an IF (Interface) unit 10d, an I / O (Input / Output) unit 10e, and a reading unit 10f, for example.

[0015] The processor 10a is an example of a processing unit that performs various controls and calculations. The processor 10a may be connected to each block in the computer 10 via a bus 10i so that they can communicate with each other. The processor 10a may be a multiprocessor including multiple processors, a multi-core processor having multiple processor cores, or a configuration having multiple multi-core processors.

[0016] The processor 10a may be, for example, an integrated circuit (IC) such as a CPU, MPU, GPU, APU, DSP, ASIC, or FPGA. Note that a combination of two or more of these integrated circuits may be used as the processor 10a. MPU is an abbreviation for Micro Processing Unit. GPU is an abbreviation for Graphics Processing Unit, and APU is an abbreviation for Accelerated Processing Unit. DSP is an abbreviation for Digital Signal Processor, ASIC is an abbreviation for Application Specific IC, and FPGA is an abbreviation for Field-Programmable Gate Array.

[0017] For example, the processor 10a may be a combination of a processing device such as a CPU that executes various controls in the generation device 1 and an accelerator that executes machine learning processing in the generation device 1. Examples of the accelerator include the above-mentioned GPU, APU, DSP, ASIC, or FPGA.

[0018] The memory 10b is an example of HW that stores various types of data, programs, and other information. Examples of the memory 10b include volatile memory such as DRAM, and / or nonvolatile memory such as NVDIMM and PM (Persistent Memory).

[0019] The storage unit 10c is an example of HW that stores various types of data, programs, and other information. Examples of the storage unit 10c include various storage devices such as a magnetic disk device such as a hard disk drive (HDD), a semiconductor drive device such as an SSD, and a nonvolatile memory. Examples of nonvolatile memory include a flash memory, a storage class memory (SCM), and a read-only memory (ROM).

[0020] The storage unit 10c may also store a program 10g (information processing program) that implements all or part of the various functions of the computer 10. For example, the processor 10a can implement the function of a generating device 1 (e.g., a control unit 17) described below by expanding the program 10g stored in the storage unit 10c into the memory 10b and executing it.

[0021] The IF unit 10d is an example of a communication IF that controls connection and communication with one or both of the networks. For example, the IF unit 10d may include an adapter that complies with a LAN (Local Area Network) such as Ethernet (registered trademark) or optical communication such as FC (Fibre Channel). The adapter may support one or both of wireless and wired communication methods. For example, the generation device 1 may be communicatively connected to other devices, such as a device that provides a relational graph and natural language sentences to the generation device 1 and a device that receives the processing results of the inference processing from the generation device 1, via the IF unit 10d and a network (not shown). Furthermore, for example, the program 10g may be downloaded from the network to the computer 10 via the communication IF and stored in the storage unit 10c.

[0022] The I / O unit 10e may include one or both of an input device and an output device. Examples of input devices include a keyboard, a mouse, a touch panel, etc. Examples of output devices include a monitor, a projector, a printer, etc.

[0023] The reading unit 10f is an example of a reader that reads data and program information recorded on the recording medium 10h. The reading unit 10f may include a connection terminal or device to which the recording medium 10h can be connected or inserted. Examples of the reading unit 10f include an adapter that complies with USB (Universal Serial Bus) or the like, a drive device that accesses a recording disk, and a card reader that accesses a flash memory such as an SD card. Note that the recording medium 10h may store the program 10g, and the reading unit 10f may read the program 10g from the recording medium 10h and store it in the memory unit 10c.

[0024] Examples of the recording medium 10h include non-transitory computer-readable recording media such as magnetic / optical disks and flash memories. Examples of magnetic / optical disks include flexible disks, CDs (Compact Discs), DVDs (Digital Versatile Discs), Blu-ray Discs, and HVDs (Holographic Versatile Discs). Examples of flash memories include semiconductor memories such as USB memories and SD cards.

[0025] The above-described hardware configuration of the computer 10 is an example. Therefore, the hardware in the computer 10 may be increased or decreased (for example, adding or deleting any block), divided, integrated in any combination, or buses may be added or deleted as appropriate. For example, at least one of the I / O unit 10e and the reading unit 10f may be omitted.

[0026] [A-2] Software configuration example 2 is a block diagram showing an example of the software configuration of a generation device 1 according to an embodiment. The generation device 1 is an example of an information processing device, and executes a generation process for generating natural sentences based on a weighted relation graph.

[0027] 2, the generation device 1 may include, as a software (functional) configuration, a memory unit 11, an acquisition unit 12, a graph conversion unit 13, a vector generation unit 14, a word generation unit 15, and an output unit 16. The acquisition unit 12, the graph conversion unit 13, the vector generation unit 14, the word generation unit 15, and the output unit 16 are examples of a control unit 17.

[0028] The memory unit 11 is an example of a storage area and stores various data used by the generation device 1. As shown in Fig. 2, the memory unit 11 may be capable of storing, for example, a relationship graph 11a, natural sentences 11b, and words 11c. Note that the memory unit 11 may be capable of storing various information generated in the course of processing by the generation device 1, which will be described later, such as graphs, vectors, model data of machine learning models, parameters, and the like.

[0029] The acquisition unit 12 acquires the relationship graph 11a and the natural sentence 11b and stores them in the memory unit 11. For example, the acquisition unit 12 may generate one or both of the relationship graph 11a and the natural sentence 11b in the generation device 1, or may receive them from a computer external to the generation device 1 via a network (not shown).

[0030] The relationship graph 11a is an example of first graph node data in which the relationships between a plurality of first nodes, each of which is assigned a weight, are defined as edges.

[0031] The natural sentence 11b is an example of a plurality of words related to the contents of the nodes included in the relationship graph 11a, and is a sentence that reflects the weights assigned to the respective nodes included in the relationship graph 11a.

[0032] For example, the acquiring unit 12 may acquire the relationship graph 11a and the natural language sentence 11b as training data (teacher data) in the machine learning phase, and may acquire the relationship graph 11a to be subjected to the inference process in the inference phase.

[0033] Fig. 3 is a diagram for explaining an example of the relationship between the relationship graph 11a and the natural sentence 11b. In Fig. 3, symbol A1 indicates the relationship graph 11a which is the input graph, and symbol A2 indicates the natural sentence 11b which is the output sentence (correct label) generated by reflecting the weight based on the relationship graph 11a. The example in Fig. 3 shows a case where the relationship graph 11a is a gene information graph.

[0034] The relationship graph 11a includes multiple nodes denoted by symbols A11 to A15, and is a graph in which the relationships between the nodes are connected by edges. The relationship graph 11a also includes weights (see numbers in square boxes) assigned to the nodes. The weights are information associated with nodes and edges, and, as an example, indicate the importance of other nodes (nodes A12 to A15) relative to a certain target node (node A11).

[0035] In the example of FIG. 3, node A12 "mutation type B" is connected to node A11 "mutation A" (target) by edge "mutation type". Furthermore, a weight (importance) of "0.5" is assigned to node A12 "mutation type B" and its edge "mutation type". In this case, nodes A11 to A12 in the relationship graph 11a indicate that "mutation A is a mutation of mutation type B" (importance: "0.5"). Note that the weight of nodes connected in multiple stages to target node A11 (e.g., node A15; A11←A14←A15) may be weighted by multiplication or the like with the weight of each node (e.g., node A14) leading up to the target.

[0036] The natural sentence 11b is a sentence about the contents of a plurality of nodes, written so that the greater the weight of the relationship in the relationship graph 11a, the closer it is to the beginning of the sentence, as indicated by reference symbol A2 in FIG.

[0037] The generation device 1 performs training of the machine learning model through processing described below in the machine learning phase so that the output (plurality of words) when the relationship graph 11a is input approaches the natural sentence 11b.

[0038] Furthermore, the generating device 1 outputs a plurality of words 11c, which are the output when the relationship graph 11a is used as input to a trained machine learning model, through processing in the inference phase, which will be described later.

[0039] Returning to the explanation of FIG. 2, the graph conversion unit 13 converts the relationship graph 11a, which is an input weighted directed graph, into a weighted relationship graph to be input to the machine learning model.

[0040] The vector generation unit 14 generates a graph vector that does not take weights into consideration and a weight-considered graph vector that takes weights into consideration, based on the relationship graph converted by the graph conversion unit 13. The vector generation unit 14 may, for example, perform machine learning of a machine learning model for generating these vectors in the machine learning phase. An example of a machine learning model for generating vectors is a neural network such as GCN-LSTM (Graph Convolutional Networks - Long Short Term Memory).

[0041] The word generation unit 15 executes a process of generating words 11c based on the graph vector and the weight-considered graph vector. For example, in the machine learning phase, the word generation unit 15 executes machine learning of a machine learning model for generating a plurality of words (for example, natural sentences). An example of a machine learning model for generating a plurality of words (for example, natural sentences) is a neural network such as LSTM including an Attenuation Mechanism (Att). The attention mechanism is a mechanism that weights important parts such as graph edges through machine learning, and in one embodiment, the hidden layer on the input side is weighted to generate words while taking important words into consideration.

[0042] For example, the word generation unit 15 uses a hidden layer that takes edge weights into account in parallel with a normal hidden layer by inputting the graph vector and the weight-aware graph vector into separate attention mechanisms. This allows the word generation unit 15 to take into account the weights added to the relationship graph 11a in word generation in addition to the important parts of the graph vector obtained from the attention mechanism.

[0043] In the inference phase, the output unit 16 outputs the multiple words 11c output from the word generation unit 15 as inference results.

[0044] [B] Example of operation Next, an example of the operation of the generating device 1 according to an embodiment will be described.

[0045] (machine learning phase) FIG. 4 is a flowchart illustrating an example of the operation of the machine learning process by the generating device 1 according to an embodiment, and FIG. 5 is a diagram for simply explaining the example of the operation of the machine learning process by the generating device 1.

[0046] As shown in FIG. 4, in the machine learning phase, the acquisition unit 12 of the generation device 1 acquires a set of a relation graph 11a and a natural sentence 11b as training data (step S1).

[0047] 5, the generation device 1 inputs training data to the encoder unit 2 (process P1 in FIG. 5). The encoder unit 2 has a function of executing graph conversion and graph vectorization by the graph conversion unit 13 and the vector generation unit 14.

[0048] The graph conversion unit 13 of the encoder unit 2 converts the relationship graph 11a into a graph for vector processing (step S2).

[0049] The vector generation unit 14 of the encoder unit 2 generates a node vector 21 of the node based on the transformed graph (step S3; process P21 in FIG. 5). The node vector 21 is a node vector that holds information on the weight 22.

[0050] Moreover, the vector generation unit 14 generates a weight-considered vector 23 that takes the weight 22 into consideration by numerically multiplying the node vector 21 by the weight 22 (see FIG. 5) (step S4; process P22).

[0051] The vector generation unit 14 generates graph vectors from the node vector 21 and the weight consideration vector 23 (step S5).

[0052] For example, the vector generation unit 14 generates a graph vector 24 including entities and relation nodes from the node vector 21 (process P23).

[0053] Furthermore, the vector generation unit 14 generates a weight consideration graph vector 25 including entities and relation nodes from the weight consideration vector 23 (process P24).

[0054] The encoder unit 2 inputs the graph vector 24 and the weight-considered graph vector 25 to the decoder unit 3. The decoder unit 3 is a function that executes a word generation process by the word generation unit 15 using the graph vector 24 and the weight-considered graph vector 25.

[0055] The word generation unit 15 of the decoder unit 3 generates a graph vector (Att) based on the graph vector 24 and an AW (Attenuation Weight) 31 for the node vector of Att (step S6; process P31).

[0056] Furthermore, the word generation unit 15 generates a weight consideration graph vector (Att) based on the weight consideration graph vector 25 and AW32 for the weight consideration vector of Att (step S7; process P32).

[0057] The word generation unit 15 generates a combined graph vector by combining the graph vector (Att) generated in the processes P31 and P32 with the weight-considered graph vector (Att) (step S8; process P33).

[0058] The word generation unit 15 generates a new context vector based on the context vectors generated so far and the joint graph vector generated in step S8 (process P33) (step S9; process P34).

[0059] The word generation unit 15 predicts the word 11c based on the probability from the generated context vector (step S10; process P35).

[0060] The word generation unit 15 updates parameters of a machine learning model for generating a plurality of words (e.g., natural sentences) based on step S10 (process P35) (step S11). For example, the word generation unit 15 may generate a new context vector and update parameters of a process for predicting the word 11c, as well as parameters of a process for generating a graph vector (Att) and a weight-considered graph vector (Att).

[0061] The word generation unit 15 determines whether or not to end the machine learning process (step S12), and if the machine learning process is to be ended (YES in step S12), the process ends.

[0062] If the machine learning process does not end (NO in step S12), the process proceeds to step S6. For example, the word generation unit 15 inputs the word 11c predicted in process P35 to processes P31, P32, and P34, respectively.

[0063] In this way, the word generation unit 15 repeats the processes P31 to P35 for each word, and executes the machine learning process by updating the parameters of the machine learning model so that the generated words become closer to the natural sentence 11b.

[0064] (Inference phase) FIG. 6 is a flowchart illustrating an example of the operation of the inference processing by the generating device 1 according to an embodiment, and FIG. 7 is a diagram for simply explaining the example of the operation of the inference processing by the generating device 1.

[0065] As shown in FIG. 6, in the inference phase, the acquisition unit 12 of the generating device 1 acquires input data (relationship graph 11a) for the inference process (step S21).

[0066] As shown in FIG. 7, the generating device 1 inputs input data to the encoder unit 2 (process P41 in FIG. 5).

[0067] In steps S2 to S5 (processing P21 to P24), the encoder unit 2 acquires the graph vector 24 and the weighted graph vector 25 from the relationship graph 11a using a trained machine learning model. Note that the relationship graph 11a used in the inference phase may have different contents from the relationship graph 11a used in the machine learning phase.

[0068] In steps S6 to S10 (processing P31 to P35), the decoder unit 3 acquires a plurality of words 11c from the graph vector 24 and the weight consideration graph vector 25 using the trained machine learning model and AWs 31 and 32.

[0069] After step S10, the word generation unit 15 determines whether or not to end the inference process (step S22). If the inference process is to be ended (YES in step S22), the generation device 1 (output unit 16) outputs the generated multiple words 11c as inference results (step S23; process P42), and the process ends.

[0070] If the inference process is not to be ended (NO in step S22), the process proceeds to step S6. For example, the word generation unit 15 inputs the word 11c predicted in process P35 to processes P31, P32, and P34, respectively.

[0071] [C] Explanation of each process Next, an example of each process shown in FIG. 4 and FIG. 6 will be described.

[0072] [C-1] Graph transformation processing Fig. 8 is a diagram for explaining an example of the graph conversion process (step S2 in Figs. 4 and 6). As shown in Fig. 8, the graph conversion unit 13 converts the edges of the relationship graph 11a (see symbol B1) into nodes (maintaining the information on the weights 22 in Figs. 5 and 7), thereby generating a weighted relationship graph 26 (see symbol B2) in which the relationships between nodes are converted into nodes.

[0073] The weighted relationship graph 26 is an example of second graph node data in which the relationships between multiple first nodes in the relationship graph 11a are defined as multiple second nodes, and the weights assigned to each of the multiple first nodes are defined as edges.

[0074] For example, the edges (relationship edges) within a one-hop range from the target node B11 in the relationship graph 11a are (B11, B12) [0.5], (B11, B13) [0.2], and (B11, B14) [0.1]. Note that the parentheses indicate the signs of the nodes at both ends of the edge, and the brackets indicate the weight.

[0075] In contrast, the edges within one hop from the target node B21 in the weighted relationship graph 26 are (B21, B26) [0.5], (B21, B27) [0.2], and (B21, B28) [0.1]. In this way, in the weighted relationship graph 26, edges are defined to have at least a weight.

[0076] In the graph conversion process, if there is a common term (node B21 in the example of Figure 8) among multiple nodes in the relationship graph 11a, the graph conversion unit 13 may combine the common terms into one to generate a weighted relationship graph 26 shown by symbol B2 in Figure 8.

[0077] In this way, by converting edges into nodes through graph conversion processing, it becomes possible to take into account relationships including weights between nodes in the vector generation processing described below.

[0078] In the graph conversion process, the graph conversion unit 13 may convert the relationship graph 11a into a text model instead of a graph model.

[0079] FIG. 9 is a diagram for explaining another example of graph conversion processing. The relationship between the graph model and the text model is shown by reference symbol B3. As shown by reference symbol B3, for example, a graph model including nodes B31 and B32 and an edge from node B32 to node B31 has node B31 as <h>(Head), Node B32 <t>(Tail), edge <r>It can be converted into a linear form as (Relation).

[0080] 9, the graph conversion unit 13 may convert the relationship graph 11a into a text model 26a in accordance with the relationship indicated by reference symbol B3. For example, the graph conversion unit 13 may convert the relationship graph 11a into a text model 26a in accordance with the relationship indicated by reference symbol B3. <h>Mutation A <r>Mutation Type <t>A text model 26a is generated so that it becomes "mutation type B".

[0081] In the following explanation, the generation device 1 will be described as an example of the processing performed when converting the relationship graph 11a into a weighted relationship graph 26 (see Figure 8) of a graph model, but similar processing may also be performed when converting into a text model 26a (see Figure 9).

[0082] [C-2] Vector generation processing 10 is a diagram for explaining an example of the vector generation process (step S3 in FIGS. 4 and 6). The weighted relationship graph 26 generated in FIG. 8 is shown by reference symbol C1, and the node vector 21 generated by the vector generation process is shown by reference symbol C2.

[0083] 10, the vector generation unit 14 generates a node vector 21 (see symbol C2) taking into consideration peripheral nodes from each node (nodes C11 to C19) of the weighted relationship graph 26 (see symbol C1). The node vector 21 includes element nodes and relationship nodes, and holds weights 22.

[0084] For example, the vector generation unit 14 may perform machine learning of a GCN that takes into account neighboring nodes (for example, neighboring nodes up to two hops away).

[0085] As shown by reference symbol C2 in FIG. 10, node C11 of "mutation A" is converted into element vector C21 of [0.1, 0.43, 0.9, ...], and node C16 of "mutation type" is converted into relation vector C22 of [0.2, 0.13, 0.3, ...]. Also, node C12 of "mutation type B" is converted into element vector C23 of [0.03, 0.9, 0.1, ...]. At this time, nodes C22 and C23 hold "0.5" as weight 22. The number of dimensions D of node vector 21 is, for example, 1*d n , the relation node is 1*d r It may be expressed as:

[0086] The vector generating unit 14 also performs a similar process on the text model 26a to generate the node vector 21 from the text model 26a while maintaining the weights 22.

[0087] Furthermore, the vector generation unit 14 generates a weight-considered vector 23 by numerically reflecting (for example, multiplying) the weight 22 on the generated node vector 21.

[0088] 11 is a diagram for explaining an example of the weight considering vector generation process (step S4 in FIGS. 4 and 6). The node vector 21 generated in FIG. 10 is indicated by reference symbol D1, the weight considering vector 23 generated by the weight considering vector generation process is indicated by reference symbol D3, and a calculation example of the weight considering vector 23 is indicated by reference symbol D5.

[0089] As shown in FIG. 11, the vector generation unit 14 generates a weight-considered vector 23 (see symbol D2) (hereinafter, sometimes referred to as "vector ri") by multiplying a node vector 21 (see symbol D1) (hereinafter, sometimes referred to as "vector hi") by a weight 22.

[0090] As shown by symbols D2 and D5 in Figure 11, the "mutation type" relationship vector D22 is converted into a relationship vector D42 of [0.1, 0.065, 0.15, ...] by numerically multiplying each element by the weight 22, "0.5". Also, the "mutation type B" node D22 is converted into an element vector D43 of [0.015, 0.45, 0.05, ...] by numerically multiplying each element by the weight 22, "0.5". The number of dimensions D between vectors hi and ri remains the same.

[0091] The vector generation unit 14 converts the node vector 21 and the weight consideration vector 23 into a graph vector 24 and a weight consideration graph vector 25, respectively, as the process of step S5 in FIGS.

[0092] For example, the vector generation unit 14 may generate a graph vector 24 from the node vector 21 and a weight-considered graph vector 25 from the weight-considered vector 23 by inputting the node vector 21 and the weight-considered vector 23 into a machine learning model such as LSTM in descending order of edge weight.

[0093] In this way, the vector generation unit 14 may generate a weight-considered vector 23, and then generate a graph vector 24 and a weight-considered graph vector 25 from the node vector 21 that does not take weight into consideration and the weight-considered vector 23 that does take weight into consideration using LSTM or the like.

[0094] The graph vector 24 is an example of a first graph vector obtained from a plurality of second nodes of the weighted relationship graph 26. The weight-considered graph vector 25 is an example of a second graph vector obtained from a plurality of second nodes of the weighted relationship graph 26 and an edge having a weight 22 defined therein.

[0095] As described above, the vector generation unit 14 converts the input relationship graph 11a and generates a hidden layer taking into account the weight of the edges.

[0096] In this way, the vector generation unit 14 externally changes the numerical values of the vectors using the graph weights 22. Even when the weights 22 are variable, the same words may be treated as different words in the model of the relationship graph 11a. For example, even for the same "Mutation B," the vectors will be different when the weights 22 are "0.2" and "0.5."

[0097] For this reason, the vector generation unit 14 generates a weight-considered graph vector 25 (weight-considered vector 23) for making the model consider (reflect) the importance of the word. Also, in order to secure the original word information, the vector generation unit 14 generates a graph vector 24 (node vector 21) from which the original word can be generated, in addition to the weight-considered graph vector 25 (weight-considered vector 23).

[0098] [C-3] Word generation processing The word generation unit 15 uses a hidden layer that takes into account edge weights in parallel with a normal hidden layer, thereby performing word generation that takes into account not only the importance of words but also their weights.

[0099] FIG. 12 is a diagram for explaining an example of the word generation process by the word generation unit 15 (see symbol E; steps S6 to S12 in FIG. 4, steps S6 to S10 and S22 in FIG. 6).

[0100] The word generation unit 15 generates a graph vector (Att) 33 based on the graph vector 24 and an AW31 for a normal vector of Att (symbol E1). The word generation unit 15 also generates a weight-considered graph vector (Att) 34 based on the weight-considered graph vector 25 and an AW32 for a weight-considered vector of Att (symbol E2). The graph vector (Att) 33 is an example of a first vector obtained by inputting the graph vector 24 to a first attention mechanism of a machine learning model that performs word generation. The weight-considered graph vector (Att) 34 is an example of a second vector obtained by inputting the weight-considered graph vector 25 to a second attention mechanism of a machine learning model that performs word generation.

[0101] Here, the weight consideration vector 23 in the weight consideration graph vector 25 is denoted by ri (i is a variable), and the node vector 21 in the graph vector 24 is denoted by hj (j is a variable). If the AW32 of Att for generating the weight consideration graph vector (Att) 34 is denoted by α, and the AW31 of Att for generating the graph vector (Att) 33 is denoted by β, then α and β may be calculated by the following formulas (1) and (2). α is an example of a parameter of the second attention mechanism, and β is an example of a parameter of the first attention mechanism.

number

[0102] The importance is calculated for different vectors using the above formulas (1) and (2). The calculation of α is an example of symbol E2 in FIG. 12, and the calculation of β is an example of symbol E1 in FIG.

[0103] The word generation unit 15 calculates α as shown in the following formula (3): ti and vector r i The weight-considered graph vector (Att)34 is obtained by multiplying and adding the results for each node i. u In addition, the word generation unit 15 calculates β tj and vector h j By multiplying and adding the results for each node j, we obtain the graph vector (Att)33, c v Calculate.

number

[0104] Then, the word generation unit 15 calculates c u and c v The concatenated vector c t is calculated (reference E3 in FIG. 12).

number

[0105] The word generation unit 15 generates a combined vector c t and the previous context vector s t Based on the input vector s t ~(In the following formula (5), "s t Generate "~" on top of " (reference E4 in Figure 12). Input vector s t ~ is the combined vector c t and the previous context vector s t and are combined with weight W c is the context vector obtained by multiplying W c is the weight that takes into account the context vector. t is a context vector generated by the word generation executed since the input of the relationship graph 11a.

number

[0106] The word generation unit 15 performs a probability calculation as shown in the following formula (6) to generate the word y t (E5 in FIG. 12).

number

[0107] In the above formula (6), y t indicates the word at t (this time), and P is y1~y t-1 Context vector s for (the previous word) t The most likely predicted value of y based on t (See reference E6 in Figure 12.) t The sentence before (y1~y t-1 ) is a probability function that finds the most probable word to be the next word in W. v is the input vector s t is the weight applied to ~, and is used in word prediction to determine which word fits the context. For example, if the training data is "I have a pen", the process of the above formula (6) uses "I" in y1 to predict "am" in y2, and y to predict "a" in y3. 1~2 Use "I am".

[0108] The word generation unit 15 receives the predicted (generated) word as an input (the previous context vector s t ), and the processes of the above equations (1) to (6) are repeated (reference E7 in FIG. 12).

[0109] In the machine learning phase, the word generation unit 15 trains a machine learning model including AW31 and AW32 based on the natural text 11b (teaching data) and the result of the above formula (6).

[0110] For example, the word generation unit 15 uses the loss function L shown in the following formula (7) G Train the machine learning model to reduce

number

[0111] The word generation unit 15 uses the parameters α, β, and W c , W v Update each of the following.

[0112] In this way, the word generation unit 15 uses the graph vector 24 and the weight-considered graph vector 25 to generate the context vector s t Generate a context vector s t The word 11c is predicted based on the above, and the attention mechanisms AW31 and AW32 are trained independently. The word generation unit 15 generates a plurality of words 11c (natural sentences) by continuously generating words 11c based on the generated words 11c using the attention mechanism.

[0113] [C-4] Output processing The output unit 16 outputs output data based on the plurality of words 11c (natural sentences) generated by the word generation unit 15 in the inference phase. The output data may include, for example, the plurality of words 11c themselves. In addition, the output unit 16 outputs the parameters α, β, W trained by the word generation unit 15. c , W v At least one of the following may be included in the output data.

[0114] When outputting the output data, the output unit 16 may, for example, transmit (provide) the output data to another computer (not shown), or may store the output data in the memory unit 11 and manage it so that it can be acquired from the generating device 1 or another computer. Alternatively, when outputting the output data, the generating device 1 may output information indicating the output data to the screen of an output device such as the generating device 1, or may output the output data in various other ways.

[0115] [D] Effects of one embodiment As described above, the generation device 1 according to one embodiment generates a weighted relationship graph 26 in which the relationships between a plurality of first nodes, each of which has a weight assigned thereto, are defined as edges, based on a relationship graph 11a. The weighted relationship graph 26 defines the relationships between the plurality of first nodes as a plurality of second nodes and the weights assigned to each of the plurality of first nodes as edges. The generation device 1 then performs word generation based on graph vectors 24 acquired from the plurality of second nodes of the weighted relationship graph 26 and weight-considered graph vectors 25 acquired from the plurality of second nodes of the weighted relationship graph 26 and edges to which weights 22 are assigned.

[0116] As a result, the generating device 1 generates the graph vector 24 (node vector h j ) as well as the important part based on the weight-considered graph vector 25 (weight-considered vector r i ) can be taken into consideration when generating words. Therefore, the generation device 1 can generate natural sentences that reflect the weights 22 based on graph node data that defines the relationships between multiple nodes, each of which has a weight 22 assigned to it, as edges.

[0117] Furthermore, the generation device 1 incorporates a vector that takes into account the graph weight 22 into word generation, for example, by numerically considering the graph weight 22 in a hidden layer, the weight 22 can be reflected in the vector. This allows the generation device 1 to clearly distinguish between important nodes and unimportant nodes, thereby improving the quality of the generated natural-sentences, such as explanatory sentences.

[0118] Furthermore, the generation device 1 performs machine learning so that the attention mechanism of the word generation process by the decoder unit 3 takes into account the weighted hidden layer of the graph. As a result, even when a large-scale relationship graph 11a is input, for example, the hidden layer of information with a small graph weight 22 becomes small, thereby reducing the redundancy of the natural sentence. For example, when the generation device 1 does not take the weighted hidden layer into account, the "unimportant parts" that are incorporated into the generation of the natural sentence can be excluded from the natural sentence, or can be placed near the end of the natural sentence as unimportant information.

[0119] 13 is a diagram showing an example of the processing result of word generation (reference symbol F2) based on the relationship graph F1. In FIG. 13, reference symbol F3 indicates a natural sentence generated by a method that does not consider weights, and reference symbol F4 indicates a natural sentence generated by a method that considers weights according to one embodiment.

[0120] As shown by symbol F3, when weights are not taken into consideration, natural sentence F3 is generated that incorporates all of the information contained in relationship graph F1. For example, the larger the relationship graph F1, the more information is packed into natural sentence F3, making it difficult to focus on the important parts, and this can result in poor readability.

[0121] In contrast, when weights are taken into consideration, as shown by symbol F4, natural sentence F4 is generated by using the weights to focus on important parts. By taking weights into consideration, natural sentence F4 is arranged in a way that the order of sentences, etc., corresponds to the weights, for example, sentences with higher weights are arranged to be at the beginning of the sentence, thereby increasing readability. Furthermore, the word generation process that takes weights into consideration allows the exclusion of unimportant information, so natural sentence F4 is a concise sentence that does not include unimportant information.

[0122] [E] Other The technology according to the embodiment described above can be implemented with the following modifications and variations.

[0123] For example, the acquisition unit 12, graph conversion unit 13, vector generation unit 14, word generation unit 15, and output unit 16 included in the generation device 1 shown in FIG. 2 may be merged in any combination, or may be separated.

[0124] 2 may be a configuration (system) in which a plurality of devices cooperate with each other via a network to realize each processing function. As an example, the memory unit 11 may be a DB server, the acquisition unit 12 and the output unit 16 may be a Web server or an application server, and the graph conversion unit 13, the vector generation unit 14, and the word generation unit 15 may be an application server, etc. In this case, the DB server, the application server, and the Web server may cooperate with each other via a network to realize each processing function of the generation device 1.

[0125] Furthermore, in the embodiment, the data input to the generation device 1 is the relationship graph 11a, but the present invention is not limited to this. For example, the method according to the embodiment can be applied to a case where the input data is a weighted directed graph by converting the graph and referring to a category lookup table.

[0126] Although the embodiment has been described with reference to an example in which supervised learning is performed, the present invention is not limited to this example, and the technique according to the embodiment can also be applied to unsupervised learning or self-supervised learning.

[0127] [F] Notes The following additional notes are provided regarding the above-described embodiments.

[0128] (Appendix 1) Based on first graph node data that defines relationships between a plurality of first nodes, each of which has been assigned a weight, as edges, second graph node data is generated that defines relationships between the plurality of first nodes as a plurality of second nodes and defines the weights assigned to each of the plurality of first nodes as edges; execute word generation based on a first graph vector obtained from the plurality of second nodes of the second graph node data and a second graph vector obtained from the plurality of second nodes of the second graph node data and the edges with the weights defined; An information processing program that causes a computer to execute processing.

[0129] (Appendix 2) the second graph vector is obtained by multiplying a node vector based on a plurality of second nodes of the second graph node data by the weight; An information processing program as described in Appendix 1.

[0130] (Appendix 3) the process of performing word generation includes a process of performing word generation based on a first vector obtained by inputting the first graph vector to a first attention mechanism of a machine learning model that performs the word generation, and a second vector obtained by inputting the second graph vector to a second attention mechanism of the machine learning model. An information processing program according to appendix 1 or appendix 2.

[0131] (Appendix 4) the process of executing the word generation includes a process of executing the word generation based on a combined vector obtained by combining the first vector and the second vector and a context vector generated by the word generation executed since the input of the first graph node data. An information processing program as described in Appendix 3.

[0132] (Appendix 5) The computer, updating parameters of the first attention mechanism using first training graph vectors obtained from a plurality of second nodes of second training graph node data; updating parameters of the second attention mechanism using second graph vectors for training obtained from the plurality of second nodes and edges with defined weights of the second graph node data for training; 5. The information processing program according to claim 3 or 4, which causes a process to be executed.

[0133] (Appendix 6) Based on first graph node data that defines relationships between a plurality of first nodes, each of which has been assigned a weight, as edges, second graph node data is generated that defines relationships between the plurality of first nodes as a plurality of second nodes and defines the weights assigned to each of the plurality of first nodes as edges; execute word generation based on a first graph vector obtained from the plurality of second nodes of the second graph node data and a second graph vector obtained from the plurality of second nodes of the second graph node data and the edges with the weights defined; An information processing method in which processing is performed by a computer.

[0134] (Appendix 7) the second graph vector is obtained by multiplying a node vector based on a plurality of second nodes of the second graph node data by the weight; 1. The information processing method described in Appendix 6.

[0135] (Appendix 8) the process of performing word generation includes a process of performing word generation based on a first vector obtained by inputting the first graph vector to a first attention mechanism of a machine learning model that performs the word generation, and a second vector obtained by inputting the second graph vector to a second attention mechanism of the machine learning model. 10. The information processing method according to claim 6 or 7.

[0136] (Appendix 9) the process of executing the word generation includes a process of executing the word generation based on a combined vector obtained by combining the first vector and the second vector and a context vector generated by the word generation executed since the input of the first graph node data. 10. The information processing method according to claim 8.

[0137] (Appendix 10) The computer updating parameters of the first attention mechanism using first training graph vectors obtained from a plurality of second nodes of second training graph node data; updating parameters of the second attention mechanism using second graph vectors for training obtained from the plurality of second nodes and edges with defined weights of the second graph node data for training; 10. The information processing method according to claim 8 or 9, wherein the information processing method executes processing.

[0138] (Appendix 11) Based on first graph node data that defines relationships between a plurality of first nodes, each of which has been assigned a weight, as edges, second graph node data is generated that defines relationships between the plurality of first nodes as a plurality of second nodes and defines the weights assigned to each of the plurality of first nodes as edges; execute word generation based on a first graph vector obtained from the plurality of second nodes of the second graph node data and a second graph vector obtained from the plurality of second nodes of the second graph node data and the edges with the weights defined; An information processing device comprising a control unit.

[0139] (Appendix 12) the second graph vector is obtained by multiplying a node vector based on a plurality of second nodes of the second graph node data by the weight; 12. The information processing device according to claim 11.

[0140] (Appendix 13) In the process of executing the word generation, the control unit executes the word generation based on a first vector obtained by inputting the first graph vector to a first attention mechanism of a machine learning model that performs the word generation, and a second vector obtained by inputting the second graph vector to a second attention mechanism of the machine learning model. 13. The information processing device according to claim 11 or 12.

[0141] (Appendix 14) the control unit, in the process of executing the word generation, executes the word generation based on a combined vector obtained by combining the first vector and the second vector and a context vector generated by the word generation executed up to now after the input of the first graph node data. 14. The information processing device according to claim 13.

[0142] (Appendix 15) The control unit updating parameters of the first attention mechanism using first training graph vectors obtained from a plurality of second nodes of second training graph node data; updating parameters of the second attention mechanism using second graph vectors for training obtained from the plurality of second nodes and edges with defined weights of the second graph node data for training; 15. The information processing device according to claim 13 or 14. [Explanation of symbols]

[0143] 1 generator 11 Memory section 11a Relationship graph 11b Natural sentences 11c Words 12 Acquisition Department 13 Graph transformation section 14 Vector generation unit 15 Word Generation Unit 16 Output section 17 Control Unit 21 node vectors 22 Weight 23 Weight Consideration Vector 24 Graph Vectors 25 Weight-sensitive graph vector 26 Weighted Relationship Graphs 31st and 32nd Autumn / Winter 33 Graph Vector (Att) 34 Weight-aware graph vector (Att)< / t> < / r> < / h> < / r> < / t> < / h>

Claims

1. based on first graph node data in which relationships between a plurality of first nodes, each of which is assigned a weight, are defined as edges, second graph node data is generated in which relationships between the plurality of first nodes are defined as a plurality of second nodes and the weights assigned to each of the plurality of first nodes are defined as edges; performing word generation based on a first graph vector obtained from the plurality of second nodes of the second graph node data and a second graph vector obtained from the plurality of second nodes of the second graph node data and the edges with the weights defined; An information processing program that causes a computer to execute processing.

2. the second graph vector is obtained by multiplying a node vector based on a plurality of second nodes of the second graph node data by the weight; The information processing program according to claim 1 .

3. the process of performing word generation includes a process of performing word generation based on a first vector obtained by inputting the first graph vector to a first attention mechanism of a machine learning model that performs the word generation, and a second vector obtained by inputting the second graph vector to a second attention mechanism of the machine learning model.

3. The information processing program according to claim 1.

4. the process of executing the word generation includes a process of executing the word generation based on a combined vector obtained by combining the first vector and the second vector and a context vector generated by the word generation executed since the input of the first graph node data. The information processing program according to claim 3 .

5. The computer, updating parameters of the first attention mechanism using first training graph vectors obtained from a plurality of second nodes of second training graph node data; updating parameters of the second attention mechanism using second training graph vectors obtained from the plurality of second nodes and edges with weights defined in the second training graph node data; 5. The information processing program according to claim 3, which causes a process to be executed.

6. based on first graph node data in which relationships between a plurality of first nodes, each of which is assigned a weight, are defined as edges, second graph node data is generated in which relationships between the plurality of first nodes are defined as a plurality of second nodes and the weights assigned to each of the plurality of first nodes are defined as edges; performing word generation based on a first graph vector obtained from the plurality of second nodes of the second graph node data and a second graph vector obtained from the plurality of second nodes of the second graph node data and the edges with the weights defined; An information processing method in which processing is performed by a computer.

7. based on first graph node data in which relationships between a plurality of first nodes, each of which is assigned a weight, are defined as edges, second graph node data is generated in which relationships between the plurality of first nodes are defined as a plurality of second nodes and the weights assigned to each of the plurality of first nodes are defined as edges; performing word generation based on a first graph vector obtained from the plurality of second nodes of the second graph node data and a second graph vector obtained from the plurality of second nodes of the second graph node data and the edges with the weights defined; An information processing device comprising a control unit.

Images