Learning device, learning method, and learning program

Abstract

A learning device (100) is provided with an embedding learning unit (132) for: generating embedding vectors corresponding to subject elements on the basis of subject vector representations corresponding to the subject elements and text information corresponding to the subject elements, the subject elements being each of the elements of a triplet (D1) composed of two entities and one relation included in a subject knowledge graph, which is a knowledge graph; and learning a knowledge graph embedding representation corresponding to the subject knowledge graph on the basis of the generated embedding vectors (D6).

Description

Learning device, learning method, and learning program

[0001] The present disclosure relates to a learning device, a learning method, and a learning program.

[0002] Knowledge graphs are a way to describe real-world knowledge in a computer-readable format. Knowledge expressed in knowledge graphs can be applied to a variety of purposes, including search systems and dialogue systems. A specific example is the use of knowledge graphs in product recommendation systems for customers.

[0003] Patent No. 6954049

[0004] Patent Literature 1 discloses a technique for embedding entity-relationships into a vector space by searching an external database to identify relations that are inversely equivalent to each other, and increasing the number of entity-relationship combinations based on the identified relations. However, this technique has a problem in that when entity-relationships are accompanied by auxiliary information such as text, the auxiliary information cannot be used to learn embedded representations. The present disclosure aims to improve the expressive power of knowledge graph embedding by utilizing text information corresponding to entity-relationships.

[0005] The learning device according to the present disclosure includes an embedding learning unit that, when each element of a triplet consisting of two entities and one relation included in a target knowledge graph, which is a knowledge graph, is taken as a target element, generates an embedding vector corresponding to the target element based on a target vector representation corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedded representation corresponding to the target knowledge graph based on the generated embedding vector.

[0006] According to the present disclosure, when each element of a triplet is a target element, the embedding learning unit generates an embedding vector corresponding to the target element based on the target vector representation corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedding representation based on the generated embedding vector. Here, a triplet consists of two entities and one relation included in the target knowledge graph. Therefore, according to the present disclosure, the expressive power of knowledge graph embedding can be improved by utilizing text information corresponding to the entities and relations.

[0007] FIG. 1 is a diagram showing an example of the configuration of an information processing system 90 according to the first embodiment. FIG. 2 is a diagram showing an example of the configuration of a learning device 100 according to the first embodiment. FIG. 3 is a diagram showing a specific example of a triplet D1 according to the first embodiment. FIG. 4 is a diagram showing a specific example of a triplet D1 according to the first embodiment. FIG. 5 is a diagram showing an example of the hardware configuration of the learning device 100 according to the first embodiment. A flowchart showing the operation of the learning device 100 according to the first embodiment. FIG. 6 is a diagram showing an example of the hardware configuration of the learning device 100 according to a modification of the first embodiment. FIG. 7 is a diagram showing an example of the configuration of an information processing system 91 according to the second embodiment. FIG. 8 is a diagram showing an example of the configuration of a learning device 101 according to the second embodiment. A flowchart showing the operation of the learning device 101 according to the second embodiment. FIG. 9 is a diagram showing an example of the configuration of an information processing system 92 according to the third embodiment. FIG. 10 is a diagram showing an example of the configuration of a learning device 102 according to the third embodiment. It is a diagram for explaining the processing of an acquisition unit 160 according to the third embodiment, where (a) is a diagram for explaining the processing of generating a query D9 and (b) is a diagram for explaining the processing of generating a prompt D10. A flowchart showing the operation of the learning device 102 according to the third embodiment.

[0008] In the description of the embodiments and the drawings, the same elements and corresponding elements are given the same reference numerals. The description of elements given the same reference numerals will be omitted or simplified as appropriate. Arrows in the drawings mainly indicate the flow of data or the flow of processing. Furthermore, "unit" may be read as "circuit," "step," "procedure," "process," or "circuitry" as appropriate.

[0009] First Embodiment Hereinafter, the present embodiment will be described in detail with reference to the drawings.

[0010] ***Configuration*** FIG. 1 shows an example configuration of an information processing system 90 according to the first embodiment. The information processing system 90 includes a learning device 100, a triple DB 200, a document DB 300, and a language model 400. DB is an abbreviation for database. The information processing system 90 has a function for learning knowledge graph embedded representations. Specifically, the information processing system 90 may be used for knowledge graph completion tasks such as link prediction, such as finding the author of a paper from the body of the paper. As described below, the information processing system 90 learns knowledge graph embedded representations using triples consisting of entities and relations in the knowledge graph. When learning the knowledge graph embedded representations, if there is text corresponding to an element in the triple, that text is used. The knowledge graph learned by the information processing system 90 is used, specifically, in a product or service recommendation system, a search system, a dialogue system, a question-answering system, or a decision-making support system.

[0011] 2 shows an example of the configuration of the learning device 100 according to embodiment 1. The learning device 100 includes a data receiving unit 110, an acquiring unit 120, a training unit 130, and a storage unit 140.

[0012] The data receiving unit 110 has the function of receiving a triple D1 from the triple DB 200, acquiring any element contained in the received triple D1 as element D2, and acquiring a vector representation D3 of the acquired element D2. The triple D1 is data indicating a combination of a head h and a tail t, which correspond to two entities contained in a knowledge graph, and one relation r corresponding to the two entities. In other words, the triple D1 consists of two entities and one relation contained in the target knowledge graph, which is a knowledge graph. The triple D1 is usually expressed in the format <h, r, t>. The element D2 is an element selected from the elements contained in the triple D1. In other words, the element D2 represents one of h, r, and t. Specific examples of the triple D1 and the element D2 will be described using Figures 3 and 4.

[0013] FIG. 3 shows an example of each triple D1 received by the data receiving unit 110. As described above, each triple D1 consists of a head h, a tail t, and a relation r. In the triple D1, label strings representing each element of the head h, the tail t, and the relation r are stored. As a specific example, the triple <Bob, is_born_on, 14 July 1990> represents the fact that "Bob was born on July 14, 1990." The data receiving unit 110 selects an arbitrary element from the triple D1 and stores the selected element in element D2. That is, in the above specific example, the value stored in element D2 is either "Bob," "is_born_on," or "14 July 1990."

[0014] FIG. 4 shows another example of a triple D1 received by the data receiving unit 110. The values ​​stored in the head h, tail t, and relation r included in the triple D1 are not necessarily label strings representing each element. As a specific example, each element may be converted into a format that is easy to handle internally in a computer, such as an ID, such as E0001. In such a case, the data receiving unit 110 receives, as triple D1, each ID representing the triple, along with a correspondence table between IDs and labels. At this time, the data receiving unit 110 performs a process of restoring each element of the triple to a label string by associating each ID with a label using the input correspondence table.

[0015] The vector representation D3 acquired by the data receiving unit 110 is a structural vector representation corresponding to the element D2. The vector representation D3 is expressed as a real-valued vector of any dimension representing the element D2. The data receiving unit 110 queries the storage unit 140 to determine whether a vector representation corresponding to the element D2 exists. If a vector representation corresponding to the element D2 exists, the data receiving unit 110 designates the existing vector representation as the vector representation D3. On the other hand, if a vector representation corresponding to the element D2 does not exist, the data receiving unit 110 generates a real-valued vector using any initialization method and designates the generated real-valued vector as the vector representation D3. Here, the data receiving unit 110 may generate the real-valued vector using random numbers following a uniform distribution, or may use a normal distribution instead of the uniform distribution. However, the initialization method is not an essential feature of the present disclosure. The data receiving unit 110 outputs the element D2 to the acquiring unit 120 and outputs the vector representation D3 to the training unit 130.

[0016] The acquisition unit 120 has a function of acquiring text D4 corresponding to element D2 input from the data receiving unit 110 from the document DB 300. The acquisition unit 120 may also have a function of determining whether text information corresponding to element D2 exists in the document DB 300. The document DB 300 is a database consisting of document data. The text D4 corresponding to element D2 corresponds to text information corresponding to element D2, and specifically, is text that explains element D2 or text that provides supplementary information to element D2. As a specific example, if element D2 is "The Mona Lisa" shown in FIG. 3, the text D4 corresponding to element D2 is text that explains what kind of painting the Mona Lisa is. Note that the text D4 corresponding to element D2 does not necessarily exist in the document DB 300. The text D4 corresponding to element D2 may also be text accompanying element D2. In the first embodiment, the acquisition unit 120 includes an information acquisition unit 121.

[0017] The information acquisition unit 121 acquires text D4 corresponding to the target element from the document DB 300. The target element is each element D2 of the triplet D1. As a specific example, the information acquisition unit 121 acquires text D4 corresponding to element D2 input from the data receiving unit 110 from the document DB 300. At this time, if text corresponding to element D2 does not exist in the document DB 300, a value indicating that no text exists is stored in text D4 instead of the text. The stored value may be a special character such as Null. However, the stored value is not an essential feature of the present disclosure. The information acquisition unit 121 outputs the acquired text D4 to the feature extraction unit 131.

[0018] The training unit 130 has a function of creating an embedding vector D6 corresponding to the element D2 by combining the vector representation D3 output by the data receiving unit 110 with a text vector D5 obtained by vectorizing the text D4 output by the information acquiring unit 121. In the first embodiment, the training unit 130 includes a feature extracting unit 131 and an embedding learning unit 132.

[0019] The feature extraction unit 131 generates a target text vector corresponding to the text D4 acquired by the information acquisition unit 121 based on the target language model. The target text vector corresponds to the text vector D5. The target language model corresponds to the language model 400. As a specific example, the feature extraction unit 131 has a function of extracting the text vector D5 by vectorizing the text D4 output by the information acquisition unit 121 based on the language model 400. The language model 400 may be a model using a classic model such as Bag of Words or TF-IDF (Term Frequency-Inverse Document Frequency), or may be a model using a deep neural network such as BERT (Bidirectional Encoder Representations from Transformers). As described above, the text D4 output by the information acquisition unit 121 may contain a special value indicating that no text exists. If a special value is stored in the text D4, the feature extraction unit 131 does not perform vectorization processing using the language model 400, and stores the special value indicating that no text vector exists in the text vector D5. The stored value may be a special character such as Null. However, the stored value is not an essential feature of the present disclosure. The feature extraction unit 131 outputs the text vector D5 to the embedding learning unit 132.

[0020] The embedding learning unit 132 generates an embedding vector D6 corresponding to the target element based on a target vector representation corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedding representation corresponding to the target knowledge graph based on the generated embedding vector D6. Specifically, the embedding learning unit 132 generates an embedding vector D6 based on the target vector representation and the target text vector. The target vector representation corresponds to the vector representation D3. If text information corresponding to the target element does not exist in the document DB 300, the embedding learning unit 132 uses the target vector representation as the embedding vector D6. The embedding learning unit 132 may match the number of dimensions between the target vector representation and the target text vector, and generate the embedding vector D6 by combining the target vector representation and the target text vector after matching the number of dimensions. As a specific example, the embedding learning unit 132 has a function of learning an embedding vector D6 corresponding to element D2 using the vector representation D3 output by the data receiving unit 110 and the text vector D5 output by the feature extraction unit 131 as input. The processing of the embedded learning unit 132 differs depending on whether or not a special value indicating that the text vector does not exist is stored in the text vector D5.

[0021] The processing of the embedding learning unit 132 when a text vector exists, i.e., when the value stored in the text vector D5 is other than a special value indicating that the text vector does not exist, will be described. First, the embedding learning unit 132 performs a dimension adjustment process to match the number of dimensions of the text vector D5 with the number of dimensions of the vector representation D3. As a specific example, if the number of dimensions of the vector representation D3 is 200 and the number of dimensions of the text vector D5 is 768, the embedding learning unit 132 performs an operation to increase the number of dimensions of the vector representation D3 to 768 or an operation to reduce the number of dimensions of the text vector D5 to 200. In the dimension adjustment process, the embedding learning unit 132 may use a dimension adjustment method such as principal component analysis or a neural network. Next, the embedding learning unit 132 performs a combination process, i.e., combines the vector representation D3 after the dimension adjustment with the text vector D5 to create an embedding vector D6. Here, the embedding vector D6 is a vector representation corresponding to the element D2. In the combining process, the embedding learning unit 132 may calculate the sum of each element of the vector representation D3 and the text vector D5, and use the vector formed from the calculated sums as the embedding vector D6. Alternatively, the embedding learning unit 132 may input the vector representation D3 and the text vector D5 to two neural networks, respectively, and use the output vectors as the embedding vector D6.

[0022] If the value stored in text vector D5 is a special value indicating that the text vector does not exist, the embedding learning unit 132 does not perform the dimension adjustment process and the combining process, and instead uses vector representation D3 as embedded vector D6. In other words, if text corresponding to element D2 does not exist in document DB 300, vector representation D3 corresponding to element D2 is used as embedded vector D6 corresponding to element D2.

[0023] When the embedding learning unit 132 obtains the embedding vector D6 corresponding to each of all elements (h, r, and t) included in the triplet D1, it uses the obtained embedding vector D6 to calculate a score using any known knowledge graph embedding learning algorithm. As a specific example, when TransE is used as the knowledge graph embedding learning algorithm, the embedding learning unit 132 calculates the score using [Equation 1].

[0024]

[0025] Each of h, r, and t on the right-hand side of [Equation 1] is a vector representation obtained by the embedding learning unit 132. When the embedding learning unit 132 obtains a score for the triplet D1, it calculates the error, gradient, etc. using an arbitrary knowledge graph embedding learning algorithm. The calculated error, gradient, score, and vector representation are stored in the storage unit 140. Note that the learning according to this embodiment is a process of generating an embedding vector D6 corresponding to the head h, an embedding vector D6 corresponding to the relation r, and an embedding vector D6 corresponding to the tail t, respectively, h1, r1, and t1, and adjusting the representations of h1, r1, and t1 so that the score value calculated using the generated h1, r1, and t1 approaches an optimal value.

[0026] The storage unit 140 has a function of storing the embedding vector D6 learned by the embedding learning unit 132, the embedding representation generated by the embedding learning unit 132, and various parameters of the model. The storage unit 140 is implemented by a storage device such as a read-only memory (ROM) or a hard disk. The storage unit 140 may be implemented by a single server, multiple servers distributed on a cloud, or as part of an edge storage device. Each embedding vector D6 is stored in the storage unit 140 in association with the element D2 received by the data receiving unit 110. When vector representations have been learned for all triples included in the triple DB 200, a vector representation corresponding to any element can be acquired via the storage unit 140. The acquired vector representation can be applied to key downstream tasks for knowledge graph completion, particularly link prediction and entity alignment.

[0027] 5 shows an example of the hardware configuration of learning device 100 according to this embodiment. Learning device 100 is composed of a computer. Learning device 100 may also be composed of multiple computers.

[0028] As shown in the figure, the learning device 100 is a computer that includes hardware such as a processor 11, a memory 12, an auxiliary storage device 13, an input / output IF (Interface) 14, and a communication device 15. These pieces of hardware are appropriately connected via signal lines 19.

[0029] The processor 11 is an integrated circuit (IC) that performs arithmetic processing and controls the hardware of a computer. Specific examples of the processor 11 include a central processing unit (CPU), a digital signal processor (DSP), or a graphics processing unit (GPU). The learning device 100 may include multiple processors that replace the processor 11. The multiple processors share the role of the processor 11.

[0030] The memory 12 is typically a volatile storage device, specifically a random access memory (RAM). The memory 12 is also called a primary storage device or a main memory. Data stored in the memory 12 is saved in the secondary storage device 13 as needed.

[0031] The auxiliary storage device 13 is typically a non-volatile storage device, and specific examples thereof include a ROM (Read Only Memory), an HDD (Hard Disk Drive), or a flash memory. Data stored in the auxiliary storage device 13 is loaded into the memory 12 as needed. The memory 12 and the auxiliary storage device 13 may be configured integrally.

[0032] The input / output IF 14 is a port to which an input device and an output device are connected. Specific examples of the input / output IF 14 include a USB (Universal Serial Bus) terminal. Specific examples of the input device include a keyboard and a mouse. Specific examples of the output device include a display.

[0033] The communication device 15 is a receiver and a transmitter, and is specifically a communication chip or a NIC (Network Interface Card).

[0034] Each part of the learning device 100 may use the input / output IF 14 and the communication device 15 as appropriate when communicating with other devices.

[0035] Auxiliary storage device 13 stores a learning program. The learning program is a program that causes a computer to realize the functions of each unit of learning device 100. The learning program is loaded into memory 12 and executed by processor 11. The functions of each unit of learning device 100 are realized by software.

[0036] Data used when executing the learning program and data obtained by executing the learning program are stored in a storage device as appropriate. Each part of the learning device 100 uses a storage device as appropriate. Specific examples of the storage device include at least one of the memory 12, the auxiliary storage device 13, a register in the processor 11, and a cache memory in the processor 11. Note that the terms "data" and "information" may have the same meaning. The storage device may be independent of the computer. The functions of the memory 12 and the auxiliary storage device 13 may be realized by other storage devices.

[0037] The learning program may be recorded on a computer-readable non-volatile recording medium. Specific examples of the non-volatile recording medium include an optical disk and a flash memory. The learning program may be provided as a program product.

[0038] ***Explanation of Operation*** The operating procedure of the learning device corresponds to the learning method. Also, the program that realizes the operation of the learning device corresponds to the learning program.

[0039] 6 is a flowchart showing an example of the operation of the learning device 100. The operation of the learning device 100 will be described with reference to FIG.

[0040] (Step S1) The data receiving unit 110 acquires a triple D1 from the triple DB 200 as learning data.

[0041] (Step S2) The data receiving unit 110 acquires any element (h, r, or t) included in the acquired triplet D1 as an element D2.

[0042] (Step S3) The data receiving unit 110 determines whether or not a vector expression corresponding to the element D2 exists. If the data receiving unit 110 determines that a vector expression corresponding to the element D2 exists, the process proceeds to step S4. If the data receiving unit 110 determines that a vector expression corresponding to the element D2 does not exist, the process proceeds to step S5.

[0043] (Step S4) The data receiving unit 110 reads and acquires the vector representation D3 corresponding to the element D2 from a storage medium (not shown).

[0044] (Step S5) The data receiving unit 110 generates a vector expression D3 corresponding to the element D2 using an arbitrary vector initialization method.

[0045] (Step S6) The information acquisition unit 121 acquires text D4 corresponding to element D2 from the document DB 300. If text corresponding to element D2 does not exist in the document DB 300, the information acquisition unit 121 stores a special value indicating that no corresponding text exists in the text D4.

[0046] (Step S7) The information acquisition unit 121 determines whether text corresponding to element D2 is stored in text D4. If the information acquisition unit 121 determines that text corresponding to element D2 is stored in text D4, the process proceeds to step S8. If the information acquisition unit 121 determines that text corresponding to element D2 is not stored in text D4, the process proceeds to step S11.

[0047] (Step S8) The feature extraction unit 131 vectorizes the text stored in the text D4 using the language model 400 to obtain a text vector D5.

[0048] (Step S9) The embedded learning unit 132 adjusts either the number of dimensions of the vector representation D3 corresponding to the element D2 or the number of dimensions of the text vector D5 corresponding to the element D2, thereby aligning the number of dimensions of the vector representation D3 with the number of dimensions of the text vector D5.

[0049] (Step S10) The embedding learning unit 132 generates a final embedding vector D6 corresponding to the element D2 by combining the vector representation D3 corresponding to the element D2 with the text vector D5 corresponding to the element D2.

[0050] (Step S11) The embedding learning unit 132 uses the vector representation D3 corresponding to the element D2 as the final embedding vector D6 corresponding to the element D2.

[0051] (Step S12) The embedded learning unit 132 determines whether or not an unprocessed element exists. If the embedded learning unit 132 determines that an unprocessed element exists, the process returns to step S2. If the embedded learning unit 132 determines that an unprocessed element does not exist, the process proceeds to step S13.

[0052] (Step S13) The embedding learning unit 132 updates the parameters of the knowledge graph embedding model by performing processes such as error calculation and gradient calculation using the vector representation obtained for each of the elements (h, r, and t) of the triplet D1.

[0053] (Step S14) The embedding learning unit 132 determines whether or not there are any unprocessed triplet(s). If the embedding learning unit 132 determines that there are any unprocessed triplet(s), the process returns to step S1. If the embedding learning unit 132 determines that there are no unprocessed triplet(s), the process proceeds to step S15.

[0054] (Step S15) The storage unit 140 stores the updated model and parameters.

[0055] ***Explanation of Effects of Embodiment 1*** Through the operations described above, the learning device 100 according to embodiment 1 learns embedded representations by combining structural information about entity relations with the text information corresponding to them when generating a vector representation of any element included in a triplet. Therefore, according to this embodiment, the expressive power of knowledge graph embedding can be improved, and the performance of key tasks for knowledge graph completion, such as link prediction and entity alignment, can be improved.

[0056] Furthermore, when generating a vector representation of any element included in a triplet, if there is no text explaining each element, the learning device 100 according to the first embodiment learns using only the structural vector representation of each element. Therefore, according to this embodiment, even in a knowledge graph in which auxiliary text information does not correspond to all elements, the accuracy of link prediction, entity alignment, and the like can be improved.

[0057] ***Other Configurations*** <Variation 1> Figure 7 shows an example hardware configuration of learning device 100 according to this variation. Learning device 100 includes processing circuitry 18 instead of processor 11, processor 11 and memory 12, processor 11 and auxiliary storage device 13, or processor 11, memory 12, and auxiliary storage device 13. Processing circuitry 18 is hardware that realizes at least some of the components of learning device 100. Processing circuitry 18 may be dedicated hardware, or may be a processor that executes a program stored in memory 12.

[0058] When processing circuitry 18 is dedicated hardware, processing circuitry 18 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. Learning device 100 may include multiple processing circuits that replace processing circuitry 18. The multiple processing circuits share the role of processing circuitry 18.

[0059] In the learning device 100, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0060] The processing circuitry 18 is implemented, for example, by hardware, software, firmware, or a combination of these. The processor 11, memory 12, auxiliary storage device 13, and processing circuitry 18 are collectively referred to as the "processing circuitry." In other words, the functions of each functional component of the learning device 100 are implemented by the processing circuitry. Learning devices according to other embodiments may also have a configuration similar to this modification.

[0061] Second Embodiment. The following mainly describes differences from the above-described embodiments, with reference to the drawings. In the second embodiment, the document DB 300 is removed from the configuration of the first embodiment, and a string generation unit 152 is added to the configuration of the first embodiment, normalizing the strings directly assigned to any element included in the triplet. When the total amount of text corresponding to an element is extremely small, learning that fully utilizes the text information may not be possible. The learning device 101 according to the second embodiment acquires a label directly assigned to any element included in the triplet, normalizes the acquired label, and vectorizes the normalized label using a language model. This process makes it possible to utilize label information representing the entity / relation included in the label, improve the expressiveness of knowledge graph embedding, and improve link prediction accuracy, even when the total amount of text corresponding to an element is extremely small.

[0062] ***Description of Configuration*** Fig. 8 shows an example of the configuration of an information processing system 91 according to embodiment 2. The configuration of the information processing system 91 is the same as the configuration of the information processing system 90 according to embodiment 1, except that the information processing system 91 does not have the document DB 300 according to embodiment 1.

[0063] 9 shows an example configuration of a learning device 101 according to embodiment 2. The learning device 101 includes a data receiving unit 110, an acquiring unit 150, a training unit 130, and a storage unit 140. The data receiving unit 110, the training unit 130, and the storage unit 140 are each similar to those in embodiment 1.

[0064] The acquiring unit 150 has a function of acquiring a label string D7 directly assigned to an element D2 input from the data receiving unit 110, generating a normalized label string D8 by performing a string normalization process on the acquired label string D7, and outputting the generated normalized label string D8. In the second embodiment, the acquiring unit 150 includes an information acquiring unit 151 and a string generating unit 152.

[0065] The information acquisition unit 151 acquires the label string D7 directly assigned to the element D2 input from the data receiving unit 110. The element D2 input from the data receiving unit 110 stores a string representing the element D2. That is, if the triplet input to the data receiving unit 110 is <Tokyo, is_capital_of, Japan> and the element "Tokyo" is selected, the string stored in the element D2 is "Tokyo." Therefore, the actual function of the information acquisition unit 151 is simply to assign the value of the element D2 to the label string D7.

[0066] The string generation unit 152 generates a normalized label string D8 by performing a normalization process on the target label string. The normalized label string D8 corresponds to text information corresponding to the target element. The target label string corresponds to the label string D7. As a specific example, the string generation unit 152 generates a normalized label string D8 by performing a string normalization process on the label string D7 input from the information acquisition unit 151, and outputs the generated normalized label string D8 to the feature extraction unit 131. As the string normalization process, the string generation unit 152 performs deletion of underscores and deletion of frequently occurring words such as "is" and "of" included in the label string D7. As a specific example, if the string stored in the label string D7 is "is_capital_of," the string stored in the normalized label string D8 becomes "is capital of" by deleting the underscores, and becomes "capital" by deleting the frequently occurring words. Note that this example is merely an example of normalization processing. However, the specific method of character string normalization processing is not an essential feature of the present disclosure. Note that the character string generation unit 152 does not have to perform character string normalization processing. In other words, the value of the label character string D7 may be used as the normalized label character string D8 as is.

[0067] The feature extraction unit 131 according to this embodiment generates a target text vector corresponding to the normalized label string D8 generated by the string generation unit 152 based on the target language model.

[0068] ***Explanation of Operation*** Figure 10 is a flowchart showing an example of the operation of the learning device 101. The operation of the learning device 101 will be described using Figure 10. In the process shown in Figure 10, steps S6, S7, and S11 are not executed. Also, in the process shown in Figure 10, steps S6a and S6b are executed, unlike the process shown in Figure 6. Therefore, steps S6a and S6b will be described with reference to Figure 10. The other steps shown in Figure 10 are assigned the same step numbers as those shown in Figure 6, and description of the process will be omitted.

[0069] (Step S6a) The information acquiring unit 151 acquires the label character string directly assigned to the element D2 input from the data receiving unit 110, and stores the acquired label character string in the label character string D7.

[0070] (Step S6b) The string generation unit 152 generates a normalized label string by performing a string normalization process on the label string D7 input from the information acquisition unit 151, and stores the generated normalized label string in a normalized label string D8.

[0071] ***Explanation of the Effects of Embodiment 2*** The learning device 101 according to embodiment 2 substitutes element labels for the text corresponding to the elements. Therefore, according to this embodiment, even for a knowledge graph that has no elements linked to document DB 300 or a knowledge graph with extremely few elements linked to document DB 300, the accuracy of downstream tasks such as link prediction and entity alignment can be improved.

[0072] Third Embodiment. The following mainly describes differences from the above-described embodiments, with reference to the drawings. In the third embodiment, a configuration is described in which a string generation unit 162, a search unit 163, and an explanation model 500 are added to the configuration of the first embodiment. When the total amount of text corresponding to a certain element is extremely small, learning that fully utilizes the information in the text may not be possible. Furthermore, as in the specific example described above, the label of each element is often "Tokyo" or "capital." However, the information obtained from such labels is limited. The learning device 102 according to the third embodiment acquires a label directly assigned to any element included in a triplet and executes a process of searching for text related to the acquired label, or a process of acquiring text that explains the acquired label using the explanation model 500. These processes can improve the expressiveness of knowledge graph embedding and the accuracy of downstream tasks such as link prediction and entity alignment, even when the total amount of text corresponding to a certain element in a knowledge graph is extremely small and the information obtained from the label of the certain element is limited.

[0073] 11 shows a configuration example of an information processing system 92 according to embodiment 3. The information processing system 92 includes an explanatory model 500 in addition to the components included in the information processing system 90 according to embodiment 1.

[0074] When the explanation model 500 receives input data, it outputs output data corresponding to the input data based on the received input data. The explanation model 500 is a model that outputs an explanation corresponding to input text. The explanation model 500 may be generative AI (Artificial Intelligence). As a specific example, the explanation model 500 is a model that has been machine-learned using any of supervised learning, reinforcement learning, and unsupervised learning. As a specific example, the explanation model 500 is a model obtained by executing learning according to deep learning, a genetic program, a functional logic program, or any other known algorithm or method. Specific examples of the explanatory model 500 include a Neural Network (NN) model, a Convolutional Neural Network (CNN) model, a Recurrent Neural Network (RNN), a Variational Autoencoder (VAE), Generative Adversarial Networks (GAN), a Diffusion model, a Transformer model, a Large Language Model (LLM), a Visual Language Model (VLM), a BERT, and a Generative Pre-trained Model (GPT). The explanatory model 500 may be a model called a Transformer (Language-Image Pre-training) or a CLIP (Contrastive Language-Image Pre-training). Furthermore, the explanatory model 500 may be a rule-based model that obtains an output result by referencing a predetermined table or making a judgment based on predetermined conditions. The above-mentioned models are not necessarily exclusive. Specific examples include LLM, VLM, BERT, and GPT, all of which are included in Transformer models. Specific examples include Transformer models, which are included in NN models. Each of the learning algorithms and models may be a combination of multiple types. Models also include what are called multimodal models, which are trained by combining multiple different types of data.

[0075] 12 shows an example of the configuration of a learning device 102 according to embodiment 3. The learning device 102 includes a data receiving unit 110, an acquiring unit 160, a training unit 130, and a storage unit 140. The data receiving unit 110, the training unit 130, and the storage unit 140 are each the same as those in embodiment 1.

[0076] The acquisition unit 160 has a function of acquiring a label string D7 directly assigned to an element D2 input from the data receiving unit 110, creating a query D9 for searching for text related to the acquired label string D7 or a prompt D10 for outputting text explaining the acquired label string, searching for or generating text using the created query D9 or prompt D10, and outputting the obtained text D11. In the third embodiment, the acquisition unit 160 includes an information acquisition unit 161, a string generation unit 162, and a search unit 163. The information acquisition unit 161 is the same as in the second embodiment.

[0077] The string generation unit 162 generates a query D9 used to search the document DB 300 for text D11 corresponding to a target label string included in a target element. The string generation unit 162 generates a prompt D10 used to cause the explanation model 500 to generate text D11 corresponding to a target label string included in a target element. The query D9 is used to search for text D11 related to an input label string D7. The prompt D10 is used to cause the explanation model 500 to generate text D11 that explains the input label string D7. As a specific example, the string generation unit 162 generates the query D9 or the prompt D10 when the label string D7 is input from the information acquisition unit 161.

[0078] A specific example of the processing of the character string generation unit 162 will be described with reference to Fig. 13. Fig. 13 is a conceptual diagram showing a specific example of the processing of the character string generation unit 162. First, a specific example of the processing when the character string generation unit 162 generates a query D9 will be described, and then a specific example of the processing when the character string generation unit 162 generates a prompt D10 will be described.

[0079] When generating query D9, label string D7 corresponding to element D2 is input to string generation unit 162 from information acquisition unit 161. String generation unit 162 generates query D9 for searching for text including input label string D7. The generated query D9 is queried to document DB 300 via search unit 163. Query D9 shown in FIG. 13A is an example of query D9 when searching for text including the string "tokyo" in a GET request to a Hypertext Transfer Protocol (HTTP) server when document DB 300 is implemented by the HTTP server. Note that the format of query D9 generated by string generation unit 162 is a format that corresponds to the format of document DB 300.

[0080] When generating the prompt D10, the label string D7 corresponding to the element D2 is input to the string generation unit 162 from the information acquisition unit 161. The string generation unit 162 generates the prompt D10 that generates text that explains the label string D7. The generated prompt D10 is queried to the explanatory model 500 via the search unit 163. The prompt D10 shown in FIG. 13(b) is an example of the prompt D10 when a prompt that causes the explanatory model 500 to generate text that explains the string "tokyo" is input to the generative language model when the explanatory model 500 is a generative language model configured using GPT or the like. Note that the format and wording of the prompt D10 generated by the string generation unit 162 are format and wording that correspond to the form of the explanatory model 500.

[0081] The search unit 163 acquires text information corresponding to the target element by searching the document DB 300 based on the query D9 generated by the character string generation unit 162. The search unit 163 inputs the prompt D10 generated by the character string generation unit 162 to the explanatory model 500 to acquire text information corresponding to the target element. As a specific example, the search unit 163 inputs the query D9 or prompt D10 input from the character string generation unit 162 to the document DB 300 or the explanatory model 500, and outputs the obtained response to the feature extraction unit 131 as text D11 corresponding to the element D2. First, a specific example of processing when the search unit 163 inputs the query D9 to the document DB 300 will be described. Next, a specific example of processing when the search unit 163 inputs the prompt D10 to the explanatory model 500 will be described.

[0082] The search unit 163 inputs the query D9 generated by the character string generation unit 162 to the document DB 300. As a specific example, the document DB 300 outputs one of the texts containing the label character string D7 based on the query D9. The output text is stored in the text D11. The text D11 is then input to the feature extraction unit 131.

[0083] The search unit 163 inputs the prompt D10 generated by the character string generation unit 162 to the explanation model 500. As a specific example, the explanation model 500 outputs text that explains the label character string D7 based on the prompt D10. The output text is stored in text D11. Thereafter, the text D11 is input to the feature extraction unit 131.

[0084] The feature extraction unit 131 according to this embodiment generates a target text vector corresponding to the text D11 acquired by the search unit 163 based on the target language model.

[0085] ***Explanation of Operation*** Figure 14 is a flowchart showing an example of the operation of the learning device 102. The operation of the learning device 102 will be described using Figure 14. In the process shown in Figure 14, steps S6c and S6d are executed, unlike the process shown in Figure 10. Therefore, steps S6c and S6d will be described with reference to Figure 14. The other steps shown in Figure 14 are assigned the same step numbers as those shown in Figure 10, and description of the process will be omitted.

[0086] (Step S6c) The character string generating unit 162 generates a query D9 for searching for text related to the label character string D7, or a prompt D10 for generating text that explains the label character string D7.

[0087] (Step S6d) The search unit 163 inputs the query D9 input from the character string generation unit 162 to the document DB 300, or inputs the prompt D10 input from the character string generation unit 162 to the explanatory model 500. Thereafter, the search unit 163 stores the obtained text in text D11.

[0088] ***Explanation of the Effects of Embodiment 3*** The learning device 102 according to embodiment 3 acquires text describing elements from external resources by searching the document DB 300 for text related to the label string of an element as text corresponding to the element. Therefore, according to this embodiment, it is possible to improve the accuracy of downstream tasks such as link prediction and entity alignment.

[0089] The learning device 102 according to the third embodiment generates text that explains the label string of an element as text corresponding to the element for the explanatory model 500. Therefore, according to this embodiment, text that explains an element can be flexibly used even in a knowledge graph that has no elements linked to the document DB 300 or a knowledge graph that has extremely few elements linked to the document DB 300. Furthermore, according to this embodiment, the accuracy of downstream tasks such as link prediction and entity alignment can be improved.

[0090] ***Other Embodiments*** The above-described embodiments can be freely combined, or any of the components of each embodiment can be modified, or any of the components can be omitted from each embodiment. Furthermore, the embodiments are not limited to those shown in embodiments 1 to 3, and various modifications are possible as needed. The procedures described using flowcharts, etc., can be modified as appropriate.

[0091] REFERENCE SIGNS LIST 11 Processor, 12 Memory, 13 Auxiliary storage device, 14 Input / output IF, 15 Communication device, 18 Processing circuit, 19 Signal line, 90, 91, 92 Information processing system, 100, 101, 102 Learning device, 110 Data receiving unit, 120 Acquisition unit, 121 Information acquisition unit, 130 Training unit, 131 Feature extraction unit, 132 Embedding learning unit, 140 Memory unit, 150 Acquisition unit, 151 Information acquisition unit, 152 String generation unit, 160 Acquisition unit, 161 Information acquisition unit, 162 String generation unit, 163 Search unit, 200 Triplet DB, 300 Document DB, 400 Language model, 500 Explanation model, D1 Triplet, D2 Element, D3 Vector representation, D4 Text, D5 Text vector, D6 Embedding vector, D7, D8 Label string, D9 Query, D10 Prompt, D11 Text.

Claims

1. A learning device having an embedding learning unit that, when each element of a triplet consisting of two entities and one relation contained in a target knowledge graph, which is a knowledge graph, is taken as a target element, generates an embedding vector corresponding to the target element based on a target vector representation corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedded representation corresponding to the target knowledge graph based on the generated embedding vector.

2. The learning device according to claim 1, further comprising: an information acquisition unit that acquires text information corresponding to the target element from a document database consisting of document data; and a feature extraction unit that generates a target text vector corresponding to the acquired text based on a target language model, which is a language model; and wherein the embedding learning unit generates the embedding vector based on the target vector representation and the target text vector.

3. The learning device according to claim 2, wherein the embedding learning unit sets the target vector representation as the embedding vector when text information corresponding to the target element does not exist in the document database.

4. The learning device according to claim 1, further comprising: a string generation unit that generates normalized label strings as text information corresponding to the target elements by performing a normalization process on target label strings included in the target elements; and a feature extraction unit that generates target text vectors corresponding to the generated normalized label strings based on a target language model that is a language model, wherein the embedding learning unit generates the embedding vectors based on the target vector representation and the target text vectors.

5. The learning device according to claim 1 further comprises: a string generation unit that generates a query used to search a document database made up of document data for text corresponding to a target label string included in the target element; a search unit that acquires text information corresponding to the target element by searching the document database based on the generated query; and a feature extraction unit that generates a target text vector corresponding to the acquired text based on a target language model that is a language model; and the embedding learning unit generates the embedding vector based on the target vector representation and the target text vector.

6. The learning device according to claim 1 further comprises: a string generation unit that generates a prompt used to generate text corresponding to a target label string included in the target element for an explanation model that outputs an explanation corresponding to input text; a search unit that inputs the generated prompt to the explanation model to obtain text information corresponding to the target element; and a feature extraction unit that generates a target text vector corresponding to the obtained text based on a target language model that is a language model, wherein the embedding learning unit generates the embedding vector based on the target vector representation and the target text vector.

7. A learning device described in any one of claims 2 to 6, wherein the embedding learning unit matches the number of dimensions between the target vector representation and the target text vector, and generates the embedding vector by combining the target vector representation and the target text vector after matching the number of dimensions.

8. A learning method in which a computer takes each element of a triplet consisting of two entities and one relation contained in a target knowledge graph, which is a knowledge graph, as a target element, generates an embedding vector corresponding to the target element based on a target vector representation corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedding representation corresponding to the target knowledge graph based on the generated embedding vector.

9. A learning program that causes a learning device, which is a computer, to execute an embedding learning process in which, when each element of a triplet consisting of two entities and one relation contained in a target knowledge graph, which is a knowledge graph, is taken as a target element, an embedding vector corresponding to the target element is generated based on a target vector expression corresponding to the target element and text information corresponding to the target element, and a knowledge graph embedding expression corresponding to the target knowledge graph is learned based on the generated embedding vector.

Images