Learning device, learning method, and learning program

Abstract

The learning device (100) is provided with an embedding learning unit (132) that, when each element of a triplet (D1) consisting of two entities and one relation included in a target knowledge graph, which is a knowledge graph, is taken as a target element, generates an embedding vector corresponding to the target element based on a target vector representation corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedding representation corresponding to the target knowledge graph based on the generated embedding vector (D6).

Description

[Technical field]

[0001] The present disclosure relates to a learning device, a learning method, and a learning program. [Background technology]

[0002] A knowledge graph is a way to describe real-world knowledge in a form that is easy for computers to read. Knowledge expressed by a knowledge graph can be applied to various purposes such as search systems and dialogue systems. As a specific example, a knowledge graph is used in a product recommendation system for customers. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent No. 6954049 Summary of the Invention [Problem to be solved by the invention]

[0004] Patent Document 1 discloses a technique for embedding entity-relationships in a vector space by searching an external database to identify relations that are in an inverse equivalence relationship with each other, and increasing the number of entity-relationship combinations based on the identified relations. However, this technique has a problem in that when auxiliary information such as text is attached to the entity-relationships, the auxiliary information such as text cannot be used to learn the embedded representation. This disclosure aims to improve the expressiveness of knowledge graph embedding by leveraging text information corresponding to entity-relationships. [Means for solving the problem]

[0005] The learning device according to the present disclosure includes: an embedding learning unit that, when each element of a triplet consisting of two entities and one relation included in a target knowledge graph, which is a knowledge graph, is taken as a target element, generates an embedding vector corresponding to the target element based on a target vector expression corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedding expression corresponding to the target knowledge graph based on the generated embedding vector; Equipped with. Effect of the Invention

[0006] According to the present disclosure, when each element of a triplet is a target element, an embedding learning unit generates an embedding vector corresponding to the target element based on a target vector representation corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedding representation based on the generated embedding vector. Here, a triplet consists of two entities and one relation included in the target knowledge graph. Therefore, according to the present disclosure, the expressiveness of knowledge graph embedding can be improved by utilizing text information corresponding to entity relations. [Brief description of the drawings]

[0007] [Figure 1] 1 is a diagram showing an example of the configuration of an information processing system 90 according to a first embodiment. [Diagram 2] FIG. 1 shows an example of the configuration of a learning device 100 according to a first embodiment. [Diagram 3] FIG. 2 is a diagram showing a specific example of a triplet D1 according to the first embodiment. [Figure 4] FIG. 2 is a diagram showing a specific example of a triplet D1 according to the first embodiment. [Diagram 5] FIG. 2 is a diagram showing an example of the hardware configuration of the learning device 100 according to the first embodiment. [Figure 6] 4 is a flowchart showing the operation of the learning device 100 according to the first embodiment. [Figure 7] FIG. 13 is a diagram showing an example of a hardware configuration of a learning device 100 according to a modification of the first embodiment. [Figure 8] FIG. 13 is a diagram showing an example of the configuration of an information processing system 91 according to a second embodiment. [Figure 9] FIG. 13 shows an example of the configuration of a learning device 101 according to a second embodiment. [Figure 10] 10 is a flowchart showing the operation of the learning device 101 according to the second embodiment. [Figure 11] FIG. 13 is a diagram showing an example of the configuration of an information processing system 92 according to a third embodiment. [Figure 12] FIG. 13 is a diagram showing an example of the configuration of a learning device 102 according to a third embodiment. [Figure 13] 13A and 13B are diagrams illustrating the processing of an acquisition unit 160 according to the third embodiment, where FIG. 13A illustrates the processing of generating a query D9 and FIG. 13B illustrates the processing of generating a prompt D10. [Figure 14] 11 is a flowchart showing the operation of the learning device 102 according to the third embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0008] In the description of the embodiments and the drawings, the same elements and corresponding elements are given the same reference numerals. Descriptions of elements given the same reference numerals are omitted or simplified as appropriate. Arrows in the drawings primarily indicate data flow or processing flow. In addition, "part" may be read as "circuit," "step," "procedure," "processing," or "circuitry" as appropriate.

[0009] Embodiment 1 Hereinafter, the present embodiment will be described in detail with reference to the drawings.

[0010] ***Configuration Description*** 1 shows an example of the configuration of an information processing system 90 according to embodiment 1. The information processing system 90 includes a learning device 100, a triplet DB 200, a document DB 300, and a language model 400. DB is an abbreviation for database. The information processing system 90 has a function of learning a knowledge graph embedding representation. As a specific example, the information processing system 90 may be used for a knowledge graph completion task such as link prediction, such as finding the author of a paper from the body of the paper. As described below, the information processing system 90 learns the knowledge graph embedded representation using triples each consisting of an entity and a relation in the knowledge graph. When learning the knowledge graph embedded representation, if there is text corresponding to an element in the triplet, the text is used. The knowledge graph learned by the information processing system 90 is used, for example, in a product or service recommendation system, a search system, a dialogue system, a question answering system, or a decision support system.

[0011] FIG. 2 shows an example of the configuration of the learning device 100 according to the first embodiment. The learning device 100 includes a data receiving unit 110, an acquiring unit 120, a training unit 130, and a memory unit 140.

[0012] The data receiving unit 110 has a function of receiving a triplet D1 from the triplet DB 200, acquiring an arbitrary element contained in the received triplet D1 as an element D2, and acquiring a vector representation D3 of the acquired element D2. The triple D1 is data indicating a combination of a head h and a tail t, which correspond to two entities included in the knowledge graph, and a relation r corresponding to the two entities. In other words, the triple D1 is composed of two entities and one relation included in the target knowledge graph, which is a knowledge graph. The triple D1 is<h,r,t> It is usually expressed in the form: Element D2 is an element selected from the elements included in triplet D1, i.e., element D2 represents any one of h, r, and t. A specific example of the triplet D1 and element D2 will be described with reference to FIGS.

[0013] FIG. 3 shows an example of each triplet D1 received by the data receiver 110. As described above, each triple D1 consists of a head h, a tail t, and a relation r. In the triple D1, label strings representing the elements of the head h, the tail t, and the relation r are stored. As a specific example,<Bob,is_born_on,14 July 1990> expresses the fact that "Bob was born on July 14, 1990." The data receiving unit 110 selects an arbitrary element from the triplet D1 and stores the selected element in element D2. That is, in the above specific example, the value stored in element D2 is either "Bob", "is_born_on", or "14 July 1990".

[0014] FIG. 4 shows another example of the triplet D1 received by the data receiver 110. In FIG. The values ​​stored in the head h, tail t, and relation r included in the triplet D1 are not necessarily label strings representing each element. As a specific example, each element may be converted into a format that is easy to handle inside a computer, such as an ID, such as E0001. In such a case, a correspondence table between IDs and labels is input to the data receiving unit 110 as the triplet D1, along with each ID indicating the triplet. At this time, the data receiving unit 110 executes a process of restoring each element of the triplet to a label string by associating each ID with a label using the input correspondence table.

[0015] The vector expression D3 acquired by the data receiving unit 110 is a structural vector expression, and corresponds to the vector expression corresponding to the element D2. The vector expression D3 is expressed as a real-valued vector of any dimension that represents the element D2. The data receiving unit 110 inquires of the storage unit 140 whether or not a vector expression corresponding to the element D2 exists. When a vector expression corresponding to the element D2 exists, the data receiving unit 110 sets the existing vector expression as a vector expression D3. On the other hand, when a vector expression corresponding to the element D2 does not exist, the data receiving unit 110 generates a real-valued vector by an arbitrary initialization method, and sets the generated real-valued vector as a vector expression D3. Here, the data receiving unit 110 may generate a real-valued vector by random numbers following a uniform distribution, or may use a normal distribution instead of the uniform distribution. However, the initialization method is not an essential feature of the present disclosure. The data receiving unit 110 outputs the element D2 to the obtaining unit 120 and outputs the vector representation D3 to the training unit 130.

[0016] The acquiring unit 120 has a function of acquiring text D4 corresponding to element D2 input from the data receiving unit 110 from the document DB 300. The acquiring unit 120 may have a function of determining whether or not text information corresponding to element D2 exists in the document DB 300. The document DB 300 is a database made up of document data. Text D4 corresponding to element D2 corresponds to text information corresponding to element D2, and specifically, is text that explains element D2 or text that is supplementary information to element D2. As a specific example, if element D2 is "The Mona Lisa" shown in FIG. 3, text D4 corresponding to element D2 is text that explains what kind of painting the Mona Lisa is. Note that text D4 corresponding to element D2 does not necessarily exist in document DB300. Text D4 corresponding to element D2 may be text accompanying element D2. In the first embodiment, the acquisition unit 120 includes an information acquisition unit 121 .

[0017] The information acquiring unit 121 acquires text D4 corresponding to the target element from the document DB 300. The target elements are each element D2 of the triplet D1. As a specific example, the information acquiring unit 121 acquires text D4 corresponding to element D2 input from the data receiving unit 110 from the document DB 300. At this time, if the text corresponding to element D2 does not exist in the document DB 300, a value indicating that no text exists is stored in the text D4 instead of the text. The stored value may be a special character such as Null. However, the stored value is not an essential feature of the present disclosure. The information acquisition unit 121 outputs the acquired text D4 to the feature extraction unit 131.

[0018] The training unit 130 has the function of creating an embedding vector D6 corresponding to element D2 by combining the vector representation D3 output by the data receiving unit 110 with a text vector D5 which is vectorized from the text D4 output by the information acquisition unit 121. In the first embodiment, the training unit 130 includes a feature extraction unit 131 and an embedded learning unit 132 .

[0019] The feature extraction unit 131 generates a target text vector corresponding to the text D4 acquired by the information acquisition unit 121 based on the target language model. The target text vector corresponds to the text vector D5. The target language model corresponds to the language model 400. As a specific example, the feature extraction unit 131 has a function of vectorizing the text D4 output by the information acquisition unit 121 based on the language model 400, thereby extracting a text vector D5. The language model 400 may be a model using a classic model such as Bag of Words or Term Frequency-Inverse Document Frequency (TF-IDF), or may be a model using a deep neural network such as Bidirectional Encoder Representations from Transformers (BERT). As described above, a special value indicating that no text exists may be stored in the text D4 output by the information acquisition unit 121. When a special value is stored in the text D4, the feature extraction unit 131 does not perform vectorization processing using the language model 400, and stores a special value indicating that no text vector exists in the text vector D5. The stored value may be a special character such as Null. However, the stored value is not an essential feature of the present disclosure. The feature extraction unit 131 outputs the text vector D5 to the embedded learning unit 132.

[0020] The embedding learning unit 132 generates an embedding vector D6 corresponding to the target element based on a target vector representation corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedding representation corresponding to the target knowledge graph based on the generated embedding vector D6. Specifically, the embedding learning unit 132 generates an embedding vector D6 based on a target vector representation and a target text vector. The target vector representation corresponds to the vector representation D3. When text information corresponding to the target element does not exist in the document DB300, the embedding learning unit 132 sets the target vector representation as the embedding vector D6. The embedding learning unit 132 may match the number of dimensions between the target vector representation and the target text vector, and generate the embedding vector D6 by combining the target vector representation and the target text vector after matching the number of dimensions. As a specific example, the embedding learning unit 132 has the function of receiving the vector representation D3 output by the data receiving unit 110 and the text vector D5 output by the feature extraction unit 131 as input, and learning the embedding vector D6 corresponding to the element D2. The process of embedded learning unit 132 differs depending on whether or not a special value indicating that a text vector does not exist is stored in text vector D5.

[0021] The process of embedded learning unit 132 in the case where a text vector exists, that is, the value stored in text vector D5 is other than a special value indicating that a text vector does not exist, will be described. First, the embedding learning unit 132 performs a dimension adjustment process to make the number of dimensions of the text vector D5 match the number of dimensions of the vector representation D3. As a specific example, when the number of dimensions of the vector representation D3 is 200 and the number of dimensions of the text vector D5 is 768, the embedding learning unit 132 performs an operation to increase the number of dimensions of the vector representation D3 to 768, or an operation to reduce the number of dimensions of the text vector D5 to 200. In the dimension adjustment process, the embedding learning unit 132 may use a dimension adjustment method such as principal component analysis, or may use a neural network. Next, the embedding learning unit 132 executes a combining process, that is, creates an embedding vector D6 by combining the vector representation D3 after adjusting the dimension with the text vector D5. Here, the embedding vector D6 is a vector representation corresponding to the element D2. In the combining process, the embedding learning unit 132 may calculate the sum of each element of the vector representation D3 and the text vector D5, and use the vector formed from the calculated sums as the embedding vector D6. Alternatively, the embedding learning unit 132 may input the vector representation D3 and the text vector D5 to two neural networks, respectively, and use the output vector as the embedding vector D6.

[0022] If the value stored in the text vector D5 is a special value indicating that the text vector does not exist, the embedding learning unit 132 does not execute the above-mentioned dimension adjustment process and merging process, and directly sets the vector representation D3 as the embedding vector D6. In other words, if the text corresponding to the element D2 does not exist in the document DB 300, the vector representation D3 corresponding to the element D2 is set as the embedding vector D6 corresponding to the element D2.

[0023] When the embedding learning unit 132 obtains an embedding vector D6 corresponding to each of all elements (h, r, and t) included in the triplet D1, the embedding learning unit 132 uses the obtained embedding vector D6 to calculate a score using any known knowledge graph embedding learning algorithm. As a specific example, when TransE is used as the knowledge graph embedding learning algorithm, the embedded learning unit 132 calculates the score using [Mathematical Expression 1].

[0024]

number

[0025] Each of h, r, and t on the right-hand side of [Equation 1] is a vector expression obtained by embedded learning unit 132. When the embedded learning unit 132 obtains a score for the triplet D1, it calculates an error, a gradient, and the like by an arbitrary knowledge graph embedded learning algorithm. Each of the calculated error, gradient, score, and vector representation is stored in the memory unit 140. The learning in this embodiment is a process of generating an embedding vector D6 corresponding to the head h, an embedding vector D6 corresponding to the relation r, and an embedding vector D6 corresponding to the tail t, and adjusting the representations of h1, r1, and t1 so that the score value calculated using the generated h1, r1, and t1 is close to the optimal value.

[0026] The storage unit 140 has a function of storing the embedding vector D6 learned by the embedding learning unit 132, the embedded expression generated by the embedding learning unit 132, various parameters of the model, and the like. The storage unit 140 is realized by a storage device such as a ROM (Read Only Memory) or a hard disk. The storage unit 140 may be realized by one server, or by multiple servers distributed on a cloud, or may be realized as a part of an edge storage device. Each embedding vector D6 is stored in the storage unit 140 in association with the element D2 received by the data receiving unit 110. When vector representations are learned for all triples included in the triple DB 200, a vector representation corresponding to any element can be acquired via the storage unit 140. The acquired vector representation can be applied to key downstream tasks for knowledge graph completion, particularly link prediction and entity alignment.

[0027] 5 shows an example of the hardware configuration of learning device 100 according to this embodiment. Learning device 100 is composed of a computer. Learning device 100 may be composed of multiple computers.

[0028] As shown in the figure, the learning device 100 is a computer equipped with hardware such as a processor 11, a memory 12, an auxiliary storage device 13, an input / output IF (Interface) 14, and a communication device 15. These pieces of hardware are appropriately connected via signal lines 19.

[0029] The processor 11 is an integrated circuit (IC) that performs arithmetic processing and controls the hardware of the computer. Specific examples of the processor 11 include a central processing unit (CPU), a digital signal processor (DSP), and a graphics processing unit (GPU). The learning device 100 may include a plurality of processors that replace the processor 11. The plurality of processors share the role of the processor 11.

[0030] The memory 12 is typically a volatile storage device, and a specific example is a RAM (Random Access Memory). The memory 12 is also called a primary storage device or a main memory. Data stored in the memory 12 is saved in the auxiliary storage device 13 as necessary.

[0031] The auxiliary storage device 13 is typically a non-volatile storage device, and specific examples thereof include a ROM (Read Only Memory), an HDD (Hard Disk Drive), or a flash memory. Data stored in the auxiliary storage device 13 is loaded into the memory 12 as necessary. The memory 12 and the auxiliary storage device 13 may be integrated into one unit.

[0032] The input / output IF 14 is a port to which an input device and an output device are connected. A specific example of the input / output IF 14 is a Universal Serial Bus (USB) terminal. Specific examples of the input device are a keyboard and a mouse. A specific example of the output device is a display.

[0033] The communication device 15 is a receiver and a transmitter, and is, for example, a communication chip or a network interface card (NIC).

[0034] Each part of the learning device 100 may use the input / output IF 14 and the communication device 15 as appropriate when communicating with other devices.

[0035] Auxiliary storage device 13 stores a learning program. The learning program is a program that causes a computer to realize the functions of each unit of learning device 100. The learning program is loaded into memory 12 and executed by processor 11. The functions of each unit of learning device 100 are realized by software.

[0036] Data used when executing the learning program and data obtained by executing the learning program are appropriately stored in a storage device. Each part of the learning device 100 uses a storage device as appropriate. As a specific example, the storage device is composed of at least one of the memory 12, the auxiliary storage device 13, a register in the processor 11, and a cache memory in the processor 11. Note that the terms "data" and "information" may have the same meaning. The storage device may be independent of the computer. The functions of the memory 12 and the auxiliary storage device 13 may be realized by other storage devices.

[0037] The learning program may be recorded in a computer-readable non-volatile recording medium. Specific examples of the non-volatile recording medium include an optical disk and a flash memory. The learning program may be provided as a program product.

[0038] ***Explanation of Operation*** The operating procedure of the learning device corresponds to a learning method, and the program that realizes the operation of the learning device corresponds to a learning program.

[0039] 6 is a flowchart showing an example of the operation of learning device 100. The operation of learning device 100 will be described with reference to FIG.

[0040] (Step S1) The data receiving unit 110 acquires the triplet D1 from the triplet DB 200 as learning data.

[0041] (Step S2) The data receiving unit 110 acquires any element (h, r, or t) included in the acquired triplet D1 as element D2.

[0042] (Step S3) The data receiving unit 110 determines whether or not a vector expression corresponding to the element D2 exists. If the data receiving unit 110 determines that a vector expression corresponding to the element D2 exists, the process proceeds to step S4, and if the data receiving unit 110 determines that a vector expression corresponding to the element D2 does not exist, the process proceeds to step S5.

[0043] (Step S4) The data receiving unit 110 reads and acquires the vector representation D3 corresponding to the element D2 from a storage medium (not shown).

[0044] (Step S5) The data receiving unit 110 generates a vector representation D3 corresponding to the element D2 by using an arbitrary vector initialization method.

[0045] (Step S6) The information acquisition unit 121 acquires text D4 corresponding to the element D2 from the document DB 300. If the text corresponding to the element D2 does not exist in the document DB 300, the information acquisition unit 121 stores a special value representing that the corresponding text does not exist in the text D4.

[0046] (Step S7) The information acquisition unit 121 determines whether or not text corresponding to element D2 is stored in text D4. If the information acquisition unit 121 determines that text corresponding to element D2 is stored in text D4, the process proceeds to step S8, and if the information acquisition unit 121 determines that text corresponding to element D2 is not stored in text D4, the process proceeds to step S11.

[0047] (Step S8) The feature extraction unit 131 obtains a text vector D5 by vectorizing the text stored in the text D4 using the language model 400.

[0048] (Step S9) The embedded learning unit 132 adjusts either the number of dimensions of the vector representation D3 corresponding to the element D2 or the number of dimensions of the text vector D5 corresponding to the element D2, thereby aligning the number of dimensions of the vector representation D3 with the number of dimensions of the text vector D5.

[0049] (Step S10) The embedding learning unit 132 generates a final embedding vector D6 corresponding to the element D2 by combining the vector representation D3 corresponding to the element D2 and the text vector D5 corresponding to the element D2.

[0050] (Step S11) The embedding learning unit 132 uses the vector representation D3 corresponding to the element D2 as the final embedding vector D6 corresponding to the element D2.

[0051] (Step S12) The embedded learning unit 132 determines whether or not an unprocessed element exists. If the embedded learning unit 132 determines that an unprocessed element exists, the process returns to step S2. If the embedded learning unit 132 determines that an unprocessed element does not exist, the process proceeds to step S13.

[0052] (Step S13) The embedding learning unit 132 updates the parameters of the knowledge graph embedding model by performing processes such as error calculation and gradient calculation using the vector representation obtained for each of the elements (h, r, and t) of the triplet D1.

[0053] (Step S14) The embedded learning unit 132 determines whether or not there are any unprocessed triplet(s). If the embedded learning unit 132 determines that there are any unprocessed triplet(s), the process returns to step S1. If the embedded learning unit 132 determines that there are no unprocessed triplet(s), the process proceeds to step S15.

[0054] (Step S15) The storage unit 140 stores the updated model and parameters.

[0055] ***Explanation of the Effects of the First Embodiment*** Through the above operations, the learning device 100 according to the first embodiment learns an embedding representation by combining structural information of entity relations with the corresponding text information when generating a vector representation of any element included in a triplet. Therefore, according to the present embodiment, the expressive power of knowledge graph embedding can be improved, and the performance of main tasks for knowledge graph completion, such as link prediction and entity alignment, can be improved.

[0056] Furthermore, when generating a vector representation of any element included in a triplet, if there is no text explaining each element, the learning device 100 according to the first embodiment learns using only the structural vector representation of each element. Therefore, according to this embodiment, even in a knowledge graph in which auxiliary text information does not correspond to all elements, the accuracy of link prediction, entity alignment, and the like can be improved.

[0057] ***Other configurations*** <Variation 1> FIG. 7 shows an example of the hardware configuration of the learning device 100 according to this modification. The learning device 100 includes a processing circuit 18 in place of the processor 11 , the processor 11 and memory 12 , the processor 11 and auxiliary storage device 13 , or the processor 11 , memory 12 , and auxiliary storage device 13 . The processing circuitry 18 is hardware that realizes at least a portion of each unit of the learning device 100. The processing circuitry 18 may be dedicated hardware, or may be a processor that executes programs stored in the memory 12 .

[0058] When processing circuitry 18 is dedicated hardware, processing circuitry 18 may be, for example, a single circuit, a composite circuit, a programmed processor, a parallel programmed processor, an ASIC (Application Specific Integrated Circuit), an FPGA (Field Programmable Gate Array), or a combination thereof. The learning device 100 may include a plurality of processing circuits that replace the processing circuit 18. The plurality of processing circuits share the role of the processing circuit 18.

[0059] In the learning device 100, some functions may be realized by dedicated hardware, and the remaining functions may be realized by software or firmware.

[0060] Processing circuitry 18 is illustratively implemented in hardware, software, firmware, or a combination thereof. The processor 11, memory 12, auxiliary storage device 13, and processing circuit 18 are collectively referred to as the “processing circuitry.” In other words, the functions of each of the functional components of the learning device 100 are realized by the processing circuitry. The learning devices according to the other embodiments may also have the same configuration as this modified example.

[0061] Embodiment 2 The following mainly describes the differences from the above-described embodiment with reference to the drawings. In the second embodiment, the document DB 300 is removed from the configuration of the first embodiment, and a string generation unit 152 is added to the configuration of the first embodiment, which performs normalization processing on strings directly assigned to any element included in a triplet. When the total amount of text corresponding to an element is extremely small, it may not be possible to perform learning that fully utilizes the information in the text. The learning device 101 according to the second embodiment acquires a label directly assigned to any element included in a triplet, performs a normalization process on the acquired label, and executes a process of vectorizing the normalized label using a language model. By this process, even when the total amount of text corresponding to an element is extremely small, it is possible to utilize the label information representing the entity-relation included in the label, improve the expressiveness of knowledge graph embedding, and improve the link prediction accuracy.

[0062] ***Configuration Description*** FIG. 8 shows an example of the configuration of an information processing system 91 according to the second embodiment. The configuration of the information processing system 91 is similar to the configuration of the information processing system 90 according to the first embodiment, except that the information processing system 91 does not have the document DB 300 according to the first embodiment.

[0063] FIG. 9 shows an example of the configuration of a learning device 101 according to the second embodiment. The learning device 101 includes a data receiving unit 110 , an acquiring unit 150 , a training unit 130 , and a memory unit 140 . The data receiving unit 110, the training unit 130, and the storage unit 140 are each similar to those in the first embodiment.

[0064] The acquisition unit 150 has a function of acquiring a label string D7 directly assigned to the element D2 input from the data receiving unit 110, generating a normalized label string D8 by performing a string normalization process on the acquired label string D7, and outputting the generated normalized label string D8. In the second embodiment, the acquiring unit 150 includes an information acquiring unit 151 and a character string generating unit 152 .

[0065] The information acquiring unit 151 acquires the label character string D7 that is directly assigned to the element D2 input from the data receiving unit 110. The element D2 input from the data receiving unit 110 stores a character string representing the element D2. That is, the triplet input to the data receiving unit 110 is<Tokyo,is_capital_of,Japan> When the element "Tokyo" is selected, the character string stored in element D2 is "Tokyo." Therefore, the actual function of information acquisition unit 151 is simply to assign the value of element D2 to label character string D7.

[0066] The character string generating unit 152 performs a normalization process on the target label character string to generate a normalized label character string D8. The normalized label character string D8 corresponds to text information corresponding to the target element. The target label character string corresponds to the label character string D7. As a specific example, the string generation unit 152 generates a normalized label string D8 by performing a string normalization process on the label string D7 input from the information acquisition unit 151, and outputs the generated normalized label string D8 to the feature extraction unit 131. The string generation unit 152 performs string normalization processing by deleting underscore symbols contained in the label string D7 and deleting frequently occurring words such as "is" and "of." As a specific example, when the string stored in the label string D7 is "is_capital_of," the string stored in the normalized label string D8 becomes "is capital of" by deleting the underscore symbols, and becomes "capital" by deleting the frequently occurring words. Note that this example is merely one example of normalization processing. However, the specific method of string normalization processing is not an essential feature of the present disclosure. The character string generating unit 152 may not perform character string normalization processing. That is, the value of the label character string D7 may be used as the normalized label character string D8 as it is.

[0067] The feature extraction unit 131 according to this embodiment generates a target text vector corresponding to the normalized label character string D8 generated by the character string generation unit 152, based on the target language model.

[0068] ***Explanation of Operation*** 10 is a flowchart showing an example of the operation of the learning device 101. The operation of the learning device 101 will be described with reference to FIG. In the process shown in Fig. 10, steps S6, S7, and S11 are not executed. Also, in the process shown in Fig. 10, steps S6a and S6b are executed, unlike the process shown in Fig. 6. Therefore, steps S6a and S6b will be described with reference to Fig. 10. The other steps shown in Fig. 10 are given the same step numbers as those shown in Fig. 6, and descriptions of the processes will be omitted.

[0069] (Step S6a) The information acquiring unit 151 acquires the label character string that is directly assigned to the element D2 input from the data receiving unit 110, and stores the acquired label character string in the label character string D7.

[0070] (Step S6b) The character string generating unit 152 generates a normalized label character string by performing a character string normalization process on the label character string D7 input from the information acquiring unit 151, and stores the generated normalized label character string in a normalized label character string D8.

[0071] ***Explanation of the Effects of the Second Embodiment*** The learning device 101 according to the second embodiment substitutes the label of an element as the text corresponding to the element. Therefore, according to the present embodiment, even for a knowledge graph that has no element linked to the document DB 300 or a knowledge graph that has extremely few elements linked to the document DB 300, it is possible to improve the accuracy of downstream tasks such as link prediction and entity alignment.

[0072] Embodiment 3 The following mainly describes the differences from the above-described embodiment with reference to the drawings. In the third embodiment, a configuration in which a character string generating unit 162, a search unit 163, and an explanatory model 500 are added to the configuration of the first embodiment will be described. When the total amount of text corresponding to a certain element is extremely small, it may not be possible to perform learning that fully utilizes the information in the text. Furthermore, the labels of each element are often "Tokyo" or "capital" as in the specific example mentioned above. Here, the information that can be obtained from such labels is limited. The learning device 102 according to the third embodiment executes a process of acquiring a label directly assigned to any element included in a triplet, and searching for text related to the acquired label, or acquiring text explaining the acquired label using the explanatory model 500. By performing these processes, even in a knowledge graph in which the total amount of text corresponding to a certain element is extremely small and information obtained from the label of the certain element is limited, it is possible to improve the expressiveness of knowledge graph embedding and improve the accuracy of downstream tasks such as link prediction and entity alignment.

[0073] ***Configuration Description*** FIG. 11 shows an example of the configuration of an information processing system 92 according to the third embodiment. The information processing system 92 includes an explanatory model 500 in addition to the components included in the information processing system 90 according to the first embodiment.

[0074] When the explanation model 500 receives input data, it outputs output data corresponding to the input data based on the received input data. The explanation model 500 is a model that outputs an explanation corresponding to the input text. The explanation model 500 may be a generative AI (Artificial Intelligence). The explanatory model 500 is a model that has been machine-learned by any of supervised learning, reinforcement learning, and unsupervised learning, as a specific example. The explanatory model 500 is a model that has been obtained by executing learning according to deep learning, a genetic program, a functional logic program, or other known algorithms or methods, as a specific example. In addition, the explanatory model 500 may be, as a specific example, a model called a Neural Network (NN) model, a Convolutional Neural Network (CNN) model, a Recurrent Neural Network (RNN), a Variational Autoencoder (VAE), a Generative Adversarial Networks (GAN), a Diffusion model, a Transformer model, a Large Language Model (LLM), a Visual Language Model (VLM), a BERT, a Generative Pre-trained Transformer (GPT), or a Contrastive Language-Image Pre-training (CLIP). In addition, the explanatory model 500 may be a model described in a rule base that obtains an output result by referring to a predetermined table or making a judgment based on a predetermined condition. The above-mentioned models are not necessarily exclusive. As a specific example, each of LLM, VLM, BERT, and GPT is included in the Transformer model. As a specific example, the Transformer model is included in the NN model. Each of the learning algorithms and models may be a combination of multiple types. The models also include those called multimodal models that are learned by combining multiple different types of data.

[0075] FIG. 12 shows an example of the configuration of a learning device 102 according to the third embodiment. The learning device 102 includes a data receiving unit 110 , an acquiring unit 160 , a training unit 130 , and a memory unit 140 . The data receiving unit 110, the training unit 130, and the storage unit 140 are each similar to those in the first embodiment.

[0076] The acquisition unit 160 has a function of acquiring a label string D7 directly assigned to element D2 input from the data receiving unit 110, creating a query D9 for searching for text related to the acquired label string D7 or a prompt D10 for outputting text explaining the acquired label string, searching for or generating text using the created query D9 or prompt D10, and outputting the acquired text D11. In the third embodiment, the acquiring unit 160 includes an information acquiring unit 161 , a character string generating unit 162 , and a searching unit 163 . The information acquisition unit 161 is the same as in the second embodiment.

[0077] The character string generation unit 162 generates a query D9 used to search the document DB 300 for text D11 corresponding to a target label character string included in a target element. The character string generation unit 162 generates a prompt D10 used to cause the explanation model 500 to generate text D11 corresponding to a target label character string included in a target element. The query D9 is used to search for text D11 related to an input label character string D7. The prompt D10 is used to cause the explanation model 500 to generate text D11 explaining the input label character string D7. As a specific example, when the label string D7 is input from the information acquisition unit 161, the string generation unit 162 generates a query D9 or a prompt D10.

[0078] A specific example of the process of the character string generating unit 162 will be described with reference to FIG. FIG. 13 is a conceptual diagram showing a specific example of the process of the character string generating unit 162. As shown in FIG. First, a specific example of the process in which the character string generating unit 162 generates a query D9 will be described, and then a specific example of the process in which the character string generating unit 162 generates a prompt D10 will be described.

[0079] When generating query D9, label string D7 corresponding to element D2 is input to string generation unit 162 from information acquisition unit 161. String generation unit 162 generates query D9 for searching for text including input label string D7. The generated query D9 is queried to document DB 300 via search unit 163. 13(a) is an example of a query D9 when searching for text including the character string "tokyo" in a hypertext transfer protocol (HTTP) server by a GET request when the document DB 300 is implemented by the HTTP server. The format of the query D9 generated by the character string generating unit 162 corresponds to the form of the document DB 300.

[0080] When the prompt D10 is generated, the label string D7 corresponding to the element D2 is input to the string generation unit 162 from the information acquisition unit 161. The string generation unit 162 generates the prompt D10 that generates text that explains the label string D7. The generated prompt D10 is queried to the explanatory model 500 via the search unit 163. 13(b) is an example of a prompt D10 when inputting a prompt to generate text explaining the character string "tokyo" to the generative language model when the explanatory model 500 is a generative language model configured by GPT or the like. Note that the format and wording of the prompt D10 generated by the character string generation unit 162 are in accordance with the form of the explanatory model 500.

[0081] The search unit 163 acquires text information corresponding to the target element by searching the document DB 300 based on the query D9 generated by the character string generation unit 162. The search unit 163 inputs the prompt D10 generated by the character string generation unit 162 into the explanatory model 500 to acquire text information corresponding to the target element. As a specific example, the search unit 163 inputs the query D9 or prompt D10 received from the string generation unit 162 into the document DB 300 or the explanatory model 500, and outputs the obtained response to the feature extraction unit 131 as text D11 corresponding to the element D2. First, a specific example of processing will be described in which the search unit 163 inputs a query D9 to the document DB 300. Next, a specific example will be described in which the search unit 163 inputs a prompt D10 to the explanatory model 500.

[0082] The search unit 163 inputs the query D9 generated by the character string generating unit 162 to the document DB 300. As a specific example, the document DB 300 outputs one of the texts containing the label character string D7 based on the query D9. The output text is stored in text D11. Thereafter, text D11 is input to feature extraction unit 131.

[0083] The search unit 163 inputs the prompt D10 generated by the character string generation unit 162 to the explanatory model 500. As a specific example, the explanatory model 500 outputs text that explains the label character string D7 based on the prompt D10. The output text is stored in text D11. Thereafter, text D11 is input to feature extraction unit 131.

[0084] The feature extraction unit 131 according to this embodiment generates a target text vector corresponding to the text D11 acquired by the search unit 163, based on a target language model.

[0085] ***Explanation of Operation*** 14 is a flowchart showing an example of the operation of the learning device 102. The operation of the learning device 102 will be described with reference to FIG. In the process shown in Fig. 14, steps S6c and S6d are executed, unlike the process shown in Fig. 10. Therefore, steps S6c and S6d will be described with reference to Fig. 14. The other steps shown in Fig. 14 are given the same step numbers as those shown in Fig. 10, and descriptions of the processes will be omitted.

[0086] (Step S6c) The string generation unit 162 generates a query D9 for searching for text related to the label string D7, or a prompt D10 for generating text that explains the label string D7.

[0087] (Step S6d) The search unit 163 inputs the query D 9 input from the character string generation unit 162 to the document DB 300 , or inputs the prompt D 10 input from the character string generation unit 162 to the explanatory model 500 . After that, the search unit 163 stores the obtained text in the text D11.

[0088] ***Explanation of the Effects of the Third Embodiment*** The learning device 102 according to the third embodiment acquires text explaining an element from an external resource by searching the document DB 300 for text related to the label string of the element as text corresponding to the element. Therefore, according to the present embodiment, it is possible to improve the accuracy of downstream tasks such as link prediction and entity alignment.

[0089] The learning device 102 according to the third embodiment generates text that explains the label character string of an element as text corresponding to the element for the explanatory model 500. Therefore, according to this embodiment, even in a knowledge graph that has no element linked to the document DB 300 or a knowledge graph in which the number of elements linked to the document DB 300 is extremely small, text that explains the element can be flexibly used. Furthermore, according to this embodiment, the accuracy of downstream tasks such as link prediction and entity alignment can be improved.

[0090] ***Other embodiments*** The above-described embodiments may be freely combined, or any of the components in each embodiment may be modified, or any of the components in each embodiment may be omitted. Moreover, the embodiments are not limited to those shown in the first to third embodiments, and various modifications are possible as necessary. The procedures explained using the flowcharts and the like may be modified as appropriate. [Explanation of symbols]

[0091] 11 processor, 12 memory, 13 auxiliary storage device, 14 input / output IF, 15 communication device, 18 processing circuit, 19 signal line, 90, 91, 92 information processing system, 100, 101, 102 learning device, 110 data receiving unit, 120 acquisition unit, 121 information acquisition unit, 130 training unit, 131 feature extraction unit, 132 embedding learning unit, 140 memory unit, 150 acquisition unit, 151 information acquisition unit, 152 string generation unit, 160 acquisition unit, 161 information acquisition unit, 162 string generation unit, 163 search unit, 200 triple DB, 300 document DB, 400 language model, 500 explanation model, D1 triple, D2 element, D3 vector representation, D4 text, D5 text vector, D6 embedding vector, D7, D8 Label string, D9 query, D10 prompt, D11 text.

Claims

1. an embedding learning unit that, when each element of a triplet consisting of two entities and one relation included in a target knowledge graph, which is a knowledge graph, is taken as a target element, generates an embedding vector corresponding to the target element based on a target vector expression corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedding expression corresponding to the target knowledge graph based on the generated embedding vector; A learning device comprising:

2. The learning device further an information acquisition unit that acquires text information corresponding to the target element from a document database that includes document data; a feature extraction unit that generates a target text vector corresponding to the acquired text based on a target language model that is a language model; Equipped with The learning device according to claim 1 , wherein the embedding learning unit generates the embedding vector based on the target vector representation and the target text vector.

3. The learning device according to claim 2 , wherein the embedding learning unit sets the target vector representation as the embedding vector when text information corresponding to the target element does not exist in the document database.

4. The learning device further a character string generation unit that performs a normalization process on a target label character string included in the target element to generate a normalized label character string as text information corresponding to the target element; a feature extraction unit that generates a target text vector corresponding to the generated normalized label string based on a target language model that is a language model; Equipped with The learning device according to claim 1 , wherein the embedding learning unit generates the embedding vector based on the target vector representation and the target text vector.

5. The learning device further a character string generation unit that generates a query used to search a document database including document data for text corresponding to a target label character string included in the target element; a search unit that searches the document database based on the generated query to obtain text information corresponding to the target element; a feature extraction unit that generates a target text vector corresponding to the acquired text based on a target language model that is a language model; Equipped with The learning device according to claim 1 , wherein the embedding learning unit generates the embedding vector based on the target vector representation and the target text vector.

6. The learning device further a character string generation unit that generates a prompt used to generate text corresponding to a target label character string included in the target element for an explanation model that outputs an explanation corresponding to the input text; a search unit that inputs the generated prompt into the explanation model to obtain text information corresponding to the target element; a feature extraction unit that generates a target text vector corresponding to the acquired text based on a target language model that is a language model; Equipped with The learning device according to claim 1 , wherein the embedding learning unit generates the embedding vector based on the target vector representation and the target text vector.

7. 4. The learning device according to claim 3, wherein, when text information corresponding to the target element exists in the document database, the embedding learning unit matches the number of dimensions between the target vector representation and the target text vector, and generates the embedded vector by combining the target vector representation and the target text vector after matching the number of dimensions.

8. A learning method in which, when a computer takes each element of a triplet consisting of two entities and one relation contained in a target knowledge graph, which is a knowledge graph, as a target element, the computer generates an embedding vector corresponding to the target element based on a target vector representation corresponding to the target element and text information corresponding to the target element, and learns a knowledge graph embedded representation corresponding to the target knowledge graph based on the generated embedding vector.

9. When each element of a triplet consisting of two entities and one relation included in a target knowledge graph, which is a knowledge graph, is taken as a target element, an embedding vector corresponding to the target element is generated based on a target vector expression corresponding to the target element and text information corresponding to the target element, and a knowledge graph embedding expression corresponding to the target knowledge graph is learned based on the generated embedding vector. A learning program that executes the above on a learning device, which is a computer.

Images