Apparatus and computer-implemented method for filling a knowledge graph
The method uses language models to assess rule quality in knowledge graphs, ensuring only high-probability facts are added, addressing the incompleteness and bias in existing knowledge graph prediction methods, thereby enhancing the accuracy and completeness of the knowledge graph.
Patent Information
- Application Number
- JP2025543661
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-01-26
- Filing Date
- 2024-01-11
- Publication Date
- 2026-01-23
AI Technical Summary
Knowledge graphs are typically incomplete, leading to biased rules for predicting new facts, which complicates the process of automatically learning new facts.
A computer-implemented method and apparatus that uses pre-trained language models to determine the quality of rules for predicting additional facts in a knowledge graph, ensuring only high-probability facts are added, by querying the model with prompts that include subjects, predicates, and masks for objects, and evaluating rule quality based on predictions and existing facts.
Effectively populates the knowledge graph with highly probable facts, compensating for its incompleteness and improving the accuracy of rule-based predictions.
Smart Images
Figure 2026502690000001_ABST
Abstract
Description
[Technical Field]
[0001] Prior art The present invention relates to an apparatus and computer-implemented method for filling a knowledge graph. [Background technology]
[0002] "Ho, V.T., Stepanova, D., Gad-Elrab, M.H., Kharlamov, E., Weikum, G., "Rule learning from knowledge graphs guided by embedding models," ISWC (1). Lecture Notes in Computer Science, vol. 11136, pp. 72-90, Springer (2018)" discloses rule learning to predict facts for knowledge graphs. [Prior art documents] [Non-patent literature]
[0003] [Non-Patent Document 1] “Ho, VT, Stepanova, D., Gad-Elrab, MH, Kharlamov, E., Weikum, G., “Rule learning from knowledge graphs guided by embedding models”, ISWC (1). Lecture Notes in Computer Science, vol. 11136, pp. 72-90, Springer (2018).” Summary of the Invention [Problem to be solved by the invention]
[0004] The problem of automatically learning new facts from knowledge graphs is particularly challenging because knowledge graphs are typically incomplete and therefore the rules extracted to predict new facts may be biased. [Means for solving the problem]
[0005] Description of the Invention The computer implemented method and apparatus for filling a knowledge graph according to the independent claims fills the knowledge graph with highly probable facts.
[0006] A computer-implemented method for filling a knowledge graph includes providing a knowledge graph including facts; determining rules for predicting additional facts, depending on the facts, based on the facts; determining the additional facts predicted by the rules; providing a pre-trained language model based on a text corpus to determine a prediction for the additional facts, depending on a prompt; determining a prompt for querying the language model, depending on the facts and / or the additional facts; determining a prediction using the language model in response to the prompt; determining a quality of the rule depending on the prediction determined for the prompt and the additional facts; and, if the quality satisfies a condition, filling the knowledge graph depending on at least one fact predicted by the rule; otherwise, not filling the knowledge graph depending on the rule. The rules are used to fill the knowledge graph based on the quality of the rules. The quality is determined depending on a prediction using the language model. Therefore, the use of the rules depends on additional information from the language model. The rules are used to predict facts that are also predicted by the language model.
[0007] Advantageously, the facts comprise a subject, a predicate and an object, and determining the prompt comprises determining a prompt for the predicate, which comprises the subject, the predicate and a mask representing the object, which means that the object of the further fact predicted by the rule is confirmed by the language model.
[0008] Preferably, determining the prompt comprises determining a further predicate dependent on the predicate, and determining the further prompt comprising a subject, the further predicate, and a mask, whereby the prompt is augmented with the further prompt derived from the predicate.
[0009] Preferably, determining the further predicate includes querying the language model and / or further language models for further predicates of the type of the predicate, predicates that are more general than the further predicate, and further predicates that are more specific than the predicate. The resulting prompt expands the prompt with hyponyms or hypernyms of the predicate.
[0010] Preferably, determining the quality comprises using the rules to determine a plurality of further facts, and determining the quality of the rules in dependence on the amount of further facts predicted by the rules that are likely according to at least one of the predictions and / or the amount of further facts predicted by the rules that are not likely according to at least one of the predictions, thereby providing a measure of quality that relies on rule-based predictions and language model-based predictions for facts that are not yet present in the knowledge graph.
[0011] Preferably, determining the quality comprises using the rule to determine a plurality of facts, and determining the quality of the rule depending on the amount of facts predicted by the rule that are present in the knowledge graph and / or the amount of facts predicted by the rule that are not present in the knowledge graph, thereby providing a measure of quality that depends on the predictions of the rule relative to facts already present in the knowledge graph.
[0012] Preferably, determining the quality includes determining the quality depending on a weighted sum of the probability that the rule correctly predicts a fact in the knowledge graph and the probability that the rule correctly predicts additional facts in the knowledge graph, where the probability that the rule correctly predicts a fact in the knowledge graph is determined depending on the amount of facts predicted by the rule that are present in the knowledge graph and / or the amount of facts predicted by the rule that are not present in the knowledge graph, and the probability that the rule correctly predicts additional facts is determined depending on the amount of additional facts predicted by the rule that are correct according to at least one of the predictions and / or the amount of additional facts predicted by the rule that are incorrect according to at least one of the predictions. This means that the quality is determined depending on the ability of the rule to predict facts that are likely to already be present in the knowledge graph and the ability of the rule to predict facts that are likely not present in the knowledge graph. According to one example, it is assumed that the knowledge graph stores correct facts. A rule learned from the knowledge graph may be inappropriate. This can be detected by this hybrid rule measure. If this hybrid rule measure is too low, the rule is deemed inappropriate. This can occur when rules rarely fit the data and the facts predicted based on the language model are not actually relevant.
[0013] Preferably, the method includes providing a threshold value, and a rule satisfies the satisfied threshold if the quality of the rule is better than the threshold value, meaning that whether a rule is selected depends on its quality, and if the quality is better than the threshold value, the rule is used.
[0014] An apparatus for filling a knowledge graph includes at least one processor and at least one storage device, the at least one processor configured to execute instructions that, when executed by the at least one processor, cause the at least one processor to perform steps of the method, and the at least one storage device configured to store the instructions. The apparatus provides advantages corresponding to those of the method.
[0015] A computer program comprising computer-readable instructions that, when executed on a computer, cause the computer to perform the method, the computer program has advantages corresponding to the advantages of the method.
[0016] Further embodiments can be derived from the following description and drawings. [Brief explanation of the drawings]
[0017] [Figure 1] 1 shows a schematic diagram of an apparatus for filling a knowledge graph; [Figure 2] 1 is a flowchart with steps of a computer-implemented method for filling a knowledge graph. DETAILED DESCRIPTION OF THE INVENTION
[0018] FIG. 1 shows a schematic diagram of an apparatus 100 .
[0019] The apparatus 100 is configured to populate a knowledge graph G. The knowledge graph G includes a number of facts, each of which includes, for example, a subject, a predicate, and an object.
[0020] The knowledge graph G may, for example, contain more than 100 facts, more than 1000 facts, more than 10,000 facts, or more than 100,000 facts.
[0021] The device 100 includes at least one processor 102 and at least one storage device 104. The at least one storage device 104 is configured to store instructions. The at least one storage device 104 is configured to store a knowledge graph.
[0022] The apparatus 100 may be, for example, a computer. The instructions may be, for example, computer-readable instructions. A computer program may include computer-readable instructions.
[0023] In this example, the at least one storage device 104 is configured to store a pre-trained language model based on a text corpus to determine predictions about further facts for the knowledge graph depending on the prompt.
[0024] At least one processor 102 is configured to execute instructions.
[0025] The instructions, when executed by at least one processor 102, cause the at least one processor 102 to perform steps in a computer-implemented method for filling a knowledge graph.
[0026] FIG. 2 shows the steps in the method.
[0027] The method includes step 200 .
[0028] Step 200 includes providing a language model, which may be provided, for example, on at least one storage device 104.
[0029] In one example, a language model such as that described in "Devlin, J., Chang, M., Lee, K., Toutanova, K., "BERT: pre-training of deep bidirectional transformers for language understanding," NAACL-HLT (1), pp. 4171-4186, Association for Computational Linguistics (2019)" is provided.
[0030] The method includes step 202 .
[0031] Step 202 includes providing a knowledge graph containing the facts. The knowledge graph may be provided, for example, on at least one storage device 104.
[0032] The method includes step 204 .
[0033] Step 204 includes determining rules dependent on the facts for predicting further facts dependent on the facts in the knowledge graph.
[0034] For example, rules such as those described in "Ho, VT, Stepanova, D., Gad-Elrab, MH, Kharlamov, E., Weikum, G., "Rule learning from knowledge graphs guided by embedding models," ISWC (1). Lecture Notes in Computer Science, vol. 11136, pp. 72-90, Springer (2018)" are provided.
[0035] predicate p r and subject x r and fact p r (x r ,y r ) object y r A rule r for predicting is, for example, r:p r (x r ,y r )←p r1 (x r ,z1),···,p ri (z i ,y r ) where x is r and y r and z1,...,z i is an existing entity in the knowledge graph, and p r1 ,···,p ri is a predicate that defines an existing relationship in the knowledge graph.
[0036] A rule r can be, for example, a fact p in the knowledge graph. r1 (x r ,z1),···,p ri (z i ,y r ) occurs, then fact p r (x r ,y r ) rule r predicts exactly one fact p r1 (x r , z1).
[0037] The method may include determining a number of different rules, including the rule described above.
[0038] The method includes step 206 .
[0039] Step 206 involves determining further facts that are predicted by the rules.
[0040] The method may include determining a number of additional facts that are predicted by the rule.
[0041] For example, multiple different predicates p r and / or multiple different subjects x r and evaluate the rule r with these to find multiple different objects y r is determined.
[0042] The method includes step 208 .
[0043] Step 208 includes determining a prompt for querying the language model depending on the fact and / or the further fact.
[0044] According to one example, a prompt for a predicate p is determined. The prompt includes a subject s, a predicate p, and a mask [MASK] representing an object o, e.g., "sp [MASK]" Includes:
[0045] For example, if the predicate is hasOffice and the subject is Bosch, the prompt "BoschhasOffice[MASK]" is determined for the fact hasOffice(Bosch,Gerlingen).
[0046] According to one example, a further prompt for a further predicate p' is determined, which further prompt includes a subject s, a further predicate p' and a mask [MASK], for example: “s p' [MASK]” Includes:
[0047] In one example, the further predicate p' is determined depending on the predicate p.
[0048] For example, for a further predicate p', a further predicate p' having the type of the predicate p, a predicate p that is more general than the further predicate p', and a further predicate p' that is more specific than the predicate p, e.g. “[MASK] is type of p” “p is more general than [MASK]” “[MASK] is more specific than p” can be determined by querying the language model.
[0049] For example, for the predicate "hasOffice", the further predicate "hasHeadquarterIn" is determined. For example, if the further predicate is hasHeadquarterIn and the subject is Bosch, the prompt "BoschhasHeadquarterIn [MASK]" is determined for the fact hasHeadquarterIn(Bosch,Gerlingen).
[0050] Alternatively or in addition to using the above language model, the further predicate may be determined by querying another language model for further predicates of the type of the predicate, predicates that are more general than the further predicate, and / or further predicates that are more specific than the predicate.
[0051] For example, other language models are trained to determine the further predicate depending on a prompt for the further predicate to have a type of the predicate, a predicate that is more general than the further predicate, and / or a further predicate that is more specific than the predicate.
[0052] The other language models may be provided, for example, on at least one storage device 104 .
[0053] The method includes step 210 .
[0054] Step 210 involves determining a prediction using a language model for the prompt.
[0055] For example, the predicate p provided to the rule r r is provided as the predicate p in the prompt, and the subject x is provided in the rule r. r By providing the subject s as a prompt, the language model can be used to find the object y r A prediction about is determined.
[0056] For example, to predict an object y, several different predicates p are provided to a rule r.r and / or multiple different subjects x r For P, a set of predictions P is determined using a language model.
[0057] For example, a set of predictions P includes facts p(s,o), where predicate p and subject s are provided in a prompt "sp [MASK]" for object o, which is predicted to [MASK] by the language model for this prompt.
[0058] The method includes step 212 .
[0059] Step 212 involves determining the quality of the rule depending on the predictions determined for the prompts and further facts.
[0060] The method may include determining a quality for a rule in the plurality of rules.
[0061] Step 212 involves determining a quality threshold.
[0062] The quality of a rule r is determined, for example, depending on the amount of further facts predicted by the rule r that are relevant according to at least one of the predictions.
[0063] The quality of a rule r is determined, for example, depending on the amount of further facts predicted by the rule r that are not correct according to at least one of the predictions.
[0064] The quality of the rule r is determined in one example depending on both quantities.
[0065] According to one example, a number of facts that are assumed to already exist in the knowledge graph G are determined using a rule r.
[0066] The quality of rule r is determined depending on, for example, the amount of facts that are predicted by rule r and are assumed to exist in knowledge graph G and that actually exist in knowledge graph G.
[0067] The quality of rule r is determined, for example, depending on the amount of facts that are predicted by rule r and are assumed to exist in knowledge graph G but do not actually exist in knowledge graph G.
[0068] For example, the probability μ1(r,G) that rule r correctly predicts a fact in knowledge graph G and the probability μ2(G) that rule r correctly predicts a further fact r ,P) and the weighted sum μ(r,P)=(1-λ)×μ1(r,G)+λ×μ2(G r ,P) The quality is determined depending on the knowledge graph G and the extended knowledge graph G that includes the further facts predicted by the rule s. r is included in the probability μ2(G r ,P) depends on the set of predictions P.
[0069] For example, a set of predictions P includes facts p(s,o) and probabilities μ(G r ,P) is μ2(G r ,P)=|G r ∩P| / |G r | is determined as follows.
[0070] The weight λ may be predetermined to a fixed value λ∈[0,1]. The weight λ can be set to be closer to 1 for more incomplete knowledge graphs than for more complete knowledge graphs.
[0071] The probability that a rule will correctly predict a fact in the knowledge graph is determined, in one example, depending on the amount of facts present in the knowledge graph that are predicted by the rule.
[0072] The probability that a rule will correctly predict a fact in the knowledge graph is determined, in one example, depending on the amount of facts predicted by the rule that are not present in the knowledge graph.
[0073] The probability that the rule will correctly predict the further facts is determined in one example depending on the amount of further facts predicted by the rule that are correct according to at least one of the predictions.
[0074] The probability that the rule will correctly predict the further facts is determined in one example depending on the amount of further facts predicted by the rule that are incorrect according to at least one of the predictions.
[0075] The method includes step 214 .
[0076] Step 214 includes populating the knowledge graph depending on at least one fact predicted by the rule if the quality is satisfactory.
[0077] According to one example, a rule is satisfied if the quality of the rule is better than a threshold.
[0078] Rules are ranked, for example, according to their quality: the threshold for one rule is, for example, the quality of another rule, meaning that a rule satisfies a condition if it is ranked higher than another rule.
[0079] Step 214 involves not relying on the rule to populate the knowledge graph if this is not the case.
[0080] The knowledge graph can be successively filled with high probability facts.
[0081] For example, steps 202 through 214 are repeated, and in step 202 at least one fact predicted by a rule is provided to the knowledge graph.
[0082] For example, the knowledge graph contains knowledge about materials science. The additional facts may contain additional knowledge about materials science, e.g., molecules. The rules can predict additional facts in the form of "If... then..." rules and leverage these to make decisions about materials. The use of pre-trained language models compensates for the incompleteness of the knowledge graph.
Claims
1. 1. A computer-implemented method for filling a knowledge graph, the method comprising: providing the knowledge graph (202) including facts; determining (204) rules for predicting further facts dependent on the facts in the knowledge graph; determining (206) the further facts predicted by the rules; providing a pre-trained language model based on a text corpus to determine a prediction about the further fact in dependence on a prompt (200); determining (208) a prompt for querying the language model depending on the fact and / or the further fact; determining (210) the prediction using the language model for the prompt; determining (212) the quality of the rule depending on the prediction determined for the prompt and the further fact; If the quality satisfies a condition, relying on at least one fact predicted by the rule to populate the knowledge graph, and if not, not relying on the rule to populate the knowledge graph (214); A method comprising:
2. A fact includes a subject, a predicate, and an object, determining 208 the prompt includes determining a prompt for a predicate, the predicate including the subject, a predicate, and a mask representing the object; The method of claim 1.
3. Determining the prompt (208) includes: determining a further predicate depending on the preceding predicate; determining a further prompt including the subject, the further predicate and the mask; The method of claim 2 , comprising:
4. 4. The method of claim 3, wherein determining the further predicates comprises querying the language model and / or further language models for the further predicates having a type of a preceding predicate, preceding predicates that are more general than the further predicate, and further predicates that are more specific than the preceding predicate.
5. Determining the quality (212) includes: determining a plurality of further facts using the rules; and determining the quality of the rule depending on the amount of the further facts predicted by the rule that are correct according to at least one of the predictions and / or the amount of the further facts predicted by the rule that are incorrect according to at least one of the predictions; 5. The method of claim 1, comprising:
6. Determining the quality (212) includes: determining a plurality of facts using the rules; determining the quality of the rule depending on the amount of the facts predicted by the rule that are present in the knowledge graph and / or the amount of the facts predicted by the rule that are not present in the knowledge graph; The method of claim 5 , comprising:
7. determining (212) the quality includes determining the quality in dependence on a weighted sum of the probability that the rule correctly predicts the fact in the knowledge graph and the probability that the rule correctly predicts the further fact; the probability that the rule will correctly predict the facts in the knowledge graph is determined depending on the amount of the facts predicted by the rule that are present in the knowledge graph and / or the amount of the facts predicted by the rule that are not present in the knowledge graph; the probability that the rule will correctly predict the further fact is determined depending on the amount of the further fact predicted by the rule that is correct according to at least one of the predictions and / or the amount of the further fact predicted by the rule that is incorrect according to at least one of the predictions, The method of claim 6.
8. The method includes providing a threshold value (212); The rule satisfies the condition if the quality is better than the threshold.
8. The method according to any one of claims 1 to 7.
9. An apparatus (100) for filling a knowledge graph, comprising: The device (100) comprises: at least one processor (102); At least one storage device (104); Equipped with The at least one processor (102) is configured to execute instructions; The instructions, when executed by the at least one processor (102), cause the at least one processor (102) to perform the steps of the method of any one of claims 1 to 8; the at least one storage device (104) is configured to store the instructions; 1. An apparatus (100) comprising:
10. A computer program comprising: the computer program includes computer readable instructions; The computer readable instructions, when executed on a computer, cause the computer to perform the method of any one of claims 1 to 8. A computer program characterized by: