Apparatus and computer-implemented method for automatically generating negative samples for training a knowledge graph embedded model.

Abstract

To automatically generate negative samples for training a knowledge graph embedding model.SOLUTION: A method comprises: providing at least one first triple 212 which is a true triple of a knowledge graph 100 and at least one second triple 214; training a knowledge graph embedding model 208 to predict triples of the knowledge graph depending on a set of triples 216 comprising the first triple and the second triple; determining vector representations 218 of entities and relations with the knowledge graph embedding model; determining a plurality of triples 228 with the vector representations of entities and relations; providing an ontology 222 comprising constraints that characterize correct triples; determining with the ontology at least one triple 224 in the plurality of triples that violates at least one constraint of the constraints or that violates a combination of at least some of the constraints.SELECTED DRAWING: Figure 2

Description

Technical Field

[0001] Background The present invention relates to an apparatus and method for automatically generating negative samples for training a knowledge graph embedding model.

Background Art

[0002] An embedding method for embedding a knowledge graph KG includes embedding entities and relations of the knowledge graph KG into a low-dimensional vector space while maintaining the structure of the knowledge graph KG.

[0003] Such methods are disclosed in 'Wang, Q., Mao, Z., Wang, B., Guo, L., "Knowledge graph embedding: A survey of approaches and applications", IEEE Trans. Knowl. Data Eng. 29 (12), 2724-2743 (2017)'.

[0004] These methods have been found to be useful for various KG curation tasks such as KG completion that expand the knowledge graph by predicting missing (subject, predicate, object) triples. Typically, the training of a KG embedding model aims to distinguish between correct (positive) triples and incorrect (negative) triples. However, since the knowledge graph KG follows an open-world assumption where missing triples are treated as unknown rather than false, the generation of negative triples is difficult.

Prior Art Documents

Non-Patent Documents

[0005]

Non-Patent Document 1

[0006] Disclosure of the invention Negative samples are essential for building many knowledge graph embedding models, and ensuring their high quality is critically important. Specifically, it is crucial that negative samples consider domain knowledge and guide the embedding model toward predicting facts consistent with available knowledge, i.e., the knowledge graph and ontology. [Means for solving the problem]

[0007] A particularly computer-implemented method for automatically generating negative samples for training a knowledge graph embedding model includes: providing at least one first triple that is a true triple of the knowledge graph; providing at least one second triple; training the knowledge graph embedding model to predict triples of the knowledge graph based on a set of triples including at least one first triple and at least one second triple; determining vector representations of entities and relations by the knowledge graph embedding model; determining a plurality of triples by the vector representations of entities and relations; providing an ontology that includes constraints characterizing the correct triples; and determining at least one triple of the plurality of triples by the ontology that violates at least one of the constraints or at least a combination of the constraints. One or more such triples are used as negative samples for training the embedding model. The negative samples can be used to improve the accuracy of the knowledge graph embedding model with respect to predicting missing relations between entities in the knowledge graph.

[0008] Determining at least one triple may involve selecting several triples that are more likely to be facts in the knowledge graph than other triples among a group of triples. In this way, facts that are mistakenly identified by the machine learning system as possible facts are identified as negative samples. These negative samples can be used to further improve training.

[0009] The method may include determining the likelihood that at least one of several triples is a fact in the knowledge graph, particularly using a knowledge graph embedding model. In this way, possible facts can be ranked according to their likelihood.

[0010] Determining at least one triple may include providing a knowledge graph fact from the knowledge graph that includes a first entity and a reference relation or representation thereof that is a reference type; determining a triple among several triples that include the first entity and the relation; determining whether the relation is of an acceptable type according to the constraint; and, if the type is not acceptable, determining that the triple violates the constraint.

[0011] Determining at least one triple may involve determining a set of triples from which the triple that violates the constraint is located, and selecting at least one triple from the set of triples that is different from the triple in that set. Thus, the newly created negative sample is different from any negative sample already available.

[0012] To automatically train a knowledge graph embedding model, the method may further include, in a first iteration, determining at least one triple, adding at least one triple to the set of triples for a second iteration, in a second iteration, training the knowledge graph embedding model with the set of triples for the second iteration, and / or, in a second iteration, determining at least one triple with the set of triples for the second iteration. The quality of the generated negative samples is iteratively improved, for example, by starting with a standard random sampling of negative samples, training the knowledge graph embedding model with respect to these, and then utilizing the converging predictions of the knowledge graph embedding model to select negative samples, i.e., at least one triple, for the next iteration of the method. Thus, the model is iteratively improved.

[0013] A device for automatically generating negative samples to train a knowledge graph embedding model comprises: a storage device configured to provide an ontology containing constraints characterizing the knowledge graph and / or true triples; a machine learning system configured to train a knowledge graph embedding model to predict the triples of the knowledge graph based on a set of triples including at least one first triple that is a true triple of the knowledge graph, at least one second triple, and to determine vector representations of entities and relations by the knowledge graph embedding model; and a generator configured to determine a plurality of triples by the vector representations of entities and relations, wherein the generator is configured to determine, by the ontology, at least one of a plurality of triples that violates at least one of the constraints or violates at least some combination of the constraints. The device improves the training system with negative samples derived based on the knowledge graph and the ontology associated with the knowledge graph.

[0014] The generator can be configured to select some triples that have a higher probability of being facts in the knowledge graph than other triples. Thus, the generator finds negative samples that the knowledge graph embedding model would mistakenly consider to be facts in the knowledge graph.

[0015] The device may include a machine learning system configured, particularly a knowledge graph embedding model, to determine the likelihood that at least one of several triples is a fact in the knowledge graph. The likelihood allows the triples to be automatically ranked for processing by the generator.

[0016] The device includes storage configured to provide knowledge graph facts from a knowledge graph, where the knowledge graph facts include a first entity and a reference relation or its representation, which is of a reference type. The generator is configured to determine a triple out of several triples, including the first entity and the relation, to determine whether the relation is of a type that is acceptable according to constraints, and if the type is not acceptable, to determine that the triple violates the constraints. Thus, reference relations are automatically generated, and triples are evaluated according to the type of reference.

[0017] To determine at least one triple, the generator may be configured to determine a set of triples from a plurality of triples that contains a triple that violates the constraint, and to select at least one triple from the plurality of triples that is different from a triple in that set. Thus, duplication of negative samples is avoided.

[0018] To automatically train a knowledge graph embedding model, the machine learning system may be further configured to determine at least one triple in a first iteration, add at least one triple to the set of triples for a second iteration, train the knowledge graph embedding model with the set of triples for the second iteration in a second iteration, and / or determine at least one triple with the set of triples for the second iteration in a second iteration. Thus, the knowledge graph embedding model is automatically generated.

[0019] A computer program may contain instructions that cause the computer to perform steps of a method when it is executed by the computer. It is preferable that a non-temporary computer-readable storage medium stores the computer program.

[0020] Further advantageous embodiments can be derived from the attached description and drawings. [Brief explanation of the drawing]

[0021] [Figure 1] It is a diagram schematically showing a knowledge graph. [Figure 2] It is a diagram schematically showing a device. [Figure 3] It is a diagram schematically showing a method.

Embodiments for Carrying out the Invention

[0022] The following description relates to the generation of negative triples that can use the ontology provided in the knowledge graph KG.

[0023] An ontology is a conceptualization of a target domain and describes general formalized constraints to be held in the knowledge graph KG.

[0024] An ontology is used in the generation of negative samples, that is, in the generation of negative triples.

[0025] An exemplary iterative method for training an embedding model proceeds as follows.

[0026] First, the embedding model is trained using an existing policy for generating negative samples. Then, the trained embedding model is used for triple prediction, and for this triple, further, the consistency with the existing data in the knowledge graph KG and the ontology is checked. Predicted triples considered to be true or correct can be used as facts. Triples in the knowledge graph KG considered to be true or correct are referred to as true triples, correct triples or facts.

[0027] Predicted triples can be taken out one by one and added to the knowledge graph KG, and ontological inferences can be applied to detect whether the added triples cause inconsistencies. If an inconsistency is caused, the predicted triple is added as a negative sample for the next iteration of embedding training.

[0028] Following the procedure described, triples predicted by the embedding model in the first iteration that resulted in inconsistencies when added to the knowledge graph KG and its ontology are stored as negative samples for the next round of embedding training.

[0029] This process can be repeated over several iterations until the model is successfully trained and predicts only consistent facts.

[0030] Domain knowledge formalized in the form of ontology is used to automatically detect predictions that can be used as negative samples.

[0031] Inconsistencies in the knowledge graph (KG) and ontology are determined, for example, as described in "Fast computation of explanations for inconsistency in large-scale knowledge graphs" by Tran, T., Gad-Elrab, MH, Stepanova, D., Kharlamov, E., and Stroetgen, J., published in WWW'20: The Web Conference 2020, Taipei, Taiwan, April 20-24, 2020, pp.2613-2619 (2020).

[0032] A knowledge graph (KG) represents a set of interconnected factual information. A knowledge graph (KG) can be encoded as a set of (subject;predicate;object) triples, such as (john;worksAt;bosch). The subject or object of such a triple is called an entity, and the predicate is called a relation. A set of triples in a knowledge graph (KG) can be represented as a directed graph with labeled vertices and edges. Triples in a knowledge graph (KG) are called facts. Facts in a knowledge graph (KG) can be represented as unary or binary ground predicates, such as man(john) or worksAt(john;bosch).

[0033] Knowledge Graph Embedding (KGE) concerns embedding the entities and relations of a knowledge graph (KG) into a continuous vector space of a user-specified dimension n. More specifically, the KGE model takes a set of KG triples as input and aims to map the entities and relations into an n-dimensional vector space such that several features reflecting the KG structure are preserved. These features are captured by the objective function of the individual embedding model. In this way, a set of numerical vectors is obtained from relational data.

[0034] An ontology is a conceptualization of a subject domain, expressed as a set of constraints, including, for example, axioms and / or statements containing axioms. The ontology reflects the schema that the knowledge graph (KG) should follow. For example,

number

[0035] A triple violates a constraint or combination of constraints if its union with a constraint or combination of constraints is inconsistent.

[0036] for example,<bosch gmbh,type,Person> In the case of the constraint "Company and Person are relatively prime",<bosch gmbh,type,Company> Assuming that is a true triple in the knowledge graph,<bosch gmbh,type,Person> This violates the constraint.

[0037] According to the first axiom, it is people who work somewhere. The second axiom shows that if the first entity has a locatedIn relation with the second entity, then the second entity must be of type location. Finally, according to the last axiom, location and person are relatively prime.

[0038] If a model does not exist for the knowledge graph KG and the ontology, in other words, if there is a contradiction in the knowledge graph KG with the terminology of the ontology, for example, a contradiction with an axiom or statement, then the knowledge graph KG is inconsistent with the ontology. For example, the above ontology O is inconsistent with the fact (john,type,person);(bosch,locatedIn,john). In fact, it is known that john is a person, and location is prohibited by the second axiom of the ontology and also by the last axiom of the ontology.

[0039] Inconsistency checking is the process of detecting inconsistencies between the knowledge graph (KG) and the ontology.

[0040] Figure 1 schematically shows Knowledge Graph 100. Knowledge Graph 100 contains multiple entities and multiple relations. The following facts of the Knowledge Graph are available from Knowledge Graph 100. The facts of the Knowledge Graph in the example are defined by triples (X, Y, Z) where X is the subject entity, Y is the relation, and Z is the object entity. That is, (102,120,104) (110,122,104) (106,124,102) (106,128,108) (110,126,106) (110,130,108) (112,132,114) That is the case.

[0041] An entity can be a type from a set of entity types. In this example, this set of entity types is represented by at least one entity in Knowledge Graph 100. In this example, entity 102 is the first entity type, entities 104 and 112 are the second entity types, entities 106 and 110 are the third entity types, and entity 108 is the fourth entity type. The first entity type in this example is "company name". The second entity type in this example is "country". The third entity type in this example is "first name". The fourth entity type in this example is "type". Other entity types may exist. An ontology may contain entity types that exist in Knowledge Graph KG. An ontology may contain entity types that are different from the entity types of Knowledge Graph KG.

[0042] The label for a relation may be selected from a set of relation labels. This set of relation labels includes, in the example, the labels type, worksAt, friendOf, livesIn, and locatedIn. In the example, the label "type" is used to link the subject entity to the object entity, which is the fourth entity type. Relations 128, 130, and 132 are labeled "type" in the example.

[0043] Relation 120 is labeled "locatedIn" in the example. Relation 122 is labeled "livesIn" in the example. Relation 124 is labeled "worksAt" in the example. Relation 126 is labeled "friendOf" in the example. Other relation types or labels may be used. The ontology may include relation types or their labels that exist in the Knowledge Graph KG. The ontology may also include relation types or their labels that are different from those in the Knowledge Graph KG.

[0044] The ontology of Knowledge Graph 100 may define constraints that characterize a correct triple. In one example, a negative sample violates a constraint. The ontology may define multiple constraints that characterize a correct triple. In one example, a negative sample violates one of the multiple constraints, multiple of the multiple constraints, or all of the multiple constraints. In one example, a negative sample violates at least some combination of the constraints that characterize a correct triple. A correct triple, in this example, is a triple that is acceptable in Knowledge Graph 100. In this example, a negative sample is a sample that is an unacceptable triple in Knowledge Graph 100. This means that the ontology defines at least one constraint that a negative sample does not satisfy.

[0045] This means that the ontology includes constraints that specify which triples are allowed and which are not. A triple that violates one or more constraints is an invalid triple.

[0046] For example, an ontology defines that a subject entity linked to an object entity by relation "type" must not be linked to any other object entity by relation "type".

[0047] For example, a subject entity linked to the object entity of type "company" by relation "type" should not be linked to the object entity "first name" by relation "locatedIn".

[0048] For example, triple<bosch,locatedIn,thomas> This would violate the constraint that the target or object of the “locatedIn” relation must be Location. This is because “thomas” is Person, as specified by other correct triples in the knowledge graph.

[0049] In the example, the relation 134 "locatedIn" from entity 112 to entity 110 is incorrect according to the ontology. This means that the triple (112, 134, 110) containing relation 134 is a negative sample.

[0050] Figure 2 schematically shows the apparatus 200.

[0051] The device 200 comprises a machine learning system 202, a generator 204, and a storage device 206.

[0052] The device 200 is configured to automatically generate negative samples for training the knowledge graph embedded model 208 based on the knowledge graph 100. The device 200 may be configured to automatically train the knowledge graph embedded model 208.

[0053] In the following description, facts or facts in a knowledge graph refer to a triple or, in particular, a numerical representation thereof of two entities and a relationship.

[0054] The storage 206 is configured to provide at least one first triple 212 of the knowledge graph 100. At least one first triple 212 is stored in and / or derived from the knowledge graph 100. In the example, multiple first triples 212 are determined.

[0055] The generator 204 is configured to provide at least one second triple 214.

[0056] The machine learning system 202 is configured to train a knowledge graph embedding model 208 to predict facts in the knowledge graph 100 in accordance with a set of triples 216, which includes at least one first triple 212 and at least one second triple 214.

[0057] The machine learning system 202 is configured to determine multiple vector representations 218 of entities and relationships using a knowledge graph embedding model 208.

[0058] The apparatus 200 includes a storage device 220 configured to provide an ontology 222 that includes several constraints characterizing negative samples.

[0059] The generator 204 may be configured to determine at least one triple 224 that violates at least one constraint that characterizes a correct triple.

[0060] The generator 204 may be configured for at least one triple 224 that violates at least some combination of constraints that characterize a correct triple.

[0061] The example machine learning system 202 is configured to determine at least one triple 224 by a set of facts 216 in a first iteration. The example machine learning system 202 is configured to add at least one triple 224 to the set of facts 216 for a second iteration.

[0062] The example machine learning system 202 is configured to train the knowledge graph embedding model 208 in the second iteration.

[0063] The example machine learning system 202 is configured to determine at least one triple 224 in a second iteration by a set of facts 216 for the second iteration.

[0064] The generator 204 may be configured by the selector 226 to select several triples 228 obtained from the vector representations 218 of entities and relations that have a higher likelihood of being facts in the knowledge graph 100 than other triples obtained from the vector representations 218 of entities and relations.

[0065] The machine learning system 202 may be configured, in particular by a knowledge graph embedding model 208, to determine the likelihood that at least one triple in multiple vector representations 218 of entities and relations is a fact in the knowledge graph 100.

[0066] The generator 204 may be configured by the inference unit 230 to determine at least one third triple 224 from several triples 228. For example, the inference unit 230 is configured to process ontology terms to select at least one third triple 224 from several triples.

[0067] To determine at least one second triple 214, the generator 204 may be configured to sample two entities of at least one second triple 214 from the entities, and / or to sample the relationship between the two entities from the relationships in the knowledge graph 100.

[0068] To determine at least one second triple 214, the generator 204 may be configured to sample representations of two entities of at least one second triple 214 from the representations of entities in the knowledge graph 100, and / or to sample the relationship between the two entities from the representations of relationships in the knowledge graph 100.

[0069] The device 200 may be configured to identify negative samples using the knowledge graph 100 and the ontology 222.

[0070] Storage 206 may, in one example, be configured to provide knowledge graph facts from knowledge graph 100. These knowledge graph facts include a first entity and a reference relation or its representation. The reference relation is a reference type. Generator 204 may be configured to determine a triple out of several triples, which include the first entity and the relation. Generator 204 may be configured to determine whether the relation is of an acceptable type according to constraints. If the type is not acceptable, generator 204 may be configured to determine that the triple violates the constraints or at least some combination of constraints. This means that the triple includes the same first entity as the knowledge graph fact. This means that the triple includes a relation type that is incompatible with the relation type of the reference relation according to the ontology. Therefore, the triple is a negative sample.

[0071] A method for automatically generating negative samples for training the knowledge graph embedded model 208, particularly a computer-implemented method, is described below with reference to Figure 3. To train iteratively with negative samples, the method includes a step as an optional means.

[0072] In this method, step 302 is performed.

[0073] In step 302, at least one first triple 212 of the knowledge graph 100 is provided. In the example, multiple first triples 212 are provided.

[0074] Then, step 304 is executed.

[0075] In step 304, at least one second triple 214 is provided.

[0076] At least one second triple 214 may be determined in the first step from the knowledge graph 100.

[0077] In one example, two entities are sampled particularly randomly from the entities, and a relation is sampled from the relation in the knowledge graph. At least one second triple 214 in this example includes the first entity and relation and the second entity.

[0078] Then, step 306 is executed.

[0079] In step 306, the knowledge graph embedded model 208 is trained to predict facts in the knowledge graph 100 in accordance with a set of triples 216, which includes at least one first triple 212 and at least one second triple 214.

[0080] The knowledge graph embedding model 208 may be configured to linearly map subject entity embeddings, e.g., vectors, to other embeddings in the vector space by relation embeddings, e.g., vectors or matrices. For an existing knowledge graph triple, the subject entity embedding that matches the matching function represents the predicted object entity. The matching function provides a means of determining whether the triple is true or false. The matching function may include, for example, vector addition or vector multiplication. The training goal may be to match the subject entity embedding to a predicted object entity embedding that is as close as possible to the object entity embedding from the knowledge graph triple by relation embedding. The corresponding loss function may include a distance metric relating the distance between the predicted object and the object of the fact in the knowledge graph. The method is not limited to this type of training.

[0081] Then, step 308 is performed.

[0082] In step 308, multiple vector representations 218 of entities and relations are determined by the knowledge graph embedding model 208, particularly depending on the set of triples 216.

[0083] Then, step 310 is executed.

[0084] In step 310, an ontology is provided that includes several constraints characterizing the negative samples of the knowledge graph 100, which is read from, for example, storage 220.

[0085] Then, step 312 is executed.

[0086] In step 312, at least one triple 224 is determined by multiple vector representations 218 of entities and relations.

[0087] In the example, a triple among multiple triples 228 that violates at least one of the constraints for a correct triple, or violates at least some combination of the constraints, is a negative sample.

[0088] In one example, at least one triple 224 is determined that violates at least one constraint for a correct triple. In another example, at least one triple 224 is determined that violates at least some combination of constraints.

[0089] Determining at least one triple 224 involves, in this example, determining the likelihood that at least one triple in multiple vector representations 218 of entities and relations is a fact in the knowledge graph 100. This likelihood is, in this example, a score determined by the knowledge graph embedding model 208. In this example, the knowledge graph embedding model 208 provides predictions along with a score for each prediction. In this example, the triples are ranked by their scores.

[0090] Determining at least one triple 224 involves, in the example, selecting several triples in the vector representations 218 of entities and relations that are more likely to be facts in the knowledge graph 100 than other triples in the vector representations 218 of entities and relations.

[0091] Ranking triples may involve selecting k facts with the highest likelihood from m triples sorted by likelihood.

[0092] Determining at least one triple may involve selecting at least one triple from multiple vector representations 218 of entities and relations that is different from the triples in the set of facts 216. This avoids the duplication of negative samples in training.

[0093] At least one triple 224 can be determined from the selected triples.

[0094] At least one triple 224 may be determined based on a triple of the knowledge graph determined from the knowledge graph 100, as described later.

[0095] In one example, two entities are randomly sampled from an entity, and a relation is sampled from a relation in the knowledge graph. The resulting knowledge graph triple contains the first entity and relation and the second entity.

[0096] The method may include finding candidate triples in multiple vector representations 218 of entities and relations, depending on a first entity, and in particular based on a vector representation of the first entity. Candidate triples are sampled, for example, from multiple vector representations 218 of entities and relations.

[0097] The method may involve determining, based on triples in the knowledge graph, whether a candidate triple violates at least one constraint for the correct triple. Relations can also be used as reference relations.

[0098] The method may include determining that a candidate triple violates the constraints for the correct triple if the relation is not acceptable according to the constraints.

[0099] In this example, at least one triple 224 is a candidate triple that violates at least one constraint for a correct triple.

[0100] At least one triple 224 may be a candidate triple that violates at least some combinations of the constraints.

[0101] As an optional measure, steps 302 to 312 are repeated without training to determine further negative samples.

[0102] As an optional means, step 314 is then performed to iteratively and automatically train the knowledge graph embedded model 208.

[0103] In step 314, at least one triple 224 is added to the set of triples 216.

[0104] Then, step 302 is executed.

[0105] This means that the knowledge graph embedded model 208 is retrained with the modified set in this way. This means that at least one triple 224 is determined based on the knowledge graph embedded model 208 that has been retrained in this way.

[0106] Determining at least one triple 224 in step 312 may involve providing a triple of the knowledge graph from knowledge graph 100.

[0107] A triple in a knowledge graph may include a first entity and a reference relation or representation thereof.

[0108] The method involves repeating, for example, the multiple triples predicted by the knowledge graph embedding model 208 until they no longer result in inconsistencies with the knowledge graph and ontology.

[0109] The predicted triple may relate to the machine's state, the characteristics of an object in a digital image, or the answer to a question.

[0110] The aforementioned triples can indicate the machine's status, the characteristics of an object in a digital image, or the answer to a question.

[0111] A knowledge graph can also represent knowledge about the mapping of machine state messages to machine states. The method may include receiving state messages and outputting machine states in response to those messages. A state can be determined by predicting, using a knowledge graph embedded model, whether a triple exists containing a subject entity representing the state and an object entity representing the machine state. The method may include outputting machine states.

[0112] In the case of digital image processing, the knowledge graph may be a description of objects recognized by object recognition for an image. Entities in the knowledge graph may represent objects and / or their characteristics. The method may include receiving objects and outputting descriptions corresponding to those objects.

[0113] In the case of Street View, an object may be a car, a person, a house, or other part of the infrastructure. In the case of Street View, a Knowledge Graph triple can describe the object's relationships with objects and / or other objects, particularly in the digital image. The method may include receiving an object and outputting a description depending on the object.

Claims

1. A method for automatically generating negative samples for training a knowledge graph embedded model (208), particularly in a computer-implemented method, To provide (302) at least one first triple (212) which is a true triple of the knowledge graph (100), To provide at least one second triple (214) (304), Training a knowledge graph embedding model (208) to predict triples in the knowledge graph (100) in accordance with a set of triples (216) including at least one first triple (212) and at least one second triple (214) (306), The vector representation (218) of entities and relations is determined by the aforementioned knowledge graph embedding model (208) (308), Determining a plurality of triples (228) by the vector representation (218) of the entities and relations (312), To provide an ontology (222) that includes constraints characterizing a correct triple (310), The ontology determines (312) at least one triple (224) of the plurality of triples (228) that violates at least one of the constraints or at least a combination of the constraints, A method characterized by including the following.

2. Determining the at least one triple (224) (312) includes selecting some of the triples (228) that have a higher likelihood of being a fact in the knowledge graph than the other triples (228) among the triples (228), The method according to claim 1.

3. In particular, the knowledge graph embedding model (208) includes determining the likelihood (308) that at least one of the plurality of triples (228) is a fact in the knowledge graph (100), The method according to claim 2.

4. Determining the aforementioned at least one triple (312) is, To provide knowledge graph facts from the aforementioned knowledge graph, which include a first entity and a reference relation or its representation that is a reference type, Determining a triple among the plurality of triples, including the first entity and relation, Determining whether the relation is of an acceptable type according to the constraints, If the aforementioned type is not acceptable, it is determined that the triple violates the aforementioned constraint, including, The method according to claim 1.

5. Determining the aforementioned at least one triple (312) is, Determining a set of triples from the plurality of triples that includes a triple that violates the aforementioned constraint, Selecting at least one triple from the set of triples that is different from the triple mentioned above, including, The method according to claim 1.

6. To automatically train the knowledge graph embedded model, the method is as follows: In the first iteration, determine the at least one triple (224) (312), Adding the aforementioned at least one triple (224) to the set of triples (216) for a second iteration, In the second iteration, the knowledge graph embedded model (208) is trained by the set of triples (216) for the second iteration, and / or, in the second iteration, the at least one triple (224) is determined by the set of triples (216) for the second iteration. Further including, The method according to claim 1.

7. In a device (200) for automatically generating negative samples to train a knowledge graph embedded model (208), A storage (220) configured to provide a knowledge graph (100) and / or an ontology (222) including constraints characterizing a correct triple, A machine learning system (202) is configured to provide at least one first triple (212) which is a true triple of the knowledge graph (100), provide at least one second triple (214), train the knowledge graph embedding model (208) to predict the triples of the knowledge graph (100) in accordance with a set of triples (216) including the at least one first triple (212) and the at least one second triple (214), and determine (308) vector representations (218) of entities and relations by the knowledge graph embedding model (208), A generator (204) configured to determine (308) a plurality of triples (228) by vector representations (218) of the entities and relations, Equipped with, The generator (204) is configured to determine by the ontology (222) at least one triple (224) of the plurality of triples (228) that violates at least one of the constraints or at least a combination of the constraints. A device (200) characterized by the following features.

8. The generator (204) is configured to select some of the triples (228) that have a higher likelihood of being facts in the knowledge graph (100) than the other triples (228). The apparatus (200) according to claim 7.

9. In particular, the knowledge graph embedding model (208) includes a machine learning system (202) configured to determine the likelihood that at least one of the plurality of triples (228) is a fact in the knowledge graph (100). The apparatus (200) according to claim 8.

10. The device includes a storage (206) configured to provide facts from the knowledge graph (100), The facts in the aforementioned knowledge graph include a first entity and a reference relation or its representation, which is a reference type. The generator (204) is Determine the triple among the plurality of triples, including the first entity and relation described above. Determine whether the relation is of an acceptable type according to the constraints, If the aforementioned type is not acceptable, then it is determined that the triple violates the constraint. It is structured in such a way. The apparatus (200) according to claim 7.

11. To determine the at least one triple (224), the generator (204) A set of triples containing a triple that violates the aforementioned constraint is determined from the plurality of triples. Select at least one triple from the plurality of triples (228) that is different from the triple in the set of triples (216). It is structured in such a way. The apparatus (200) according to claim 7.

12. In order to automatically train the knowledge graph embedded model (208), the machine learning system (202) In the first iteration, determine the at least one triple (224), Add the at least one triple (224) to the set of triples (216) for the second iteration, In the second iteration, the knowledge graph embedded model (208) is trained by the set of triples (216) for the second iteration, and / or, in the second iteration, the at least one triple (224) is determined by the set of triples (216) for the second iteration. It is further structured in the following way: The apparatus (200) according to claim 7.

13. A computer program characterized by including instructions for causing a computer to perform the steps of the method described in claim 1 when executed by the computer.

14. A non-temporary computer-readable storage medium storing the computer program described in claim 13.

Images