Apparatus and method for populating a knowledge graph by means of policy data splitting

Abstract

The invention relates to a device and a method for populating a knowledge graph, wherein a first subset of a set of data points and a second subset of the set of data points are determined (204), wherein a data point assigned with a label is assigned to a cluster of a set of clusters, wherein the data point is assigned to the cluster depending on whether a distribution of labels in data points already assigned to the cluster fulfills a condition, wherein the data points assigned to the cluster are assigned to the first subset or the second subset, wherein a plurality of models for classification are trained (208) depending on the data points in the first subset, wherein for at least one model of the plurality of models a value of a quality measure is determined (210) depending on the data points in the second subset, wherein depending on the value a model for classification is selected (212) from the plurality of models, wherein a classification is determined (216) for a data using the model for classification selected depending on the value, which defines a relation or a node or a node type in the knowledge graph for a sentence.

Description

TECHNICAL FIELD

[0001] The present application starts from a method and a device for populating a knowledge graph. SUMMARY

[0002] By means of the method and the device according to the independent claims, a model is trained for classifying data points and for populating a knowledge graph, which is better suited to unknown data points than other models.

[0003] The method for populating a knowledge graph provides that a first subset of a set of data points and a second subset of the set of data points are determined, wherein data points assigned with a label are allocated to a cluster of a set of clusters, wherein the data points are allocated to the cluster depending on whether a condition is fulfilled by a distribution of labels among data points already allocated to the cluster, wherein data points allocated to the cluster are allocated to the first subset or to the second subset, wherein a plurality of models for classifying are trained on data points of the first subset, wherein for at least one model of the plurality of models a value of a quality measure is determined on data points of the second subset, wherein a model for classifying is selected from the plurality of models depending on the value, wherein a classification is determined for data using the model for classifying selected depending on the value, which defines a relation or a node or a node type in a knowledge graph for a sentence. The first subset comprises training data points. The second subset comprises development data points. The division according to the distribution of labels enables an improved distribution of data points according to their labels, so that a distribution of data points with the same label can be set in the subsets. Thereby, the set of data points can be divided into different subsets. The number of subsets and the number of clusters can be identical. A cluster defines one of the subsets.

[0004] Preferably, a number of data points allocated to a cluster and assigned with a label is determined, wherein the data points are allocated to the cluster if the number fulfils a condition, otherwise not. The number can define a maximum number of data points with the same label allocated to the cluster. As soon as the number is exceeded, the data points are allocated to another cluster, for example.

[0005] In one aspect, a sequence of data points assigned with the label is determined, wherein the data points are allocated to the sequence according to a cluster as soon as a number of data points fulfils a condition. Thereby, the data points are allocated in sequence. In this way, a data point arranged more at the back in the sequence is allocated to the cluster only if a maximum number of data points has not been reached by data points arranged more at the front in the sequence.

[0006] The sequence can comprise data points, wherein the sequence is determined from a difference of a distance of the data points to a first cluster center of a first cluster of the set of clusters and a distance of the data points to a second cluster center of a second cluster of the set of clusters. The difference is a measure for ordering the data points in the sequence.

[0007] Preferably, the data point is arranged in the sequence in front of another data point if the difference is larger than a reference, otherwise the data point is arranged in the sequence behind the other data point. The reference is a difference determined correspondingly for the other data point. Thereby, the data point with the largest difference with respect to the other data points of the sequence in the sequence is the first data point in the sequence. This means that the data point with the largest difference is assigned first to the cluster and the other data points are assigned to the cluster in descending order of their differences.

[0008] Preferably, the first cluster center is the cluster center of the set of cluster centers that is closest to the data point and the second cluster center is the cluster center of the set of cluster centers that is farthest to the data point.

[0009] It can be provided that a third cluster center is determined from the data points assigned to the first cluster, wherein a fourth cluster center is determined from the data points assigned to the second cluster, wherein a first distance of a first data point in the first cluster to the third cluster center and a second distance of the first data point to the fourth cluster center are determined, wherein a third distance of a second data point in the second cluster to the fourth cluster center and a fourth distance of the second data point to the third cluster center are determined, wherein a first difference of the first distance and the second distance is determined, wherein a second difference of the third distance and the fourth distance is determined, wherein the first data point is assigned to the second cluster and the second data point is assigned to the first cluster if the labels of the two data points are identical and if the first difference fulfills a first condition and the second difference fulfills a second condition. Thereby, an exchange of data points can be made if there are two data points that are both farther away from their respective cluster center than from the respective other cluster center. This improves the clustering, or more precisely, the Intra-Cluster Density, i.e. the fact that the respective clusters bring their points as close together as possible, because in this way the data points that are farthest away from their current cluster center are exchanged with the highest priority.

[0010] It can be provided that a plurality of cluster centers is provided, wherein the data points of the set of data points are assigned to one of the cluster centers of the plurality of cluster centers. Thereby, the data points can be assigned to a plurality of subsets.

[0011] Preferably, a number of subsets is specified for the data points, wherein the number of subsets defines the number of cluster centers. For example, K cluster centers can be specified, wherein a first subset is formed by K-1 clusters and a second subset is formed by the remaining one cluster.

[0012] The method for generating training data provides that a first subset of the set of data points and a second subset of the set of data points are determined, wherein the data points assigned with a label are allocated to a cluster of the set of clusters, wherein the data points are allocated to the cluster depending on whether a condition is fulfilled by a distribution of labels among the data points already allocated to the cluster, wherein the data points allocated to the cluster are allocated to the first subset or the second subset.

[0013] The device is configured to implement the method. BRIEF DESCRIPTION OF DRAWINGS

[0014] Further advantageous embodiments result from the following description and the drawings. In the drawings:

[0015] Figure 1 A schematic diagram of a device for populating a knowledge graph is shown;

[0016] Figure 2 Steps in a method for populating a knowledge graph are shown;

[0017] Figure 3 Details of steps of the method are shown. DETAILED DESCRIPTION

[0018] Figure 1 Parts of a device 100 for populating a knowledge graph are shown schematically. The device 100 comprises at least one processor 102, at least one memory 104, and is configured to implement the method described hereinafter.

[0019] A set of data points 106 is stored in the at least one memory 104. In the present example, the data points in the set are defined by embeddings of words, sentences, clauses, or one or more groups of words, and by labels. The embedding of a sentence can be determined from the embeddings of the words of the sentence. Other embeddings can be determined correspondingly from the embeddings of the words. Preferably, a vector for the embedding of a sentence is determined. Preferably, the vector is normalized to one. For the embeddings of the words of the sentence, vectors can be determined which are likewise normalized to one.

[0020] For these words, word2vec embeddings can be used. This is for example implemented as described in Tomas Mikolov, Kai Chen, Greg Corrado, Jeffrey Dean: Efficient Estimation of Word Representations in Vector Space. ICLR Workshop 2013.

[0021] In this example, a set of labels is specified. The data points are assigned one of these labels.

[0022] In this example, a text corpus is set containing the sentence. For other sentences of this text corpus, in this example the processing is done as described for the data points. The set of data points 106 comprises a plurality of data points for a plurality of sentences of the text corpus. Different text corpora can be set. The text corpora can be from a domain. Different text corpora can be from the same domain or from different domains.

[0023] The device 100 comprises a model 108 for classification. The model 108 is constructed to assign an embedding of a sentence, for example of a sentence of a data point of the set of data points, to a label. In this example, the label defines a relation of the sentence in a knowledge graph. The device 100 is constructed to select the model 108 used to populate the knowledge graph from a plurality of models.

[0024] Figure 2 Steps in a method for populating a knowledge graph are shown.

[0025] The method provides a step 202 in which input data is provided. The input data comprises a set of data points.

[0026] In step 204, at least one data point assigned with a label is assigned to a cluster of a set of clusters. The data point is assigned to a first cluster or to a second cluster depending on whether a distribution of labels in data points already assigned to the cluster meets a condition. In the following, this aspect is described in more detail with reference to Figure 3

[0027] In this example, for data points of the set of data points, K clusters are produced which can comprise data points with the same label and / or different labels from each other. It can be provided that each data point is assigned to exactly one cluster.

[0028] ​In step 206, a first subset of the set of data points with training data points and a second subset of the set of data points with development data points are provided in dependence on the K clusters. In the present example, a set of training data points is provided, which set of training data points comprises data points from K-1 of the clusters. In the present example, a set of development data points is provided, which set of development data points comprises data points from one of the clusters. In the following, this aspect is described in more detail with reference to Figure 3

[0029] Preferably, the second subset does not contain data points of the set of training data points. Preferably, the set of development data points does not comprise any of the data points of the first subset or of one of the K-1 clusters.

[0030] It can be provided that another partition is set as an assignment of the K-1 clusters to the first subset and one cluster to the second subset.

[0031] In step 208, a plurality of models for classification is trained in a plurality of epochs on data points of the first subset.

[0032] At least one of the plurality of models can be an artificial neural network, which is defined by its hyperparameters. Different models of the plurality of models can be artificial neural networks with different hyperparameters. In the training, weights of the respective artificial neural network are adapted. Preferably, the training is performed in the plurality of epochs with training data points of the set of training data points. In one aspect, the hyperparameters of the artificial neural network remain unchanged in the training.

[0033] In step 210, for at least one of the plurality of models, a value of a quality measure is determined in dependence on data points of the second subset. The value can be a measure of agreement of labels in classifications of embeddings of the data points of the second subset by the at least one model with labels of the data points. In the present example, the value is determined for the plurality of models.

[0034] In step 212, a model for classification is selected from the plurality of models in dependence on the value. In the present example, the model is selected, which performs the classification of the development data points best according to the value.

[0035] Steps 208 to 212 can be repeated. It can be provided that an average of the values of the models is determined from the repetitions and the model is selected in dependence on the average.

[0036] ​In step 214, a set of data is provided. In the present example, the data comprises embeddings of sentences in a text corpus. The embeddings can be determined as described for the data points in the set of data points. In the present example, the data does not comprise labels. In the present example, the data is determined from a different text corpus than the one or more text corpora that have been used to determine the set of data points.

[0037] In step 216, a classification is determined for the data in the set of data using the model selected for classification according to the value. The classification defines a relation in a knowledge graph for the sentence.

[0038] Figure 3 Details of step 204 of the method are shown.

[0039] In step 302, a plurality of cluster centers is provided.

[0040] The cluster centers are assigned to clusters in a set of clusters. In the present example, the cluster centers are each assigned to a cluster in the set of clusters. In step 300, a plurality of clusters is specified for the data points. In the present example, the number of clusters defines the number of cluster centers. For example, K cluster centers can be specified for K clusters. Different sizes of the clusters can be set. For example, N data points can be set for a second subset, wherein (K-1)*N data points are set for a first subset. Thereby, the training set of data points comprises the (K-1)*N data points of the data points, and the development set of data points comprises the N data points of the data points. For example, it is specified that the N data points in one cluster shall form the second subset. In the present example, the other (K-1)*N data points assigned to the other clusters of the K clusters shall form the first subset.

[0041] The cluster centers are initialized or determined, for example, as described for the K-Means++ algorithm in k-means++: The Advantages of Careful Seeding; David Arthur and Sergei Vassilvitskii; http: / / ilpubs.stanford.edu:8090 / 778 / 1 / 2006-13.pdf.

[0042] In step 304, a sequence of data points assigned to the label is determined.

[0043] In the present example, the sequence is determined from a difference between a distance of a data point from a first cluster center of a first cluster in the set of clusters and a distance of the data point from a second cluster center of a second cluster in the set of clusters.

[0044] In the present example, the first cluster center is the cluster center of the set of cluster centers that is closest to the data point. In the present example, the second cluster center is the cluster center of the set of cluster centers that is farthest from the data point.

[0045] If the difference is larger than a reference, the data point is arranged, e.g. in the sequence, in front of another data point. Otherwise, the data point is arranged in the sequence behind the other data point. Thereby, the data point having the largest difference with respect to the other data points is arranged as the first data point in the sequence. The other data points are arranged in the sequence after the first data point in decreasing order of the difference.

[0046] In the present example, a sequence is determined for a plurality of labels.

[0047] Then, for a plurality of clusters, a respective number of data points is determined for a plurality of labels that have been assigned to the respective cluster and that are assigned with the respective label.

[0048] In step 306, the data points are assigned to the clusters.

[0049] In the present example, a data point is assigned to a cluster for a label if the number of data points assigned to the cluster for the label satisfies a condition. The condition can define a specified maximum number of data points allowed to be assigned to the cluster. This means that the assignment is determined depending on whether the distribution of labels among the data points already assigned to the cluster satisfies the condition.

[0050] In determining the sequence, the data points are assigned to the sequence according to the cluster until the number of data points satisfies the condition.

[0051] In the present example, the data points of the set of data points are assigned to one of the plurality of cluster centers as described for the data point.

[0052] In optional step 308, pairs of data points from different clusters whose labels coincide can be exchanged if appropriate. This is described below for a pair. Corresponding checks can be performed for a plurality of data points.

[0053] In step 308-A, a third cluster center is determined from the data points assigned to the first cluster. The third cluster center is the new cluster center of the first cluster.

[0054] In step 308-B, a fourth cluster center is determined from the data points assigned to the second cluster. The fourth cluster center is the new cluster center of the second cluster.

[0055] In step 308-C, a first distance of a first data point in the first cluster to the third cluster center and a second distance of the first data point to the fourth cluster center are determined.

[0056] In step 308-D, a third distance of the second data point in the second cluster from the fourth cluster center and a fourth distance of the second data point from the third cluster center are determined.

[0057] In step 308-E, a first difference of the first distance and the second distance is determined.

[0058] In step 308-F, a second difference of the third distance and the fourth distance is determined.

[0059] In step 308-G, it is checked whether the first difference fulfills a first condition and whether the second difference fulfills a second condition.

[0060] In the present example, in step 308-H, if both conditions are fulfilled, the first data point is assigned to the second cluster and the second data point is assigned to the first cluster. Thus, the data points are exchanged with each other. Otherwise, the data points are not exchanged with each other.

[0061] Steps 308-A to 308-H can be implemented in the order or in another order for different data points. Preferably, the steps are repeated until no more pairs suitable for exchange are found.

[0062] In a subsequent step 310, the data points assigned to the cluster are assigned to the first subset or to the second subset.

[0063] In the present example, the data points assigned to the first cluster are assigned to the first subset. In the present example, the data points assigned to the second cluster are assigned to the second subset.

[0064] It can be provided that the data points in the plurality of clusters are assigned to the first subset or to the second subset depending on which cluster the data points are assigned to.

Claims

1. A method for populating a knowledge graph, characterized in that, A first subset and a second subset of the data point set are determined (204), wherein labeled data points are assigned (306) to clusters in a cluster set, wherein the data points are assigned to clusters based on whether the distribution of labels among the data points already assigned to the clusters satisfies a condition, wherein the data points assigned to the clusters are assigned (310) to either the first subset or the second subset, wherein multiple models for classification are trained (208) based on the data points in the first subset, wherein for at least one of the multiple models, a quality metric value is determined (210) based on the data points in the second subset, wherein a model for classification is selected (212) from the multiple models based on the value, wherein a classification is determined (216) for the data using the model for classification selected based on the value, the classification defining a relation or node or node type in a knowledge graph for the statement.

2. The method according to claim 1, characterized in that, Determine (304) the number of data points assigned to the cluster and assigned the label, wherein if the number satisfies the condition, the data points are assigned (306) to the cluster, otherwise they are not.

3. The method according to claim 2, characterized in that, Determine (304) the sequence of data points assigned with the label, wherein the data points are assigned (306) to the sequence according to the clustering, provided that the number of data points meets the condition.

4. The method according to claim 3, characterized in that, The sequence includes data points, wherein the sequence is determined based on the difference between the distance of the data point from the first cluster center of the first cluster in the cluster set and the distance of the data point from the second cluster center of the second cluster in the cluster set (304).

5. The method according to claim 4, characterized in that, If the difference is greater than the reference, the data point is arranged (304) in the sequence before another data point; otherwise, the data point is arranged in the sequence after the other data point.

6. The method according to claim 4 or 5, characterized in that, The first cluster center is the cluster center closest to the data point in the cluster center set (304), wherein the second cluster center is the cluster center farthest from the data point in the cluster center set (304).

7. The method according to claim 6, characterized in that, (308-A) A third cluster center is determined based on the data points assigned to the first cluster, (308-B) A fourth cluster center is determined based on the data points assigned to the second cluster, (308-C) A first distance from the first data point in the first cluster to the third cluster center and a second distance from the first data point to the fourth cluster center are determined, (308-D) A third distance from the second data point in the second cluster to the fourth cluster center and a fourth distance from the second data point to the third cluster center are determined, (308-E) A first difference between the first distance and the second distance is determined, (308-F) A second difference between the third distance and the fourth distance is determined, wherein if the labels of the two data points are the same and if the first difference satisfies a first condition and the second difference satisfies a second condition, then the first data point is assigned (308-H) to the second cluster and the second data point (308-H) is assigned to the first cluster.

8. The method according to any one of claims 1 to 5, characterized in that, Provide (300) multiple cluster centers, wherein data points in the set of data points are assigned (306) to one of the multiple cluster centers.

9. The method according to claim 8, characterized in that, (300) subsets are specified for the data points, wherein the number of subsets limits the number of cluster centers.

10. A device, characterized in that, The device is configured to implement the method according to any one of claims 1 to 9.

11. A computer program product, characterized in that, The computer program product includes computer-readable instructions, and when the computer-readable instructions are implemented by a computer, the method according to any one of claims 1 to 9 is carried out.

Images