Method and system for learning rules from a database - Patents.com

Abstract

To provide a system and computer-implemented method for learning rules from a database including entities and relations between the entities, each entity being either a constant or a numerical value, where a relation between a constant and a numerical value is a numerical relation and a relation between two constants is a non-numerical relation.SOLUTION: A computer-implemented method 200 provided herein comprises: deriving aggregate values from numerical and / or non-numerical relations (step 210); deriving non-numerical relations from the aggregate values (step 220); adding the derived non-numerical relations to a database (step 230); constructing differentiable operators for referring to a non-numerical or a derived non-numerical relation of the database (step 240); and extracting rules from the differentiable operators (step 250).SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] background The present invention relates to a system and computer-implemented method for learning rules from a database, where the database has entities and relationships between said entities, where the entities are either constants or numbers, where the relationship between the constants and the numbers is a numerical relationship, and where the relationship between two constants is a non-numeric relationship. [Background technology]

[0002] Such databases are also known as knowledge graphs, which are primarily used for graph-based knowledge representation by describing (real-world) entities and their relationships organized as graphs.

[0003] Rules for knowledge graphs capture interpretable patterns in the data so that the rules can be used to predict, complete, and clean the knowledge graph. Several approaches are known for deriving such rules from knowledge graphs. However, such approaches are limited in their handling of numerical data.

[0004] Predictions about entities in a database are often based not only on the relationships of an entity to other entities, but also on abstractions about sets of such relationships commonly expressed by aggregates. Summary of the Invention [Problem to be solved by the invention]

[0005] It is therefore desirable to learn rules from a database representing such aggregates. [Means for solving the problem]

[0006] Summary of the Invention The above is achieved by means of an apparatus and method as claimed in the independent claims.

[0007] A computer-implemented method for learning rules from a database containing entities and relationships between the entities, where the entities are either constants or numerical values, a relationship between a constant and a numerical value is a numerical relationship, and a relationship between two constants is a non-numeric relationship, the method comprising the steps of deriving aggregate values ​​from the numerical and / or non-numeric relationships, deriving non-numeric relationships from the aggregate values, adding the derived non-numeric relationships to a database, constructing differential operators that reference the non-numeric relationships in the database or the derived non-numeric relationships, and extracting rules from the differential operators using a neural network.

[0008] An aggregate value is a single resulting value from a set of input values. Because aggregate values ​​are numeric, the relationship between an entity that is a constant and an entity that is an aggregate value is a numerical relationship. Instead of adding the aggregate value as a new entity in the form of a numeric value to the database, a non-numeric relationship is derived from the aggregate value and added to the database.

[0009] Exemplary details for constructing a differential operator, also known as the TensorLog operator, which allows deriving new facts by investigating inferences on database facts in a differentiable way, are disclosed, for example, in the reference "COHEN William W., YANG Fan, MAZAITIS Kathryn, Tensorlog: Deep Learning Meets Probabilistic Databases. CoRR, 2017, abs / 1707.05390". However, this approach is limited to handling numerical data.

[0010] The method according to the invention advantageously allows for the integration of aggregated values ​​in the rule learning process.

[0011] According to one embodiment, deriving an aggregate value from the numerical and / or non-numerical relationships comprises applying an aggregation function to the numerical and / or non-numerical relationships, which groups multiple values ​​together, performs various operations on the grouped values, and returns a single result, referred to herein as an aggregate value.

[0012] Preferably, examples of aggregation functions include average, sum, min, max, count, and various other aggregation functions. The count function is preferably applied to numerical and non-numerical relationships. Further aggregation functions, such as average, sum, min, and max, are preferably applied to numerical relationships.

[0013] According to one embodiment, the step of deriving a non-numeric relationship from the aggregated value comprises encoding a numerical relationship into a non-numeric relationship, such that a relationship between a constant and a numerical value is encoded into a relationship between two constants.

[0014] According to one embodiment, the step of encoding the numerical relations into non-numerical relations comprises comparing the entities with respect to the numerical relations to which they belong. It has been found to be useful to compare two or more entities with respect to numerical relations of similar form. Typically, comparisons with respect to numerical relations of different form are meaningless.

[0015] The above cited references provide exemplary details of constructing a differential operator, also known as a TensorLog operator, in which all entities are mapped to integers and each entity i is represented as a one-hot encoded vector v i so that only the i-th entry will be 1 and the other entries will be 0.

[0016] According to one embodiment, the step of constructing the differential operator comprises constructing a matrix referring to the non-numeric relations. Preferably, a matrix is ​​constructed for each non-numeric relation. Preferably, the matrix is ​​an i×i matrix, where the entry (j,k) is not 0, e.g. 1, if the relation is between entity j and entity k, otherwise the entry is 0.

[0017] According to one embodiment, the step of constructing the differential operator comprises constructing a negated differential operator. Preferably, constructing the negated differential operator comprises constructing a negation matrix that refers to a negated non-numeric relation. Negation of a matrix can be achieved by flipping all 0's to 1's and vice versa, resulting in a dense matrix from the corresponding (sparse) matrix. Since dense matrices cannot be materialized in TensorLog, negation relations are not directly supported by the approaches known from the cited references. Therefore, according to a preferred embodiment of the present invention, constructing the negated differential operator comprises constructing a negation matrix by flipping only elements in columns that contain at least one non-zero element.

[0018] According to one embodiment, the step of extracting rules from differential operators involves using an artificial neural network, in particular an artificial neural network built on the Neural Logic Programming, LP, framework.

[0019] Exemplary details of Neural Logic Programming, LP, frameworks and training of such frameworks to learn rules are disclosed, for example, in the reference "ZHANG Fan, YANG Zhilin and COHEN, William W, Differentiable Learning of Logical Rules for Knowledge Base Reasoning. NeurIPS, 2017", pages 2316 to 2325.

[0020] According to one embodiment, the step of extracting rules from said differential operators using a neural network is limited by the maximum number of relationships allowed in a rule.

[0021] According to one embodiment, the rules extracted from the differential operator are Horn rules, and the method further comprises the step of decoding the rules into decoded rules. Horn rules extracted from the differential operator using a Neural Logic Programming, LP, framework do not include aggregate values ​​and / or negation. By decoding the Horn rules, the Horn rules can advantageously be decoded into rules with aggregate values ​​and / or rules with negation.

[0022] According to one embodiment, the method further comprises predicting one or more relationships between two or more entities of a database by applying said rules and / or said decoded rules.

[0023] The invention also relates to a system configured to learn rules from a database comprising entities and binary relations between said entities, the system comprising a processing unit configured to execute the method according to any of the embodiments. The processing unit may be a distributed computing system, a microprocessor or a microcontroller.

[0024] According to one embodiment, the system comprises a memory, in particular a computer-readable non-volatile memory, the processing unit and the memory interacting via data lines.

[0025] According to one embodiment, the memory comprises computer readable instructions which, when executed by the processing unit, cause the processing unit to perform a method for learning rules from a database according to the embodiment described above.

[0026] According to one embodiment, the system may comprise a database or may be connected to a further memory in which the database is stored such that the processing unit may access the data of the database when the method according to the embodiment is executed.

[0027] According to one embodiment, the system further comprises a processing unit and a memory unit for the artificial neural network, in particular for the artificial neural network constructed based on a Neural Logic Programming (Neural LP) framework, said processing unit and memory unit being adapted to execute the steps described in one embodiment.

[0028] Exemplary details of Neural Logic Programming, LP, frameworks and training of such frameworks to learn rules are disclosed in the above cited references.

[0029] The invention also relates to a computer program comprising computer readable instructions which, when executed by a computer, cause the computer to carry out the method according to the embodiment.

[0030] The invention also relates to a computer program product, which includes a computer readable storage medium containing a computer program according to the embodiments.

[0031] The invention also relates to the use of the method according to the embodiments and / or the system according to the embodiments and / or the computer program according to the embodiments and / or the computer program product according to the embodiments for predicting relationships between entities of a database and / or for cleaning a database and / or for completing a database.

[0032] Further advantageous embodiments emerge from the following description and the drawings. [Brief description of the drawings]

[0033] [Figure 1] FIG. 2 shows a schematic diagram of a database according to the first embodiment. [Diagram 2] FIG. 13 shows a schematic diagram of a database according to a second embodiment. [Diagram 3] 1 illustrates a first embodiment of a computer-implemented method for learning rules from a database. [Figure 4] 1 illustrates an aspect of a system configured to learn rules from a database. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0034] FIG. 1 shows a schematic diagram of a database 100 according to a first embodiment.

[0035] Such a database 100 is also known as a knowledge graph, which is primarily used for graph-based knowledge representation by describing (real-world) entities and their relationships organized as a graph.

[0036] It should be noted that FIG. 1 is an exemplary and schematic excerpt of a database 100 for illustrative purposes only, and that the methods and systems described herein are applicable to large-scale real-world knowledge graphs that may contain millions and billions of facts.

[0037] According to the embodiment, the database 100 comprises entities 110 and relations 120 between the entities 110. The entities C1, C2, C3, C4, C5, C6 and C7 of the database 100 are constants. The entities N1, N2 and N3 are numerical values. A relation 120 between a constant and a numerical value is called a numerical relation, and a relation 120 between two constants is called a non-numerical relation.

[0038] For example, entity C3 is a first person and entity C4 is a second person. Entity C1 is a first thing and entity C2 is a second thing. The relationship 120a between entity C3 and entity C1 is, for example, "cited in", which means that the first person is cited in the first thing. Furthermore, the relationship 120a between entity C4 and entity C1 and the relationship 120a between entity C4 and entity C2 are also, for example, "cited in", which means that the second person is cited in the first thing and the second thing. According to this example, the relationship 120a is a non-numeric relationship.

[0039] For example, entity C6 is a first patent and entity C7 is a second patent. Numeric value N1 has a value "1", numeric value N2 has a value "2", and numeric value N3 has a value "3". The relationship 120b between entity C6 and entity N1 and the relationship 120b between entity C6 and entity N2 are, for example, "has a rating", which means that the first patent has a rating of "1" and a rating of "2". Furthermore, the relationship 120b between entity C7 and entity N2 and the relationship 120b between entity C7 and entity N3 are, for example, "has a rating", which means that the second patent has a rating of "2" and a rating of "3". According to this example, the relationship 120b is a numerical relationship.

[0040] Rules for database 100 capture interpretable patterns in the data so that the rules can be used to predict, complete, and clean database 100. Predictions for entities 110 in database 100 are often based on abstractions not only about an entity's relationships 120 to other entities 110, but also about sets of such relationships commonly expressed by aggregates.

[0041] It is therefore desirable to learn rules from a database representing such aggregates.

[0042] FIG. 3 illustrates a first embodiment of a computer-implemented method 200 for learning rules from a database 100 .

[0043] According to one aspect, the computer-implemented method 200 begins with a step 210 of deriving an aggregate value from the numerical and / or non-numerical relationships of the database 100. An aggregate value is a single resulting value from a set of input values.

[0044] According to one aspect, deriving 210 aggregate values ​​from the numerical and / or non-numerical relationships includes applying an aggregation function to the numerical and / or non-numerical relationships. An aggregation function calculates a single result value or aggregate value from a set of input values. Preferably, examples of aggregation functions include average aggregation functions, sum aggregation functions, minimum aggregation functions, maximum aggregation functions, count aggregation functions, and various other aggregation functions, with the count function preferably being applied to numerical and non-numerical relationships. Further aggregation functions, such as average aggregation functions, sum aggregation functions, minimum aggregation functions, and maximum aggregation functions, are preferably applied to numerical relationships.

[0045] Applying step 210 to the above example means, for example, applying a count function to the non-numeric relation 120a "cited in". Entity C3 is cited once and entity C4 is cited twice. According to this example, aggregate values ​​"1" and "2" are derived from relation 120a.

[0046] Furthermore, applying the average function to the numerical relation 120b "has a rating" results in an aggregate value of "1.5" for entity C6 and an aggregate value of "2.5" for entity C7.

[0047] Then, step 220 is executed.

[0048] In step 220, non-numeric relationships are derived from the aggregated values.

[0049] Since aggregate values ​​are numeric, the relationship between an entity that is a constant and an entity that is an aggregate value is a numerical relationship. Instead of adding the aggregate value to database 100 as a new entity in the form of a numeric value, in step 220, a non-numeric relationship is derived from the aggregate value.

[0050] According to one embodiment, deriving 220 a non-numeric relationship from the aggregated value comprises encoding a numerical relationship into a non-numeric relationship, such that a relationship between a constant and a numerical value is encoded into a relationship between two constants.

[0051] According to one aspect, the step of encoding the numerical relations into non-numerical relations includes comparing the entities with respect to the numerical relations to which they belong. It has been found to be advantageous to compare two or more entities with respect to numerical relations of similar form. Typically, comparisons with respect to numerical relations of different forms are meaningless.

[0052] Applying step 210 to the above example means, for example, comparing entity C6 and entity C7 with respect to their average ratings. Since entity C6 has an average rating of "1.5" and entity C7 has an average rating of "2.5", a non-numerical relationship of "lower rated" or "higher rated", respectively, can be derived from these aggregated values.

[0053] Moreover, applying step 210 to the above example means, for example, comparing entities C3 and C4 in terms of the amount of citations: since entity C3 is cited once and entity C4 is cited twice, a non-numerical relationship of "more cited" or "less cited", respectively, can be derived from these aggregate values.

[0054] Then, step 230 is executed.

[0055] The non-numeric relationships derived from the aggregate values ​​are added to the database in step 230. Instead of adding the aggregate values ​​as new entities in numerical form to the database, the non-numeric relationships are derived from the aggregate values ​​as described above and added to the database.

[0056] FIG. 2 shows a database 100 according to a second embodiment, to which derived non-numeric relations 130 have been added.

[0057] With respect to the above example, a derived non-numeric relationship 130a "more cited" between entity C4 and entity C3, and a derived non-numeric relationship 130b "less rated" between entity C6 and entity C7 have already been added.

[0058] Then, step 240 is executed.

[0059] In step 240, a differential operator is constructed that references the database non-numeric relations or the derived non-numeric relations.

[0060] Exemplary details of constructing a differential operator, also known as the TensorLog operator, which allows deriving new facts by investigating inferences on database facts in a differentiable way, are disclosed, for example, in the reference "COHEN William W., YANG Fan, MAZAITIS Kathryn, Tensorlog: Deep Learning Meets Probabilistic Databases. CoRR, 2017, abs / 1707.05390," which associates rule application with sparse matrix multiplication. In TensorLog, all entities are mapped to integers, and each entity i is represented by a one-hot encoded vector v i, so that only the i-th entry will be 1 and the other entries will be 0.

[0061] According to one embodiment, the step of constructing 240 the differential operator is performed by using a matrix M R Preferably, the matrix M R is constructed for each non-numerical relation. Preferably, the matrix M R is an i × i matrix, where the entry (j,k) is non-zero, e.g., 1, if the relationship is between entity j and entity k, and the entry is 0 otherwise.

[0062] According to one aspect, constructing 240 the differential operator includes constructing a negated differential operator. Preferably, constructing the negated differential operator includes constructing a negated matrix M Rneg This involves constructing the matrix M R The negation of can be achieved by flipping all 0s to 1s and vice versa, resulting in the corresponding (sparse) matrix M R A dense matrix is ​​obtained from: Since dense matrices cannot be instantiated in TensorLog, negation relations are not directly supported by the approaches known from the cited references. Therefore, to construct the negated differential operator, we use the negated matrix M Rneg is constructed by flipping only those elements in columns that contain at least one non-zero element. The construction of the negated differential operator is performed under the assumption that it can only be concluded that a negated relation between a set of entities is true if the relevant non-negated relation between the set of entities already exists. Otherwise, the negated relation remains unknown.

[0063] Then, step 250 is executed.

[0064] In step 250, rules are extracted from the differential operator.

[0065] According to one embodiment, the step of extracting 250 rules from the differential operators involves using an artificial neural network, in particular an artificial neural network built on the Neural Logic Programming, LP, framework.

[0066] Exemplary details of neural logic programming, LP, frameworks and training of such frameworks to learn rules are disclosed, for example, in the reference "ZHANG Fan, YANG Zhilin and COHEN, William W., Differentiable Learning of Logical Rules for Knowledge Base Reasoning. NeurIPS, 2017", pages 2316 to 2325.

[0067] In contrast to the approaches disclosed in the above cited references, the computer implemented method 200 described in the present invention has the following advantages: by performing steps 210 of deriving aggregate values ​​from the numerical and / or non-numerical relations, 220 of deriving non-numerical relations from the aggregate values, 230 of adding the derived non-numerical relations to a database, and 240 of constructing a differential operator based on the derived non-numerical relations, the present invention allows compiling numerical relations into differential operators. Thus, the method according to the present invention allows to effectively utilize each comparison information and incorporate the information into the rule learning process.

[0068] In contrast, incorporating numerical relations directly into differential operators would result in dense matrices, making the implementation of the numerical relations and storing them in a processor's memory impractical.

[0069] Moreover, the invention makes it possible to construct a negated differential operator that can be implemented in approaches known from the prior art.

[0070] According to one embodiment, the rules extracted from the differential operator are Horn rules, and the method 200 further comprises a step 260 of decoding said rules into decoded rules. Horn rules extracted from the differential operator using a Neural Logic Programming, LP, framework do not have aggregate values ​​and / or negation. By decoding 260 the Horn rules, the Horn rules can advantageously be decoded into rules with aggregate values ​​and / or negation.

[0071] For example, a rule that can be extracted by method 200 is, for example, "A person is influenced by people who are cited more frequently in a topic than other people." After decoding, this rule becomes, "A person is influenced by people who are cited at least twice in a topic."

[0072] An additional rule that can be extracted by method 200 can be, for example, "A patent is important to a person if it has a higher rating than other patents." After decoding, this rule becomes "A patent with a rating higher than 2 is important to a person."

[0073] According to one aspect, the method 200 further comprises predicting 270 one or more relationships 140 between two or more entities 110 of the database 100 by applying the rules and / or the decoded rules. With reference to Figure 2, for example, a relationship 140 between entities C4 and C5 and a relationship 140 between entities C5 and C7 have been predicted by applying 270 the rules.

[0074] Referring again to the above example, a rule that can be extracted and / or decoded by the method 200 can be, for example, "A person is influenced by a person who is cited at least twice in a matter." Entity C3 is cited once and entity C4 is cited twice. The rule can be applied to the database 100 to predict the relationship 140a "influenced" between entity C4 and entity C5. Another rule can be, for example, "Patents with a rating higher than 2 are important to a person." Entity C6 has a rating of "1.5" and entity C7 has a rating of "2.5." The rule can be applied to the database 100 to predict the relationship 140b "important" between entity C7 and entity C5.

[0075] According to one aspect of the present invention, the method 200 can be performed by a system 300 configured to learn rules from a database 100 including entities 110 and binary relations 120 between said entities as shown in FIG.

[0076] In one aspect, shown in FIG. 4, a system 300 includes a processing unit 310 configured to perform the method 200 according to an embodiment.

[0077] In one aspect, the system comprises a memory 320, in particular a computer-readable non-volatile memory 320, the processing unit 310 and the memory 320 interacting via a data line 330. The processing unit 310 may be a distributed computing system, a microprocessor or a microcontroller.

[0078] In one aspect, the memory 320 includes computer readable instructions that, when executed by the processing unit 310, cause the processing unit 310 to perform the method 200 for learning rules from a database according to the embodiments described above.

[0079] In one aspect, the system 300 may include the database 100 or may be connected to additional memory, not shown in FIG. 4, in which the database 100 is stored such that the processing unit 310 may access data from the database 100 when the method 200 is performed.

[0080] According to one embodiment, the system 300 further comprises a processing unit 340 for the artificial neural network and a memory 350 configured to perform the step 250 of extracting rules from said differential operators, as described above.

[0081] In one embodiment, the processing unit 340 for the artificial network is connected to the memory 320 via data line 330 .

[0082] In one aspect, the artificial neural network is constructed based on a Neural Logic Programming, LP, framework. Exemplary details of the Neural Logic Programming, LP, framework and training of such framework to learn rules are disclosed, for example, in the references cited above.

[0083] In one aspect, the method 200 according to the above-described embodiments and / or the system 300 according to the above-described embodiments and / or the computer program according to the above-described embodiments and / or the computer program product according to the above-described embodiments can be used for predicting relationships 120 between entities 110 in the database 100 and / or cleaning the database 100 and / or completing the database 100.

Claims

1. A computer-implemented method (200) for learning rules from a database (100) containing entities (110) and relationships (120) between said entities, comprising: A computer-implemented method (200) in which entities (110) are either constants (C1, C2, C3, C4, C5, C6, C7) or numerical values ​​(N1, N2, N3), a relationship (120) between the constants (C6, C7) and the numerical values ​​(N1, N2, N3) is a numerical relationship (120b), and a relationship (120) between two constants (C1, C2, C3, C4) is a non-numerical relationship (120a), The method (200) comprises: Deriving (210) an aggregate value from the numerical and / or non-numerical relationships; deriving (220) non-numerical relationships from the aggregated values; adding (230) the derived non-numeric relationships to the database; constructing (240) a differential operator that references the non-numerical relations of the database or derived non-numerical relations; Extracting rules from the differential operator (250); A computer-implemented method (200) comprising:

2. said step of deriving aggregate values ​​from said numerical and / or non-numerical relations (210) comprises applying an aggregation function to said numerical and / or non-numerical relations; 2. The method of claim 1 (200).

3. The step of deriving non-numeric relationships from the aggregate values ​​(220) includes encoding numerical relationships into non-numeric relationships.

3. The method (200) of claim 1 or 2.

4. said step of encoding numerical relations into non-numerical relations comprises comparing said entities (110) with respect to the numerical relations to which they belong; 4. The method (200) of claim 3.

5. The step of constructing the differential operator (240) includes a matrix (M Ｒ ), The method (200) of any one of claims 1 to 4.

6. Said step of constructing a differential operator (240) may include constructing a negated differential operator, in particular a negation matrix (M Ｒｎｅｇ ), The method (200) of any one of claims 1 to 5.

7. said step of extracting rules from said differential operators (250) comprises using an artificial neural network, in particular an artificial neural network built on the basis of a Neural Logic Programming, LP, framework; The method (200) of any one of claims 1 to 6.

8. the rules extracted from the differential operators are Horn rules, the method further comprising the step of decoding (260) the rules into decoded rules. The method (200) of any one of claims 1 to 7.

9. The method further comprises predicting (270) one or more relationships (140) between two or more entities (110) of the database (100) by applying the rules and / or the decoded rules.

9. The method (200) of claim 8.

10. A system (300) configured to learn rules from a database (100), the database including entities (110) and binary relations (120) between the entities; The system (300) comprises a processing unit (310) configured to perform the method (200) of any one of claims 1 to 9.

11. The system (300) further comprises a processing unit (340) and a memory unit (350) for an artificial neural network, in particular for an artificial neural network built on the basis of a Neural Logic Programming, LP, framework, the processing unit (340) and the memory unit (350) being adapted to perform the step of extracting rules from the differential operators (250). The system (300) of claim 10.

12. The method comprises computer readable instructions that, when executed by a computer, cause the computer to perform the method (200) of any one of claims 1 to 9. Computer program.

13. A computer readable storage medium containing a computer program according to claim 12.

14. Use of the method (200) according to any one of claims 1 to 9 for predicting relationships (120) between entities (110) of a database (100) and / or for cleaning the database (100) and / or for completing the database (100).

15. Use of a system (300) as described in claim 10 or 11 for predicting relationships (120) between entities (110) of a database (100) and / or for cleaning the database (100) and / or for completing the database (100).

16. Use of the computer program of claim 12 for predicting relationships (120) between entities (110) of a database (100) and / or for cleaning the database (100) and / or for completing the database (100).

Images