Extensible rule mining method fusing multi-source information

Abstract

The invention relates to an extensible rule mining method fusing multi-source information, which is a mixed learning method based on logical reasoning and representation learning, combines the advantages of the logical reasoning and representation learning, and utilizes various knowledge elements of a knowledge graph to assist rule mining. According to the method, the numerical attribute information and the concept information of the entity are utilized to enhance the embedded representation of the entity, semantic constraints are provided for the rule, in addition, the rule search process is optimized, and a large number of redundant rules are filtered out. And finally, obtaining a high-quality rule through measurement indexes such as confidence. It is worth mentioning that the method has excellent expandability, the embedding module of the method can use different knowledge graph embedding models, and different types of multi-source information are fused to assist rule learning. The method has remarkable effectiveness in learning high-quality rules and utilizing the rules to carry out link prediction tasks.

Description

Technical Field

[0001] The present invention relates to the field of knowledge graph technology, and in particular to a rule mining method that integrates multi-source information. Background Art

[0002] Since Google proposed the concept of knowledge graph, knowledge graph has provided strong support for multiple NLP tasks such as intelligent question answering and personalized recommendation. Although there are a large number of entity and fact data in the current knowledge graph, such as DBpedia, Freebase, Wikidata, YAGO, etc., which contain hundreds of millions of triples, it still faces a very serious problem of missing data. It is necessary to use knowledge reasoning to complete the missing entities or relationships in the knowledge graph.

[0003] However, the existing knowledge reasoning methods have the following shortcomings. First, the typical knowledge reasoning model uses representation learning methods to do reasoning. By mapping entities and relations into low-dimensional continuous vector spaces, the reasoning prediction is converted into a simple vector / matrix operation. However, they are black box models that lack interpretability and are not applicable to some tasks that require high security and interpretability of the model. Second, the knowledge reasoning model based on logical rules explicitly and interpretably expresses the reasoning process through rules. However, the traditional rule reasoning model simply learns closed path rules that meet the conditions, and then uses different metrics to filter the rules. Due to the incompleteness of the knowledge graph, the idea of ​​filtering out high-quality rules only through indicators such as confidence is not perfect, and may even make wrong predictions. Third, the hybrid reasoning model that combines representation learning and logical rules uses the representation of entities and relations to filter rules, or adds text description information to the representation learning of entities and relations, and uses accurate text information to filter rules. However, only a few entities and relations in the knowledge graph have text information, and the text corpus is not always available. In addition, the above methods only consider the factual information or text information of the triples, ignoring other information.

[0004] Technologies and research related to the present invention include: Galárraga et al. (LA Galárraga, et al. AMIE: Association rule mining under incomplete evidence in ontological knowledgebases. International Conference on World Wide Web, 2013: 413–422) used PCA confidence to evaluate and screen rules and mine closed rules from incomplete knowledge bases. Meilicke et al. (C. Meilicke et al. Stuckenschmidt, Fine-grained evaluation of rule and embedding-based systems for knowledge graph completion. International Semantic WebConference, 2018: 3–20) estimated the confidence by randomly sampling the paths of the rule body, which simplified the calculation process of the confidence, so that longer rules can be mined. Meilicke et al. (C. Meilicke et al. Anytime bottom-up rule learning for knowledge graph completion. International Joint Conference on Artificial Intelligence, 2019: 3137–3143) adopted a bottom-up approach to gradually expand the scale and complexity of rules. Although this type of rule mining algorithm avoids the expensive cost of manually acquiring rules, due to the incompleteness of the knowledge graph itself, indicators such as confidence may misjudge the quality of the rules, thereby generating noise rules and affecting the reasoning results.

[0005] In recent years, knowledge graph reasoning models based on representation learning have received extensive attention and development. There are also models that enhance knowledge graph embedding by fusing multi-source information, such as Xie et al. (Xie R et al. Representation Learning of Knowledge Graphs with Hierarchical Types. International Joint Conference on Artificial Intelligence, 2016: 2965-2971) taking into account the hierarchical type information of entities in the knowledge graph, Socher et al. (Socher R et al. Reasoning with neural tensor networks for knowledge base completion. Conference and Workshop on Neural Information Processing Systems, 2013: 926-934) using auxiliary news corpus to initialize entity representation, and Wang et al. (B. Wang et al. Structure-augmented text representation learning for efficient knowledge graph completion. International Conference on World Wide Web, 2021: 1737–1748) making comprehensive use of the structural information and text description information of the knowledge graph. Such models are usually highly scalable, but due to the black-box characteristics of deep learning models, they have poor interpretability. In contrast, models based on logical rule reasoning are highly interpretable and accurate, but face the challenge of poor scalability.

[0006] Hybrid learning based on representation learning and logical rules hopes to combine their respective advantages to improve existing models. For example, Omran et al. (PG Omran et al. Scalable rule learning via learning representation. International Joint Conference on Artificial Intelligence, 2018: 2149–2155) used an embedding model to embed entities and predicates in rules, and then designed scoring functions based on these embedding representations to prune rules, thereby mining high-quality rules in knowledge graphs. Ho et al. (VTHo et al. Rule learning from knowledge graphs guided by embedding models. International Semantic Web Conference, 2018: 72–90) ranked and pruned learned rules under the guidance of external text resources to more accurately judge the quality of candidate rules in incomplete knowledge graphs. Zhang et al. (Zhang W et al. Iteratively learning embeddings and rules for knowledge graph reasoning. International Conference on World Wide Web, 2019: 2366-2377) proposed 7 types of rules based on OWL2, and performed representation learning and rule mining in an iterative manner, so that each module can enhance each other in the iterative process to fully model the rules. In addition, differentiable rule learning that uses neural networks such as LSTM / RNN to simulate rule reasoning has also received widespread attention in recent years. For example, Yang et al. (F. Yang et al. Differentiable learning of logical rules for knowledge base reasoning. Conference and Workshop on Neural Information Processing Systems, 2017: 2319–2328) proposed a fully differentiable system in which differentiable matrix operations allow the model to simultaneously learn the parameters and structure of first-order logic rules. Summary of the invention

[0007] In order to overcome the shortcomings of poor interpretability of representation learning and poor scalability of traditional logic rule learning, the present invention provides a scalable rule mining method that integrates multi-source information. The present invention integrates digital attribute information and conceptual information of knowledge graphs to assist in learning high-quality rules, thereby improving the performance of knowledge graph reasoning.

[0008] In order to achieve the above object, the technical solution adopted by the present invention is as follows:

[0009] A multi-source information-integrated and scalable rule mining method comprises the following steps:

[0010] Step 1: Perform stratified sampling of entities and relations according to the target predicate to simplify the knowledge graph;

[0011] Step 2: Use the concept-aware negative sampling strategy to sample high-quality negative samples for the simplified knowledge graph;

[0012] Step 3: Use the embedding model to embed the simplified knowledge graph, and replace the negative sampling module of the embedding model with the concept-aware negative sampling in step 2 to obtain the entity embedding result that integrates multi-source information;

[0013] Step 4: Optimize the rule mining algorithm and preliminarily obtain candidate rules in the knowledge graph;

[0014] Step 5: Filter candidate rules using the entity embedding result of multi-source information fused in step 3 to obtain high-quality candidate rules;

[0015] Step 6: Evaluate and screen high-quality rules to complete the knowledge graph.

[0016] Moreover, the specific implementation method of step 1 is: for the knowledge graph , entity set and triple sets , given the target predicate , the maximum length of the rule , generate sample entity sets layer by layer And the sample triple set , , ,in for Entities in For the relationship between entities:

[0017] , where the entity set By entity set The target predicate Directly connected entities composition;

[0018] , where the entity set By entity set Entities in By any predicate Connected Entities composition;

[0019] , where the entity set By entity set Entities in By any predicate Connected Entities composition;

[0020] Covers the entity information involved in the rules to be mined, Covers To obtain the triple information related to the entities in the rule mining task, it is only necessary to perform embedding calculation on the small-scale knowledge graph after stratified sampling, and the embedding information required for the rule mining task can be obtained while improving efficiency.

[0021] Moreover, the specific implementation method of step 2 is: design a concept-aware negative sampling strategy, according to the diversity characteristics of the relationship in the knowledge graph, 1-1, 1-N, N-1, NN negative sampling is performed, and the basic embedding model obtains negative samples by simply replacing the correct entity. Different from the strategy, concept-aware negative sampling assigns weights to entities:

[0022]

[0023]

[0024]

[0025]

[0026] in, Represents a triple in the knowledge graph, and represent and Candidate negative sample entities, and They are and The negative sample entity obtained by 1-end sampling and N-end sampling, It means that the basic embedding model is used to embed negative samples Perform embedding scoring, Represents the candidate negative sample Perform embedding scoring, and Similarly, and are the probability and weight of negative samples respectively, is the hyperparameter temperature in self-adversarial sampling; due to the uniqueness of the 1-end entity, candidate negative sample entities with high weights are selected, and due to the non-uniqueness of the N-end entity, candidate entities with low weights are selected for replacement to integrate conceptual information into the entity embedding results.

[0027] Moreover, the specific implementation method of step 3 is: set a learnable function , embedding the entity and properties as input and map them to vectors of the same dimension as the entity embedding, , Represents a vector that combines entity information and its numerical attribute information, so as to integrate the numerical attribute information into the entity embedding result; It is set as a multi-layer neural network with a gating mechanism, which selectively retains useful information and discards useless information.

[0028] Moreover, the specific implementation method of step 4 is: use the method of multiplying the relational adjacency matrix to search the rule path, and perform search optimization on this basis: if a rule is found to have repeated atoms during the rule search process, this path will not be explored, otherwise this path will be explored to avoid the generation of a large number of redundant rules.

[0029] Moreover, the specific implementation method of step 5 is: setting the scoring function and , Calculate the semantic similarity of relation predicates in rules, Calculate the average entity embedding similarity of the relation predicates in the rules, and They complement each other and use the knowledge graph entity embedding results that integrate multi-source information to screen out high-quality candidate rules that conform to semantics.

[0030] Moreover, the specific implementation method of step 6 is: define two measurement indicators, head coverage HC and standard confidence SC, to evaluate the high-quality candidate rules obtained in step 5, and stipulate , The rules of are quality rules, , The rules are high-quality rules.

[0031] The advantages and positive effects of the present invention are:

[0032] The present invention mines a series of first-order logic closed rules in the knowledge graph, then uses metrics to filter the rules, and builds an extensible knowledge graph embedding model. Through this model, the numerical attribute information and conceptual information of the entity are used to enhance the entity representation, so that the entity embedding result integrates the entity structure information and rich multi-source information. Finally, in the process of using metrics to filter the rules, these multi-source information are used to provide rich semantic constraints for the rules, which has a positive impact on the learning of high-quality rules. The present invention combines multi-source information with logical rules to learn high-quality rules, thereby improving the performance of knowledge graph reasoning. The scalability of the present invention enables it to use different knowledge graph embedding models and integrate more useful external information. BRIEF DESCRIPTION OF THE DRAWINGS

[0033] Figure 1 This is a KG example graph with attribute information of the present invention;

[0034] Figure 2 This is a KG example diagram with conceptual information of the present invention;

[0035] Figure 3 This is a simple KG example diagram associated with concepts of the present invention;

[0036] Figure 4 It is a simplified schematic diagram of the knowledge graph of the present invention;

[0037] Figure 5 It is a schematic diagram of the overall process of the present invention. DETAILED DESCRIPTION

[0038] The present invention is further described in detail below with reference to the accompanying drawings.

[0039] The inventive concept of the present invention is: first, a series of first-order logic closed rules are mined in the knowledge graph, and the rules are screened using metrics. In addition, the present invention also includes an extensible knowledge graph embedding model, which uses the attribute information and concept information of the entity to enhance the entity representation, so that the embedding result of the entity integrates the entity structure information and rich multi-source information. In the process of screening rules, these multi-source information are used to provide semantic constraints for the rules to improve the quality of the rules and the learning effect.

[0040] Figure 1 The KG in contains both structural information and numerical attribute information of entities. Combining structural information and numerical attribute information can improve the accuracy of prediction, because students who go to the same school and are of the same age are more likely to know each other. In addition to attribute information, concepts are also an important resource for human cognition. Compared with knowledge, concepts are more basic and general, which is conducive to knowledge representation and rule learning. For example Figure 2In the triple (Red Sorghum, Language, Chinese) in (left), the entity "Red Sorghum" can represent both a movie and a book, which cannot be distinguished by this triple alone. When the conceptual information "movie" of the entity is added, the representation of the entity becomes clear. In addition, there are many entities whose specific content cannot be distinguished and need the assistance of conceptual information, such as Figure 2 (Right) The “apple” in the middle.

[0041] The rules mined by this method are used as an example to illustrate. There are a large number of facts in a knowledge graph that satisfy the following rules:

[0042] (1)

[0043] It meets the conditions of closed rules and its confidence may be very high, but it is still a noise rule. On the one hand, a company may have offices in many cities, but employees can only live in one city. On the other hand, employees working in the company do not necessarily live in the company's location. The present invention uses multi-source information to enhance the representation of entities and relationships, and uses these representations to filter rules, so as to filter out this low-quality rule with high confidence but inconsistent with common sense.

[0044] In order to mine high-quality rules in the knowledge graph and complete the knowledge graph, the present invention needs to define the knowledge graph and the first-order logic rules symbolically. And it is necessary to define the measurement indicators of the first-order rules to filter and select high-quality rules.

[0045] The knowledge graph is defined as . is an entity set, is a set of relation triples, where yes and A relationship between two entities (predicate). is a set of attribute triples, where Is Entity and attribute values The attribute relationship between them.

[0046] The present invention regards entities as instances of concepts, and the relationships between entities are instances of the relationships between the concepts they belong to. The concept-level triple set is represented as In addition, the present invention combines the concept-level triples containing the same relationship into one set. ,in For relationship All the corresponding head concepts A collection of For relationship All the corresponding tail concepts A collection of Figure 3 shown.

[0047] Since the search space for learning rules is very large, the present invention mainly learns the following first-order Horn clause rules:

[0048] (2)

[0049] in represents an entity in KG, Represents the relationship in KG, is the target predicate. is called an atom, is the head of the rule, is the rule body. The number of atoms in the rule body is the length of the rule.

[0050] The present invention selects some metrics from the main rule learning methods to evaluate the quality of the rules. For the rules of form (2), The support of is defined as:

[0051] (3)

[0052] in Refers to the number of inference results of this rule body, is the number of correct results. It refers to the number of correct results that the rule body can infer.

[0053] In the knowledge graph link prediction task, head coverage (HC) refers to the ratio of correct prediction results covering facts, and standard confidence (SC) refers to the proportion of correct prediction results in all prediction results. The definitions are as follows:

[0054] (4)

[0055] A scalable rule mining method that integrates multi-source information, such as Figure 5 As shown, and There are two scoring functions in the rule filtering process, including the following steps:

[0056] Step 1: Perform stratified sampling of entities and relations according to the target predicate to simplify the knowledge graph.

[0057] like Figure 4 As shown in Figure 2, in order to efficiently mine rules in large-scale knowledge graphs, we first Simplify the process, that is, for the knowledge graph , entity set and triple sets , given the target predicate , the maximum length of the rule , generate sample entity sets layer by layer And the sample triple set , , in for Entities in For the relationship between entities:

[0058] , where the entity set By entity set The target predicate Directly connected entities composition;

[0059] , where the entity set By entity set Entities in By any predicate Connected Entities composition;

[0060] , where the entity set By entity set Entities in By any predicate Connected Entities composition;

[0061] so Covers all entity information involved in the rules to be mined by the present invention, It covers all triple information. It can be seen that the present invention only needs to focus on the small-scale knowledge graph obtained by stratified sampling. Entities and facts in By performing embedding calculations, the knowledge graph embedding information required for subsequent tasks can be met.

[0062] Step 2: Use the concept-aware negative sampling strategy to sample high-quality negative samples for the simplified knowledge graph.

[0063] The purpose of using concept-aware negative sampling is to inject conceptual information into triples without changing the basic embedding model training strategy. Generally, negative triples with the same concept are more difficult to distinguish from positive triples. For example, (Paris, locatedInCountry, China) and (Paris, locatedInCountry, phone) are both negative triples, but the former is obviously of higher quality than the latter and is more difficult to distinguish from the positive triple (Paris, locatedInCountry, France) because the former satisfies the common sense of (City, locatedInCountry, Country). Therefore, the present invention hopes to use conceptual information to sample high-quality negative triples. Specifically, the present invention performs negative sampling based on the diversity characteristics of the relations in KG (i.e., 1-1, 1-N, N-1, NN), rather than obtaining negative samples by simply replacing the correct entities as in the basic embedding model. The concept-aware negative sampling strategy is as follows:

[0064] ①, 1-end sampling: For N-1 relations, high-quality negative samples can be obtained by simply replacing the tail entity with other entities in the KG that have the same concept as it. For example, in the positive sample (Paris, locatedInCountry, France), where the relation locatedInCountry is an N-1 relation, it is sufficient to replace the 1-end entity France with an entity that shares the Country concept with it.

[0065] ② N-end sampling: For N-1 relationships, replacing the N-end entity with other entities of the same concept is likely to produce false negative samples. For example, if the N-end entity Paris is replaced with other cities, the replaced entity may also be located in France. Therefore, the present invention assigns weights to candidate replacement entities through formulas (5) and (6).

[0066] in, Represents a triple in the knowledge graph, and represent and Candidate negative sample entities, and They are and The negative sample entity obtained by 1-end sampling and N-end sampling, It means that the basic embedding model is used to embed negative samples Perform embedding scoring, Represents the candidate negative sample Perform embedding scoring, and Similarly, and are the probability and weight of negative samples respectively, is the hyperparameter temperature in self-adversarial sampling. In simple terms, due to the uniqueness of the 1-end entity, a candidate replacement entity with a high weight is selected, and due to the non-uniqueness of the N-end entity, a candidate entity with a low weight is selected for replacement.

[0067]

[0068] (5)

[0069]

[0070] (6)

[0071] Step 3: Use the embedding model to embed the simplified knowledge graph, and replace the negative sampling module of the embedding model with the concept-aware negative sampling in step 2 to obtain the entity embedding result that integrates multi-source information.

[0072] Create attribute matrix , represents the kth attribute value of the i-th entity in the knowledge graph, then The i-th row It can be regarded as the attribute vector of the ith entity. The present invention sets a function , which takes as input the embedding and attributes of an entity and maps them to a vector of the same dimension as the entity embedding, . Represents a vector that combines entity information and its numerical attribute information. This vector contains the original embedding information and attribute information of the entity and can be used as the input of the basic embedding model. is a learnable function that can be a linear transformation

[0073] (7)

[0074] It can also be a nonlinear transformation

[0075] (8)

[0076] It can also be a multi-layer neural network or a multi-layer neural network with a gating mechanism

[0077] (9)

[0078] Replace the scoring function X of the base embedding model with That is, we get entity embedding with attribute information. For example:

[0079] (10)

[0080] The present invention is based on minimizing the marginal loss in the training embedding process based on the scoring function

[0081] (11)

[0082] among them is a marginal hyperparameter. Negative samples are generated through concept-aware negative sampling in step 3. , .

[0083] Step 4: Optimize the rule mining algorithm and preliminarily obtain candidate rules in the knowledge graph.

[0084] The regular path search is performed using the method of multiplying the relational adjacency matrix. Take the rule as an example:

[0085] (12)

[0086] Assume a relationship , and the target predicate The adjacency matrix of is:

[0087] (13)

[0088] So:

[0089] (14)

[0090] The results show that This rule is established. , , In addition, the present invention adopts an optimization strategy that if a rule is found to have repeated atoms during the rule search process, then this path will not be explored. This avoids the generation of a large number of redundant rules, especially Such rules.

[0091] Step 5: Utilize the entity embedding result of step 3 that fuses multi-source information to filter candidate rules and obtain high-quality candidate rules.

[0092] Given a target predicate, there are a large number of candidate rules that are structurally valid. The present invention uses two scoring functions and Filter out high-quality candidate rules that meet semantics. First, consider the semantic similarity of the relational predicates in the rules. That is, for the rules , the relation predicate in the rule body , With the target predicate in the rule header Similarity between The calculation formula is as follows:

[0093] (15)

[0094] in is the function for calculating similarity in the form of Frobenius norm:

[0095] (16)

[0096] Secondly, the present invention designs a co-occurrence-based function using entity embedding, which is widely studied in the field of natural language processing. For example, and The co-occurrence of The tail entity and Therefore, the present invention calculates the average entity embedding of the relation predicate. , its average entity embedding is defined as the average of all entity embeddings connected to it, see Equation (17). express China Relations The set of all subjects of the head entity, Representing relationships The set of all objects of is also the set of tail entities. and Respectively The average subject embedding and average object embedding of . For the rule , The subject and The subject of The object and The object of The object and The similarity of the three parts of the subject The calculation formula is as follows:

[0097] (17)

[0098] (18)

[0099] Promote to Rules

[0100] (19)

[0101] (20)

[0102] The present invention uses and The two scoring functions complement each other to screen a large number of candidate rules and obtain high-quality candidate rules.

[0103] Step 6: Evaluate and screen high-quality rules to complete the knowledge graph.

[0104] Rule evaluation evaluates and screens the high-quality candidate rules generated in step 5, and finally obtains high-quality rules. According to the indicators in step 1. The present invention first uses The valuable rules are returned, which may still contain a large number of redundant and low-quality rules, so the returned rules are further cleaned up based on two metrics SC and HC. , The rules of are quality rules, , The rules are high-quality rules.

[0105] The high-quality rules mined from the knowledge graph through the above steps are used to complete the knowledge graph. The present invention uses two data sets, FB15K and FB15K-237, and selects MRR (Mean Reciprocal Rank) and Hits@n as evaluation indicators for link prediction. In the concept-aware negative sampling in step 2, the sampling temperature is set to 0.5. In step 3, DisMult and ComplEx are selected as the basic embedding models, and the Adam optimizer is used for training, with a learning rate of 0.001, a batch size of 128, and an embedding dimension of 200. In step 4, the maximum length of the rule is set to 3. In step 5, the maximum number of candidate rules is set to 1000. In step 6, set HC min , SC min , HC max , SC max =[0.001, 0.01, 0.001, 0.5] The present invention is compared with representation learning methods and rule learning methods in link prediction tasks. The detailed results are shown in Table 1.

[0106] Table 1 Link prediction results on two datasets

[0107]

[0108] Compared with the representation learning method, the present invention performs better and achieves the best results on both FB15K-237 and FB15K. Compared with the four rule learning methods, the present invention achieves the best results in three indicators on FB15K-237. It achieves the second best results in two indicators on FB15K. The present invention can significantly improve the mining efficiency and the quality of rules, thereby improving the accuracy of knowledge graph completion.

[0109] It should be emphasized that the embodiments described in the present invention are illustrative rather than restrictive. Therefore, the present invention includes but is not limited to the embodiments described in the specific implementation manner. Any other implementation manners derived by those skilled in the art based on the technical solution of the present invention also fall within the scope of protection of the present invention.

Claims

1. A scalable rule mining method integrating multi-source information, characterized by: The following steps are involved: Step 1: Perform stratified sampling of entities and relations according to the target predicate to simplify the knowledge graph; Step 2: Use the concept-aware negative sampling strategy to sample high-quality negative samples for the simplified knowledge graph; Step 3: Use the embedding model to embed the simplified knowledge graph, and replace the negative sampling module of the embedding model with the concept-aware negative sampling in step 2 to obtain the entity embedding result that integrates multi-source information; Step 4: Optimize the rule mining algorithm and preliminarily obtain candidate rules in the knowledge graph; Step 5: Filter candidate rules using the entity embedding result of multi-source information fused in step 3 to obtain high-quality candidate rules; Step 6: Evaluate and screen high-quality rules to complete the knowledge graph.

2. The scalable rule mining method integrating multi-source information according to claim 1 is characterized by: The specific implementation method of step 1 is: for the knowledge graph , entity set and triple sets , given the target predicate , the maximum length of the rule , generate sample entity sets layer by layer And the sample triple set , , ,in for Entities in For the relationship between entities: , where the entity set By entity set The target predicate Directly connected entities composition; , where the entity set By entity set Entities in By any predicate Connected Entities composition; , where the entity set By entity set Entities in By any predicate Connected Entities composition; Covers the entity information involved in the rules to be mined, Covers To obtain the triple information related to the entities in the rule mining task, it is only necessary to perform embedding calculation on the small-scale knowledge graph after stratified sampling, and the embedding information required for the rule mining task can be obtained while improving efficiency.

3. The scalable rule mining method integrating multi-source information according to claim 1 is characterized by: The specific implementation method of step 2 is: design a concept-aware negative sampling strategy, and perform negative sampling based on the diversity characteristics of the relations in the knowledge graph, 1-1, 1-N, N-1, NN. Different from the strategy of the basic embedding model to obtain negative samples by simply replacing the correct entity, concept-aware negative sampling assigns weights to entities: ； ； ； ； in, Represents a triple in the knowledge graph, and represent and Candidate negative sample entities, and They are and The negative sample entity obtained by 1-end sampling and N-end sampling, It means that the basic embedding model is used to embed negative samples Perform embedding scoring, Represents the candidate negative sample Perform embedding scoring, and Similarly, and are the probability and weight of negative samples respectively, is the hyperparameter temperature in self-adversarial sampling; due to the uniqueness of the 1-end entity, candidate negative sample entities with high weights are selected, and due to the non-uniqueness of the N-end entity, candidate entities with low weights are selected for replacement to integrate conceptual information into the entity embedding results.

4. The scalable rule mining method integrating multi-source information according to claim 1, characterized in that: The specific implementation method of step 3 is: set a learnable function , embedding the entity and properties as input and map them to vectors of the same dimension as the entity embedding, , Represents a vector that combines entity information and its numerical attribute information; thereby integrating the numerical attribute information into the entity embedding result; It is set as a multi-layer neural network with a gating mechanism, which selectively retains useful information and discards useless information.

5. The scalable rule mining method integrating multi-source information according to claim 1 is characterized by: The specific implementation method of step 4 is: use the method of multiplying the relational adjacency matrix to search the rule path, and perform search optimization on this basis: if a rule is found to have repeated atoms during the rule search process, this path will not be explored, otherwise this path will be explored to avoid the generation of a large number of redundant rules.

6. The scalable rule mining method integrating multi-source information according to claim 1, characterized in that: The specific implementation method of step 5 is: setting the scoring function and , Calculate the semantic similarity of relation predicates in rules, Calculate the average entity embedding similarity of the relation predicates in the rules, and They complement each other and use the knowledge graph entity embedding results that integrate multi-source information to screen out high-quality candidate rules that conform to semantics.

7. The scalable rule mining method integrating multi-source information according to claim 1 is characterized by: The specific implementation method of step 6 is: define two measurement indicators, head coverage HC and standard confidence SC, to evaluate the high-quality candidate rules obtained in step 5, and stipulate , The rules of are quality rules, , The rules are high-quality rules.

Images