Knowledge graph embedding method based on chain rule combination representation
By mining and converting rule information in the knowledge graph and iterative training combined with triple data, the problem of failure to effectively utilize rule information in the existing technology is solved, and the ability of embedded representation of the knowledge graph and downstream task performance are improved.
Patent Information
- Application Number
- CN202510949676.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-10
- Publication Date
- 2025-08-08
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
The existing knowledge graph embedding methods fail to effectively utilize potential rule information, resulting in limited performance of downstream tasks.
By obtaining triplets and their inverse triplet data, using rule automatic mining tools to mine rules, convert them into chain rules, and designing energy functions for iterative training, and learning the final embedding representation.
Improve the ability to embed representation of knowledge graphs and improve the performance of downstream tasks.
Smart Images
Figure CN120450000A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a knowledge graph embedding method based on chain rule combination representation, which belongs to the field of knowledge graph representation learning. Background Art
[0002] Knowledge graphs contain rich structural information, providing richer feature information for learning entity and relationship embeddings. Knowledge graph embedding learning is a crucial step in downstream tasks such as knowledge graph completion. Learning deeper embedding representations of entities and relationships helps improve the processing efficiency of downstream tasks.
[0003] Currently, most existing knowledge graph embedding methods only utilize the triple information already in the knowledge graph, without considering the underlying information in the knowledge graph. Consequently, they fail to learn deeper features of entities and relationships, impacting the performance of downstream tasks. To address this issue, potential rule information is mined from the graph structure of the knowledge graph and used for knowledge graph embedding, further improving the knowledge graph embedding learning capability. However, due to the inconsistency of the rule structure, the mined rule information cannot be directly applied to the triple embedding representation. Therefore, how to combine the rule information mined from the knowledge graph with the triple information for knowledge graph embedding learning is a technical problem that needs to be solved urgently. Summary of the Invention
[0004] The technical problem to be solved by the present invention is to provide a knowledge graph embedding method based on chain rule combination representation, so as to integrate rule information in knowledge graph embedding learning and improve the performance of knowledge graph downstream tasks.
[0005] The technical solution of the present invention is: a knowledge graph embedding method based on chain rule combination representation, the specific steps are:
[0006] Step 1: Input the knowledge graph and obtain triples and their inverse triples as the triples to be learned;
[0007] Step 2: Use the rule automatic mining tool to mine rules in the knowledge graph, calculate the confidence of the rules, and select the two-hop rules with a confidence threshold greater than the confidence threshold to add to the rule set;
[0008] Step 3: Introduce an inverse relation for the non-chained rules in the rule set, convert the non-chained rules into chained rules, and obtain the relational expression;
[0009] Step 4: Combine each triple in the triple data to be learned with the relational expression corresponding to the relation in the triple to obtain a combined sequence as training data;
[0010] Step 5: Design the model’s energy function and optimization objectives, perform iterative training, and learn to obtain the final embedding representation.
[0011] The Step 2 is specifically as follows:
[0012] The form of the rule is defined as , where X and Y represent entity variables in the rule, is the rule body in the rule, A triple combination, , It is the rule head in the rule, consisting of a triple, the confidence of the rule The calculation formula is as follows:
[0013]
[0014] in, is the number of triple instances in the knowledge graph that satisfy both the rule body and the rule head, is the number of triple instances that satisfy the rule body in the knowledge graph, and F is the entity variable in the rule;
[0015] Define a two-hop rule as the number of triplets in the rule body The two-hop rules with a confidence level greater than the threshold P are selected and added to the rule set.
[0016] The Step 3 is specifically as follows:
[0017] The chain rule is , where Z is the entity variable in the rule, 、 and For the relationship in the triple, for the non-chained rules in the rule set , introducing the relationship The inverse relationship The chain rule that is converted to is ;
[0018] According to the chain rule , confidence ,definition The relationship is If the relationship There is no corresponding chain rule, then Relational middle , , .
[0019] The Step 4 is specifically as follows:
[0020] Step 4.1: A triple in the knowledge graph is recorded as ,in and are the head entity and tail entity in the triple respectively. The total number of entities in the knowledge graph is the number of all head entities and tail entities in the knowledge graph triple after duplication, recorded as , number each entity, and the entity number is recorded as ,satisfy , the combined sequence is recorded as , including positive samples and negative samples ;
[0021] Step 4.2: The triple Relationships in triples The corresponding relationship Combine and obtain positive samples of the combined sequence ;
[0022] Step 4.3: Combine negative samples of the sequence Including negative samples of the head entity and negative samples of the tail entity, the number of negative samples of the head entity is set to , the number of negative samples of the tail entity is set to , the operations to obtain the negative samples of the head entity and the negative samples of the tail entity are as follows:
[0023] In the interval Generate a random integer as the entity number to replace the head entity and the tail entity. For the positive sample of the combined sequence , generate a negative sample of the head entity as , generate a negative sample of the tail entity ,in and The head entity and tail entity corresponding to the randomly selected head entity number and tail entity number, the triplet in the negative sample and , which do not exist in the knowledge graph;
[0024] Repeat Step 4.3 until the number of negative samples of the generated head entity and the number of negative samples of the tail entity have reached , negative samples of the combined sequence The quantity is .
[0025] The step 5 is specifically as follows:
[0026] For the combined sequence, the energy function of the model is set as:
[0027]
[0028] in, is the energy function score, is a combined sequence; is the embedding representation corresponding to the head entity h, For the tail entity The corresponding embedding representation is, For the relationship The corresponding embedding representation is, For the relationship The corresponding embedding representation is, For the relationship The corresponding embedding representation; It's a relationship The corresponding confidence of the chain rule, when the relationship When there is no corresponding chain rule, ;
[0029] According to the designed energy function, the optimization objective is:
[0030]
[0031] in, is the contrast loss value; is the boundary parameter; is the energy function score of the positive sample of the combined sequence, is the energy function score of the negative sample of the combined sequence;
[0032] Randomly initialize the embedding representation with a dimension of d. Input the positive and negative samples of the combined sequence into the model, output the energy function value, calculate the contrast loss value, use the chain rule to calculate the gradient of the loss function with respect to the embedding representation, pass the gradient from the output layer to the input layer through the backpropagation algorithm, use the optimizer to update the embedding representation, repeat the above process, and perform iterative training until the predetermined number of training rounds is reached. After training is completed, the final embedding vector is obtained.
[0033] The beneficial effects of the present invention are as follows: first, a knowledge graph is input to obtain triples and their inverse triples, and a rule set is obtained using an automatic rule mining tool; then, an inverse relationship is introduced for the non-chained rules in the rule set, converting the non-chained rules into chained rules to obtain a relational expression; finally, the triple data is combined with the relational expression corresponding to the relationship in the triple, and an energy function and optimization objective are designed for iterative training to learn and obtain the final embedded representation. The present invention can consider the potential rule information of the relationship in the triple embedding representation, thereby improving the knowledge graph embedding representation capability. BRIEF DESCRIPTION OF THE DRAWINGS
[0034] Figure 1 is a flow chart of the steps of the present invention;
[0035] Figure 2 It is a schematic diagram of the knowledge graph;
[0036] Figure 3 It is a schematic diagram of the combination sequence in the knowledge graph embedding space. DETAILED DESCRIPTION
[0037] The present invention will be further described below with reference to the accompanying drawings and specific embodiments.
[0038] Example 1: Figure 1 As shown in FIG, a knowledge graph embedding method based on chain rule combination representation is described. The specific steps of the method are as follows:
[0039] Step 1: Input the knowledge graph and obtain triples and their inverse triples as the triples to be learned.
[0040] Specifically, knowledge graphs such as Figure 2 As shown in Figure 1, the triple entities are nodes in the knowledge graph, and the triple relationships are edges in the knowledge graph. Table 1 shows the triples and inverse triples of the knowledge graph.
[0041] Table 1 Triples and inverse triples of knowledge graph
[0042]
[0043] Step 2: Use the rule automatic mining tool to mine rules in the knowledge graph, calculate the confidence of the rules, and select the two-hop rules that are greater than the confidence threshold to add to the rule set.
[0044] Specifically, the calculation results are shown in Table 2 below.
[0045] Table 2 Calculation results
[0046]
[0047] Set the confidence threshold P = 0.6, select the two-hop rules with a confidence greater than the confidence threshold P and add them to the rule set:
[0048] The number of triplets in the rule body of Rule 1 is 1, which does not meet the definition of a two-hop rule.
[0049] Rule 2 complies with the two-hop rule definition and meets the confidence requirement, so it is added to the rule set.
[0050] Rule 3 complies with the two-hop rule definition and meets the confidence requirement, so it is added to the rule set.
[0051] Rule 4 complies with the two-hop rule definition and meets the confidence requirement, so it is added to the rule set.
[0052] Rule 5 meets the definition of a two-hop rule, but its confidence level does not meet the requirement.
[0053] Step 3: Introduce an inverse relation for the non-chained rules in the rule set, convert the non-chained rules into chained rules, and obtain the relational expression.
[0054] Specifically, the rule set includes rule 2, rule 3, and rule 4;
[0055] Rule 2 It is the chain rule, obtaining the relationship The relationship , confidence, confidence .
[0056] Rule 3 It is the chain rule, obtaining the relationship The relationship , confidence, confidence .
[0057] Rule 4 For non-chained rules in a rule set, introduce the relationship The inverse relationship The chain rule that is converted to is , get the relationship The relationship , confidence .
[0058] Step 4: Combine each triple in the triple data to be learned with the relational expression corresponding to the relation in the triple to obtain a combined sequence as training data.
[0059] Specifically, the combined sequence is recorded as , including positive samples and negative samples For a triple in the knowledge graph Relationships in triples The corresponding relationship Combine and obtain positive samples of the combined sequence .
[0060] The number of negative samples of the head entity is set to , the number of negative samples of the tail entity is set to , Can be 1. Negative samples of combined sequence Including negative samples of head entities and negative samples of tail entities, the number is . Randomly replace triples The head entity and tail entity in , generate a negative sample of the head entity as , generate a negative sample of the tail entity , among which and To randomly select the head entity and tail entity, the triplets in the negative samples do not exist in the knowledge graph.
[0061] Step 5: Design the model’s energy function and optimization objectives, perform iterative training, and learn to obtain the final embedding representation.
[0062] Specifically, the positive samples of the combined sequence The corresponding embedding representation In the knowledge graph embedding space, Figure 3 As shown, the energy function is defined as ;
[0063] Negative samples for the head entity of the combined sequence , the energy function is ;
[0064] Negative samples for the tail entity of the combined sequence , the energy function is .
[0065] According to the energy function, the optimization objective is:
[0066]
[0067] in, is the contrast loss value; is the boundary parameter, 5 is acceptable; is the energy function score of the positive sample of the combined sequence, is the energy function score of the negative sample of the combined sequence;
[0068] The embedding representation is randomly initialized with a dimension of d. The training process is as follows: positive and negative samples from the combined sequence are input into the model, the energy function is output, and the contrastive loss is calculated. The gradient of the loss function with respect to the embedding representation is calculated using the chain rule. The gradient is propagated from the output layer to the input layer via backpropagation, and the embedding representation is updated using an optimizer. This process is repeated for an iterative training cycle of epochs. After training, the final embedding vector is obtained and saved to a file for use in tasks such as knowledge graph completion and knowledge graph question answering.
[0069] The present invention is practicable as described below by way of experiments based on the specific implementation records.
[0070] Experimental data: The FB15K-237 dataset, derived from the larger Freebase knowledge graph, is an important benchmark dataset in the knowledge graph field and widely used in research areas such as knowledge graph embedding methods, link prediction, and relational reasoning. The dataset contains 14,541 entities and 237 relations, totaling 310,116 triples. The dataset is divided into training, validation, and test sets, containing 272,115, 17,535, and 20,466 triples, respectively.
[0071] In the experiment, all the triples in the training set of the FB15K-237 dataset were used as input, and the proposed method was used to learn the knowledge graph embedding. The final embedding representation was obtained and evaluated on the knowledge graph entity link prediction task. The entity link prediction task in the knowledge graph aims to predict the missing head entity or tail entity in the triple of the knowledge graph, and obtain a set of candidate entities to evaluate the performance of the knowledge graph embedding. The evaluation indicators used are MRR (Mean Reciprocal Ranking, the inverse of the ranking) and (Top-10 recommendation). The larger the values of these two evaluation indicators, the better the learning effect of the knowledge graph embedding method.
[0072]
[0073]
[0074] Where S represents the set of triples in the test set in the knowledge graph dataset; |S| represents the number of triples in the test set; represents the entity link prediction rank of the i-th triple in the test set.
[0075] Experimental settings: confidence threshold P = 0.7, embedding representation dimension d = 200, embedding representation random initialization method is Xavier, number of negative samples neg_num = 25, margin parameter γ = 5, number of epochs of training = 2000, SGD stochastic gradient descent is selected as the optimizer, and the learning rate is 1. The evaluation index is calculated after the experimental results are filtered, that is, the known real entities are removed from the candidate entity set.
[0076] Using the automatic rule mining tool AMIE+, we performed rule mining on the training set of the FB15K-237 dataset and obtained rules with a confidence threshold greater than 0.7 and a length of 2. Some of these rules are shown in Table 3, where X, Y, and Z represent three different entities in the knowledge graph.
[0077] Table 3 Partial rules
[0078]
[0079] The knowledge graph is embedded and learned using the method of the present invention to obtain the final embedding representation. An entity link prediction experiment is performed on the test set of the FB15K-237 dataset, and the experimental results are compared as shown in Table 4.
[0080] Table 4 Comparison of experimental results
[0081]
[0082] RUGE is an iterative model that uses soft rules to predict soft labels for unlabeled triples and uses labeled triples to modify the embedded representation. R-GCN+ is an enhanced version of the Relational Graph Convolutional Network (R-GCN) that can process multi-relational graph data in knowledge graphs. Experimental comparisons show that the proposed method achieves the best performance. Compared to RUGE, the proposed method achieves a significant improvement of 12.1% in the MRR metric and 13.7% in the Hits@10 metric.
[0083] The above describes the specific embodiments of the present invention in detail with reference to the accompanying drawings. However, the present invention is not limited to the above embodiments. Various changes can be made within the knowledge of ordinary technicians in this field without departing from the purpose of the present invention.
Claims
1. A knowledge graph embedding method based on chain rule combination representation, characterized by: Step 1: Input the knowledge graph and obtain triples and their inverse triples as the triples to be learned; Step 2: Use the rule automatic mining tool to mine rules in the knowledge graph, calculate the confidence of the rules, and select the two-hop rules with a confidence threshold greater than the confidence threshold to add to the rule set; Step 3: Introduce an inverse relation for the non-chained rules in the rule set, convert the non-chained rules into chained rules, and obtain the relational expression; Step 4: Combine each triple in the triple data to be learned with the relational expression corresponding to the relation in the triple to obtain a combined sequence as training data; Step 5: Design the model’s energy function and optimization objectives, perform iterative training, and learn to obtain the final embedding representation.
2. A knowledge graph embedding method based on chain rule combination representation according to claim 1, characterized in that: The Step 2 is specifically as follows: The form of the rule is defined as , where X and Y represent entity variables in the rule, is the rule body in the rule, A triple combination, , It is the rule head in the rule, consisting of a triple, the confidence of the rule The calculation formula is as follows: ； in, is the number of triple instances in the knowledge graph that satisfy both the rule body and the rule head, is the number of triple instances that satisfy the rule body in the knowledge graph, and F is the entity variable in the rule; Define a two-hop rule as the number of triplets in the rule body The two-hop rules with a confidence level greater than the threshold P are selected and added to the rule set.
3. A knowledge graph embedding method based on chain rule combination representation according to claim 1, characterized in that: The Step 3 is specifically as follows: The chain rule is , where Z is the entity variable in the rule, 、 and For the relationship in the triple, for the non-chained rules in the rule set , introducing the relationship The inverse relationship The chain rule that is converted to is ; According to the chain rule , confidence ,definition The relationship is ; If the relationship There is no corresponding chain rule, then Relational middle , , .
4. A knowledge graph embedding method based on chain rule combination representation according to claim 1, characterized in that: The Step 4 is specifically as follows: Step 4.1: A triple in the knowledge graph is recorded as ,in and are the head entity and tail entity in the triple respectively. The total number of entities in the knowledge graph is the number of all head entities and tail entities in the knowledge graph triple after duplication, recorded as , number each entity, and the entity number is recorded as ,satisfy , the combined sequence is recorded as , including positive samples and negative samples ; Step 4.2: The triple Relationships in triples The corresponding relationship Combine and obtain positive samples of the combined sequence ; Step 4.3: Combine negative samples of the sequence Including negative samples of the head entity and negative samples of the tail entity, the number of negative samples of the head entity is set to , the number of negative samples of the tail entity is set to , the operations to obtain the negative samples of the head entity and the negative samples of the tail entity are as follows: In the interval Generate a random integer as the entity number to replace the head entity and the tail entity. For the positive sample of the combined sequence , generate a negative sample of the head entity as , generate a negative sample of the tail entity ,in and The head entity and tail entity corresponding to the randomly selected head entity number and tail entity number, the triplet in the negative sample and , which do not exist in the knowledge graph; Repeat Step 4.3 until the number of negative samples of the generated head entity and the number of negative samples of the tail entity have reached , negative samples of the combined sequence The quantity is .
5. The knowledge graph embedding method based on chain rule combination representation according to claim 1 is characterized in that: The step 5 is specifically as follows: For the combined sequence, the energy function of the model is set as: ； in, is the energy function score, is a combined sequence; is the embedding representation corresponding to the head entity h, For the tail entity The corresponding embedding representation is, For the relationship The corresponding embedding representation is, For the relationship The corresponding embedding representation is, For the relationship The corresponding embedding representation; It's a relationship The corresponding confidence of the chain rule, when the relationship When there is no corresponding chain rule, ; According to the designed energy function, the optimization objective is: ； in, is the contrast loss value; is the boundary parameter; is the energy function score of the positive sample of the combined sequence, is the energy function score of the negative sample of the combined sequence; Randomly initialize the embedding representation with a dimension of d. Input the positive and negative samples of the combined sequence into the model, output the energy function value, calculate the contrast loss value, use the chain rule to calculate the gradient of the loss function with respect to the embedding representation, pass the gradient from the output layer to the input layer through the backpropagation algorithm, use the optimizer to update the embedding representation, repeat the above process, and perform iterative training until the predetermined number of training rounds is reached. After training is completed, the final embedding vector is obtained.