Knowledge graph embedding method based on parameterized vector offset
By introducing a knowledge graph embedding method with parameterized vector offset, the problem of degree of freedom limitation in the prior art is solved, dynamic adaptation to complex relationships and efficient link prediction are achieved, and the relationship representation ability of the knowledge graph is improved.
Patent Information
- Application Number
- CN202510900055.6
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-07-01
- Publication Date
- 2025-08-01
AI Technical Summary
Existing knowledge graph embedding methods are limited by degrees of freedom, making it difficult to adjust and adapt to complex relationships, resulting in the inability to accurately represent the dynamic nuances of different relationships in various relationship types.
A knowledge graph embedding method for parameterized vector offset is introduced. By constructing a scoring function and a self-adversarial negative sampling loss function, the feature distance is optimized, and the relationship mapping of entities is adjusted using the parameterized offset vector to perform link prediction.
It can better capture and analyze simple and complex relationship patterns, identify multiple relationship types, and improve the accuracy and effectiveness of link prediction, especially when processing complex relationship data.
Smart Images
Figure CN120409653A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to a knowledge graph embedding method based on parametric vector offset, belonging to the technical fields related to knowledge graphs, deep learning, and natural language processing. Background Art
[0002] Knowledge Graphs (KGs) are a kind of knowledge bases that store a large amount of structured data in the form of triples, used to describe various information in the real world, expressed as , where and respectively represent the head entity and the tail entity, represents the relationship between the head entity and the tail entity. Currently popular KGs such as WordNet, Freebase, NeverEnding Language Learning (NELL), etc. have promoted the development of downstream tasks. At present, KGs have been widely applied to semantic parsing, intelligent question answering, information retrieval, and intelligent recommendation. Although knowledge graphs contain a large number of entities and relationships, most knowledge graphs still have incompleteness. To overcome this difficulty, researchers have begun to focus on predicting these missing relationships or entities, that is, performing knowledge graph completion or link prediction. Currently, knowledge graph embedding has become the mainstream research method for knowledge graph completion tasks.
[0003] Knowledge graph embedding usually learns the low-dimensional representations of entities and relationships through the translational distance scoring function mechanism between the head entity and the tail entity. The knowledge graph embedding model based on paired relation vectors (Knowledge Graph Embeddings via Paired Relation Vectors, PairRE) performs The norm constraint restricts the entity vectors on the surface of the hyper-ellipsoid. This specific embedding space limits the optimization of the entity distribution, resulting in similar entity embeddings being entangled and unable to capture the complex relationships between entities. Recently, the Translating knowledge graph embedding with hyper-ellipsoidal restriction (TranSHER) was proposed, which relaxes the constraint of entities on the hyper-ellipsoid in PairRE by using the relationship translation between the head entity and the tail entity. However, TranSHER statically divides the relationship into a relationship head vector and a relationship tail vector and cannot dynamically adapt to the inherent changes in the relationship instances. Therefore, it is difficult for this model to accurately represent the nuances of different relationship dynamics in various relationship types. Secondly, in TranSHER, although the inclusion relaxes the restriction of entity pairs on the hyper-ellipsoid surface in PairRE, a global offset is provided for this relationship and does not directly participate in the interaction process between the head and tail entities, resulting in the inability to describe the characteristics of entities with common feature relationships. Summary of the Invention
[0004] The technical problem to be solved by the present invention is: The present invention aims to provide a knowledge graph embedding method based on parameterized vector offset to solve the problem that existing knowledge graph embedding methods are restricted by degrees of freedom, making it difficult for them to adjust and adapt to complex relationships.
[0005] The technical solution of the present invention is: A knowledge graph embedding method based on parameterized vector offset, which introduces a parameterized offset vector to adjust the relationship mapping of entities, thereby constructing a scoring function to evaluate the credibility of triples, and using a self-adversarial negative sampling loss function for feature distance optimization, and finally using the trained model to perform link prediction tasks.
[0006] The specific steps are as follows:
[0007] Step 1: Perform negative sampling preprocessing on the knowledge graph triple data, divide it into a training set and a test set, and initialize the embedding vector representations of entities and relationships;
[0008] Step 2: Based on the embedding vector representations and the training set, construct a scoring function that incorporates parameterized vector offset operations;
[0009] Step 3: Based on the scoring function, adopt a self-adversarial negative sampling strategy to optimize the model parameters, and train the knowledge graph embedding model through the contrast loss function of positive and negative samples;
[0010] Step 4: Use the trained knowledge graph embedding model to generate candidate triples by batch replacement of the head entity or the tail entity, and predict the missing links based on the scoring function according to the test set.
[0011] The specific content of Step 1 is as follows:
[0012] Step 1.1: Load the preprocessed knowledge graph triple dataset and divide it into a training set and a test set according to a certain proportion. The dataset consists of a number of triples which is the head entity, is the relationship between the head entity and the tail entity, is the tail entity. The training set is used for parameter learning of the knowledge graph embedding model, and the test set is used for verifying the prediction effect of the knowledge graph embedding model;
[0013] Specifically, input the knowledge graph data, and count the total number of entities (nentity) and the total number of relationship types (nrelation) in the knowledge graph. Then, set the number of generated negative samples according to the training requirements, select the triples in the positive sample set, and randomly replace the head entity or the tail entity of the positive sample according to the negative sample generation mode (mode), while ensuring that the replaced triples are not in the original knowledge graph to avoid introducing invalid data; To address the imbalance of entity and relationship frequencies, calculate the subsampling weights of each entity and relationship, and adjust the weight values according to their occurrence frequencies. Finally, merge the generated negative samples with the positive samples to obtain the data for model training.
[0014] Step 1.2: Input the training set, entity set E, relationship set R, fixed margin , embedding dimension k, and initialize the embedding vector representations of entities and relationships, which participate in the subsequent model training and the construction and optimization process of the scoring function. The fixed margin is used for the interval setting in the loss function to distinguish positive and negative samples, and the embedding dimension is used to specify the length of the entity and relationship embedding vectors.
[0015] The specific content of Step 2 is as follows:
[0016] Step 2.1: Initialize the entity embedding and the relationship embedding , and use a fixed norm constraint to make the norm of all entity vectors equal to 1, and map the entity vectors to the hypersphere surface. The formula is as follows:
[0017]
[0018] Step 2.2: Further refine the relationship mapping, and divide the relationships into , and ,in, and They are the projection parts of the relationship to the head entity and the tail entity respectively. It is the translation part of the relationship, introducing the parameterized vector e, normalizing the parameters, and setting the scaling factor To balance the degree of freedom and stability of the model, define two relationship mapping functions and for:
[0019]
[0020] Step 2.3: For each triple , define the scoring function as:
[0021] in, is the scoring function for triples Rating, represents the Hadamard product.
[0022] The Step 3 is specifically as follows:
[0023] Step 3.1: Define the self-adversarial negative sampling loss function :
[0024]
[0025] in, yes function, is the first artificially constructed Negative samples are used to simulate non-existent relationships between entities to assist the model in learning the characteristics of real relationships. Used to distinguish different negative samples, Represents the weight of the negative sample, specifically expressed as:
[0026]
[0027] Represents a given real triple Then, generate negative samples The weight in the scoring function, and They are negative samples and The scoring function value represents the prediction result of the scoring function for different negative samples. α represents the sampling temperature and controls the weight distribution between negative samples.
[0028] Step 3.2: Load the training set data, generate multiple negative samples for each positive sample, calculate the prediction scores of the positive and negative samples through forward propagation, and calculate the total loss using the self-adversarial negative sampling loss function;
[0029] Step 3.3: Evaluate the model performance. When the model performance continues to improve, update the model parameters. If there is no improvement for a continuous preset number of rounds, stop in advance and save the current optimal model parameters.
[0030] Specifically, in each evaluation step, the model calculates the scores of the positive and negative samples and uses these scores to determine the ranking. According to the ranking results, various evaluation metrics are calculated and logs are recorded. By continuously adjusting the model parameters, its performance is optimized. When the performance reaches the best, the current model parameters are saved.
[0031] The specific content of Step 4 is as follows:
[0032] Step 4.1: Load the trained model, which includes a scoring function and embedding vectors corresponding to entities and relationships respectively. Subsequently, receive or generate query triples containing missing entities or relationships for the model to make predictions and score;
[0033] Step 4.2: For each query triple, generate all candidate entities based on the knowledge graph or entity and relationship sets, calculate the scores of each candidate entity as the missing part using the scoring function in the loaded model, and combine the similarity or distance between the embedding vectors corresponding to the candidate entity and the known part to obtain the rationality evaluation result of each candidate entity;
[0034] Step 4.3: According to the rationality evaluation results of each candidate entity, sort them from high to low according to the scores, and select the candidate entity with the highest score as the final prediction result.
[0035] The beneficial effects of the present invention are as follows: By introducing parameterized vectors for automatically adjusting entities for relationship transformation, when mining the potential information between the head and tail entities, a higher translation freedom is obtained, thereby further reducing the translation distance between the head and tail entities. This method can comprehensively capture and analyze simple and complex relationship patterns and multiple relationship types; this method can not only identify the intuitive one-to-one relationships between entities, but also identify multi-relationship patterns involving multiple entities and complex interactions. In addition, this method can also model various relationship patterns such as symmetric / anti-symmetric, inverse relationships, and composite relationships, fully demonstrating its effectiveness and practicality in dealing with complex relationship data. BRIEF DESCRIPTION OF THE DRAWINGS
[0036] Figure 1Flowchart of the knowledge graph embedding method proposed by the present invention. Detailed implementation manners
[0037] The present invention will be further described below in conjunction with the accompanying drawings and detailed implementation manners.
[0038] Example 1: As Figure 1 shown, a knowledge graph embedding method based on parametric vector offset, the specific steps are as follows:
[0039] Step 1: Perform negative sampling preprocessing on the knowledge graph triple data, divide it into a training set and a test set, and initialize the embedding vector representations of entities and relationships.
[0040] Step 1.1: Load the preprocessed knowledge graph triple data set and divide it into a training set and a test set according to a ratio. The data set consists of several triples where is the head entity, is the relationship between the head entity and the tail entity, is the tail entity. The training set is used for parameter learning of the knowledge graph embedding model, and the test set is used for verifying the prediction effect of the knowledge graph embedding model;
[0041] Specifically, input the knowledge graph data, count the total number of entities (nentity) and the total number of relationship types (nrelation) in the knowledge graph. Then, set the number of generated negative samples according to the training requirements, select the triples in the positive sample set, and randomly replace the head entity or the tail entity of the positive sample according to the negative sample generation mode (mode), while ensuring that the replaced triples are not in the original knowledge graph to avoid introducing invalid data; to cope with the imbalance of entity and relationship frequencies, calculate the subsampling weights of each entity and relationship, and adjust the weight values according to their occurrence frequencies. Finally, merge the generated negative samples with the positive samples to obtain the data for model training.
[0042] Step 1.2: Input the training set, entity set E, relationship set R, fixed margin , embedding dimension k, learning rate (lr), batch size, number of training epochs (epochs), etc., and initialize the embedding vector representations of entities and relationships, which are involved in the subsequent model training and the construction and optimization of the scoring function. The fixed margin is used for the interval setting in the loss function to distinguish positive and negative samples, and the embedding dimension is used to specify the length of the entity and relationship embedding vectors.
[0043] Specifically, the input knowledge graph contains an entity set E = {A, B, C, D} and a relation set R = {R1, R2}, and the positive sample triple set is {(A, R1, B), (B, R2, C), (C, R1, D)}. After statistics, the total number of entities nentity = 4, and the total number of relation types nrelation = 2. If the number of negative samples generated is set to 1, the head entity or tail entity of the positive sample is randomly replaced to generate negative samples. For example, the negative sample (A, R1, C) is generated from (A, R1, B). Finally, the training data {(A, R1, B), (B, R2, C), (C, R1, D), (A, R1, C)} is obtained, and the training data and related parameters are input.
[0044] Step 2: Based on the embedding vector representation and the training set, construct a scoring function that incorporates parameterized vector offset operations.
[0045] Step 2.1: Initialize entity embeddings and relation embeddings , and adopt a fixed norm constraint to make the norm of all entity vectors equal to 1, and map the entity vectors to the hypersphere surface. The formula is as follows:
[0046]
[0047] Step 2.2: Further refine the relation mapping. Divide the relations into , and , where and are the projection parts of the head entity and the tail entity of the relation pair respectively, is the translation part of the relation. Introduce a parameterized vector e, and perform normalization on the parameter. Set a scaling factor to balance the degrees of freedom and stability of the model. Define two relation mapping functions and as:
[0048]
[0049] Step 2.3: For each triple , define the scoring function as:
[0050] where is the score of the scoring function for the triple , represents the Hadamard product.
[0051] Specifically, a scoring function is constructed, and entity and relation embeddings are initialized. Entities A, B, C, D and relations R1 and R2 are represented as vectors respectively. After refining the relation mapping, the relation mapping function and are obtained. The distance function value calculated for a triple represents the association strength between the head entity and the tail entity through the relation.
[0052] Step 3: Based on the scoring function, adopt the self-adversarial negative sampling strategy to optimize the model parameters, and train the knowledge graph embedding model through the contrastive loss function of positive and negative samples.
[0053] Step 3.1: Define the self-adversarial negative sampling loss function :
[0054]
[0055] where is function, is the negative sample artificially constructed in the knowledge graph embedding training, used to simulate the non-existent relationship between entities to assist the model in learning the characteristics of real relationships, where is used to distinguish different negative samples, represents the weight of this negative sample, specifically expressed as:
[0056]
[0057] represents the weight of the negative sample generated after the given true triple in the scoring function, and are the scoring function values of the negative samples and respectively, representing the prediction results of the scoring function for different negative samples, and α represents the sampling temperature, controlling the weight distribution between negative samples;
[0058] Step 3.2: Load the training set data, generate multiple negative samples for each positive sample, calculate the prediction scores of the positive samples and negative samples through forward propagation, and calculate the total loss using the self-adversarial negative sampling loss function;
[0059] Step 3.3: Evaluate the model performance. When the model performance continues to improve, update the model parameters. If there is no improvement for a continuous preset number of rounds, stop in advance and save the current optimal model parameters.
[0060] Specifically, in each evaluation step, the model calculates the scores of positive and negative samples and uses these scores to determine the ranking. Based on the ranking results, various evaluation metrics are calculated and logs are recorded. By continuously adjusting the model parameters, its performance is optimized. When the performance reaches the best, the current model parameters are saved.
[0061] Furthermore, the model is optimized based on the self-adversarial negative sampling loss. Through the self-adversarial negative sampling loss function, the loss of the positive sample (A, R1, B) and the weight of the negative sample (A, R1, C) are calculated, and the model parameters are adjusted through optimization.
[0062] Step 4: Use the trained knowledge graph embedding model to generate candidate triples by batch replacement of the head entity or the tail entity, and predict the missing link based on the scoring function according to the test set.
[0063] Step 4.1: Load the trained model, which includes a scoring function and embedding vectors corresponding to entities and relationships respectively. Subsequently, receive or generate query triples containing missing entities or relationships for the model to make predictions and score.
[0064] Step 4.2: For each query triple, generate all candidate entities based on the knowledge graph or the set of entities and relationships. Use the scoring function in the loaded model to calculate the score of each candidate entity as the missing part, and combine the similarity or distance between the embedding vector corresponding to the candidate entity and the known part to obtain the rationality evaluation result of each candidate entity.
[0065] Step 4.3: According to the rationality evaluation results of each candidate entity, sort them from high to low according to the scores, and select the candidate entity with the highest score as the final prediction result.
[0066] Specifically, predict the query triple (A, R1,?), and generate a list of candidate entities {B, C, D}. After calculating the scores, the score of B is the highest, and finally, the highest-scoring B is selected as the prediction result.
[0067] To verify the effectiveness and rationality of the proposed knowledge graph embedding method based on parametric vector offset, a comparative experimental study on the performance comparison between this method and the classical distance-based knowledge graph embedding model was carried out. This invention uses the Mean Reciprocal Rank (MRR) and Hits@N as the evaluation metrics of this method, and N takes values of 1, 3, and 10. MRR is to calculate the average of the reciprocal ranks of all test samples, and the larger the MRR value, the better. Hits@N refers to the average proportion of triples with a ranking less than or equal to N in link prediction. The higher the Hits@N value, the better the model can predict the real entity.
[0068] Link prediction experiments were conducted on five benchmark datasets of different scales, and the experimental results on FB15K-237 are shown in Table 1.
[0069] Table 1 Link prediction results of FB15k-237
[0070] Through the experiments conducted on the FB15K-237 dataset, the knowledge graph embedding method based on parametric vector offset of the present invention performs excellently on multiple evaluation metrics, especially achieving significant improvements in MRR, Hit@3, and Hit@10. Specifically, this method achieves good performance in MR and MRR, and also shows strong competitiveness in Hit@1. The results prove the effectiveness of the model of the present invention in dealing with complex relationships and entity prediction tasks.
[0071] The experimental results on the YOGO37 and DB100K datasets are shown in Table 2. On the YOGO37 dataset, all metrics perform best. On the DB100K dataset, except for Hit@1, this method also achieves improvements in the MRR, Hit@3, and Hit@10 metrics respectively. The experimental results show that this method effectively improves the link prediction performance, verifying the advantages of knowledge graph embedding modeling. Introducing parametric vectors enhances the freedom of the model, can better encode different relationships between entities, and further reduces the distance between the head entity and the tail entity, proving the feasibility and effectiveness of the model of this method.
[0072] Table 2 Link prediction results of DB100K and YOGO37
[0073] The experimental results on the ogbl-wikikg2 and ogbl-biokg datasets are shown in Table 3. This method is superior to the baseline model TranSHER and other translation-based models on both the ogbl-biokg and ogbl-wikikg2 datasets. This method has strong modeling capabilities on large-scale datasets and can effectively improve the link prediction performance.
[0074] Table 3 Experimental results of ogbl-wikikg2 and ogbl-biokg
[0075] Overall, this method achieves excellent performance on most datasets, indicating the effectiveness and feasibility of the method proposed by the present invention on all datasets.
[0076] The specific embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited to the above embodiments, and various changes can be made without departing from the spirit of the present invention within the scope of knowledge possessed by those of ordinary skill in the art.
Claims
1. A knowledge graph embedding method based on parameterized vector offset, characterized in that The specific steps of the method are as follows: Step 1: Perform negative sampling preprocessing on the knowledge graph triple data, divide it into a training set and a test set, and initialize the embedding vector representations of entities and relationships; Step 2: Based on the embedding vector representations and the training set, construct a scoring function that incorporates parameterized vector offset operations; Step 3: Based on the scoring function, adopt a self-adversarial negative sampling strategy to optimize the model parameters, and train the knowledge graph embedding model through the contrastive loss function of positive and negative samples; Step 4: Use the trained knowledge graph embedding model to generate candidate triples by batch replacement of the head entity or the tail entity, and predict the missing links based on the test set according to the scoring function.
2. The knowledge graph embedding method based on parametric vector offset according to claim 1, wherein The specific content of Step 1 is as follows: Step 1.1: Load the preprocessed knowledge graph triple dataset and divide it into a training set and a test set according to a ratio. The dataset consists of a number of triples constituting it, where is the head entity, is the relationship between the head entity and the tail entity, is the tail entity. The training set is used for parameter learning of the knowledge graph embedding model, and the test set is used for verifying the prediction effect of the knowledge graph embedding model; Step 1.2: Input the training set, entity set E, relation set R, fixed margin value and embedding dimension k, and initialize the embedding vector representations of entities and relations.
3. The knowledge graph embedding method based on parameterized vector offset according to claim 1, characterized in that, The specific content of Step 2 is as follows: Step 2.1: Initialize entity embeddings and relation embeddings , and adopt a fixed norm constraint to make the norm of all entity vectors equal to 1, and map the entity vectors onto the hypersphere surface. The formula is as follows: ； Step 2.2: Further refine the relationship mapping, and divide the relationships into , and , where and are the projection parts of the head entity and the tail entity of the relationship pair respectively, is the translation part of the relationship. Introduce the parameterized vector e and normalize the parameters, and set the scaling factor . Define two relationship mapping functions and as: ； ； Step 2.3: For each triple , define the scoring function as: ； Among them, is the scoring of the triple by the scoring function scoring, represents the Hadamard product.
4. The knowledge graph embedding method based on parametric vector offset according to claim 1, wherein The specific content of Step 3 is as follows: Step 3.1: Define the self-adversarial negative sampling loss function : ； Among them, is a function, which is the th artificial negative sample constructed in knowledge graph embedding training, where is used to distinguish different negative samples, represents the weight of this negative sample, specifically expressed as: ； After presenting a given true triple negative samples are generated The weights in the scoring function and are the scoring function values of the negative samples and respectively, representing the prediction results of the scoring function for different negative samples. α represents the sampling temperature, which controls the weight distribution among the negative samples; Step 3.2: Load the training set data, generate multiple negative samples for each positive sample, calculate the predicted scores of the positive samples and negative samples through forward propagation, and calculate the total loss using the self-adversarial negative sampling loss function; Step 3.3: Evaluate the model performance. When the model performance continues to improve, update the model parameters. If there is no improvement for a continuous preset number of rounds, stop in advance and save the current optimal model parameters.
5. The knowledge graph embedding method based on parameterized vector offset according to claim 1, wherein The specific content of Step 4 is as follows: Step 4.1: Load the trained model, which includes a scoring function and embedding vectors corresponding to entities and relationships respectively. Subsequently, receive or generate query triples containing missing entities or relationships for the model to make predictions and score; Step 4.2: For each query triple, generate all candidate entities based on the knowledge graph or the set of entities and relationships. Use the scoring function in the loaded model to calculate the scores of each candidate entity as the missing part, and combine the similarity or distance between the embedding vectors corresponding to the candidate entity and the known part to obtain the rationality evaluation result of each candidate entity; Step 4.3: According to the rationality evaluation results of each candidate entity, sort them from high to low according to the scores, and select the candidate entity with the highest score as the final prediction result.