A weight-aware performance evaluation method for knowledge graph embedding

By constructing a weight-aware performance evaluation method, and evaluating knowledge graph embedding using the weight-aware link prediction model, the problem of lack of distinguishing triple weights in the existing technology is solved, and the accuracy and information content of the evaluation are improved.

CN118674023BActive Publication Date: 2024-11-26PLA DALIAN NAVAL ACADEMY +1
View PDF 0 Cites 1 Cited by

Patent Information

Application Number
CN202411170646.4
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2024-08-26
Publication Date
2024-11-26
Estimated Expiration
2044-08-26

AI Technical Summary

Technical Problem

Existing knowledge graph embedding evaluation methods lack the ability to distinguish weighted triplet weights, resulting in poor evaluation because the importance of triplets varies significantly according to their weights, but existing methods consider all triplets to be equally important.

Method used

A weight-aware performance evaluation method for knowledge graph embedding is proposed. By obtaining undamaged weighted triples from the complete weighted knowledge graph, deleting and replacing the head and tail entities, the damaged triples are constructed, and the pre-trained weights are used. The perceived link prediction model evaluates the test set and uses weight-aware ranking and classification performance indicators for performance evaluation.

Benefits of technology

By introducing a weight perception mechanism, the information content of the knowledge graph embedding model is improved, so that the weight perception link prediction model can learn knowledge graph structure information more effectively and accurately evaluate the knowledge graph embedding performance.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN118674023B_ABST
    Figure CN118674023B_ABST
Patent Text Reader

Abstract

The present application provides a method for evaluating the weight-aware performance of knowledge graph embedding, which specifically relates to the technical field of knowledge graph link prediction evaluation, obtaining an undamaged weighted triple as the first triple to be tested; deleting the head entity or tail entity of the first triple to be tested, and replacing it with each entity in the entity dictionary, to obtain multiple first damaged triples or multiple second damaged triples; using the first triple to be tested, multiple first damaged triples, and multiple second damaged triples as a test set; using the test set to evaluate the weight-aware link prediction performance of the weight-aware link prediction model. By introducing weights into the weight-related model, the weight perception of the triples and the learning ability of the knowledge graph embedding model are fully utilized, the information content encoded by the knowledge graph embedding model is improved, and the weight-aware link prediction model can learn the knowledge graph structure information more effectively.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the technical field of knowledge graph link prediction evaluation, and in particular to a weight-aware performance evaluation method for knowledge graph embedding. Background Art

[0002] Knowledge graph embedding maps entities and relations in a knowledge graph into a continuous vector space to achieve efficient and meaningful representation learning of structured information. Currently, two basic tasks, link prediction and triple classification, are used to evaluate the effectiveness of knowledge graph embedding.

[0003] Since the above evaluation task assumes that all triples to be predicted are of equal importance regardless of the weight of the triple to be tested, this may be disadvantageous in many real-world scenarios because the importance of triples varies significantly according to their weights, making the above evaluation task lack the ability to distinguish the weights of triples in weighted knowledge graphs, resulting in poor evaluation results.

[0004] Based on this, this application proposes a weight-aware performance evaluation method for knowledge graph embedding to solve the above problems. Summary of the invention

[0005] This application provides a weight-aware performance evaluation method for knowledge graph embedding to solve the above problems.

[0006] In a first aspect of an embodiment of the present application, a weight-aware performance evaluation method for knowledge graph embedding is proposed, the method comprising:

[0007] Obtaining an undamaged weighted triple from the complete weighted knowledge graph as the first triple to be tested;

[0008] Deleting the head entity of the first triple to be tested, and replacing it with each entity in the entity dictionary in turn, to obtain multiple first damaged triples;

[0009] The tail entity of the first triple to be tested is deleted, and replaced with each entity in the entity dictionary in turn, to obtain a plurality of second damaged triples;

[0010] Taking the first to-be-tested triple, a plurality of the first damaged triples, and a plurality of the second damaged triples as a test set;

[0011] The test set is used to evaluate the weight-aware link prediction performance of a pre-trained weight-aware link prediction model, wherein the pre-trained weight-aware link prediction model is obtained by training with positive weighted triplets, negative weighted triplets, weight labels, and positive and negative labels as training samples.

[0012] In an optional embodiment of the present application, the weight-aware link prediction model is trained according to the following steps:

[0013] Preprocessing the head entity, the relationship, and the tail entity of the training sample to obtain the preprocessed head entity, the relationship, and the tail entity;

[0014] Performing operations on the preprocessed head entity, relationship, and tail entity to obtain an interaction vector;

[0015] Performing link prediction on the interaction vector after taking the modulus of the interaction vector using the base model to be trained, and obtaining a link prediction score for the training sample;

[0016] Calculating a loss value of the base model to be trained based on the link prediction score of the training sample and the positive and negative labels;

[0017] Updating the loss value based on the weight label to obtain an updated loss value;

[0018] The updated loss value is used to optimize the parameters of the base model to be trained to obtain the weight-aware link prediction model.

[0019] In an optional embodiment of the present application, the evaluating the weight-aware link prediction performance of the pre-trained weight-aware link prediction model using the test set includes:

[0020] Obtaining a predicted ranking of the first triple to be tested, where the predicted ranking of the first triple to be tested is used to represent a ranking of a link prediction score of the first triple to be tested;

[0021] The predicted ranking is adjusted using an activation function and a true weight of the first triplet to be tested to obtain a weight-aware ranking;

[0022] Based on the performance evaluation index and the weight-aware ranking, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is evaluated.

[0023] In an optional embodiment of the present application, obtaining the predicted ranking of the first triple to be tested includes:

[0024] Using the pre-trained weight-aware link prediction model to perform link prediction on the first triple to be tested and the plurality of first damaged triples, to obtain link prediction scores for the first triple to be tested and the plurality of first damaged triples;

[0025] sorting the link prediction scores of the first triple to be tested and the plurality of the first damaged triples, and recording the first ranking of the first triple to be tested;

[0026] Using the pre-trained weight-aware link prediction model to perform link prediction on the first triple to be tested and the plurality of second damaged triples, to obtain link prediction scores for the first triple to be tested and the plurality of second damaged triples;

[0027] sorting the link prediction scores of the first triple to be tested and the plurality of the second damaged triples, and recording the second ranking of the first triple to be tested;

[0028] The first ranking and the second ranking are used as predicted rankings of the triplet to be tested.

[0029] In an optional embodiment of the present application, the performance evaluation indicators include: weight-perceived average ranking, weight-perceived average reciprocal ranking and weight-perceived average proportion;

[0030] The weight-aware average ranking is used to calculate the average of the weight-aware rankings of all first triples to be tested, the weight-aware average reciprocal ranking is used to calculate the average of the weight-aware reciprocal rankings of all first triples to be tested, and the weight-aware average proportion is used to calculate the proportion of the first triples to be tested ranked in the top N.

[0031] In an optional embodiment of the present application, the method further includes:

[0032] Obtaining an undamaged weighted triple from the complete weighted knowledge graph as the second triple to be tested;

[0033] Deleting the head entity of the second triple to be tested, and replacing it with each entity in the entity dictionary in turn, to obtain a plurality of third damaged triples;

[0034] Mixing the second triple to be tested and a plurality of the third damaged triples as a mixed set of head damaged triples;

[0035] The tail entity of the second triple to be tested is deleted, and replaced with each entity in the entity dictionary in turn, to obtain a plurality of fourth damaged triples;

[0036] Mixing the second triple to be tested and a plurality of the fourth damaged triples as a tail damaged triple mixed set;

[0037] The mixed set of head damaged triplets and the mixed set of tail damaged triplets are used as mixed test sets, and based on the mixed test sets, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is evaluated.

[0038] In an optional embodiment of the present application, the evaluating the weight-aware link prediction performance of the pre-trained weight-aware link prediction model based on the mixed test set includes:

[0039] Using a pre-trained weight-aware link prediction model to perform link prediction on each triple in the mixed test set to obtain a link prediction score for each triple in the mixed test set;

[0040] Based on the link prediction score of each triple and a preset threshold, the triples in the mixed test set are divided into a positive set and a negative set to obtain a prediction classification result of each triple;

[0041] Based on the predicted classification results and positive and negative labels of each triplet, dividing each triplet into a true positive set, a false positive set and a false negative set;

[0042] Based on the true positive set, the false positive set and the false negative set, a classification performance evaluation index is used to evaluate the weight-aware link prediction performance of the pre-trained weight-aware link prediction model.

[0043] In an optional embodiment of the present application, the classification performance evaluation indicators include: weight-aware precision, weight-aware recall and weight-aware F1 score;

[0044] The weight-aware precision is used to calculate the precision of the second triplet to be tested in each category on the corresponding category, the weight-aware recall is used to calculate the recall of the second triplet to be tested in each category on the corresponding category, and the weight-aware F1 score is used to comprehensively measure the weight-aware precision and the weight-aware recall.

[0045] In a second aspect of an embodiment of the present application, a weight-aware performance evaluation system for knowledge graph embedding is proposed, the system comprising:

[0046] A first acquisition module, used for acquiring an intact weighted triple from the complete weighted knowledge graph as a first triple to be tested;

[0047] A first damaged triple construction module is used to delete the head entity of the first triple to be tested and replace it with each entity in the entity dictionary in turn to obtain multiple first damaged triples;

[0048] A second damaged triple construction module is used to delete the tail entity of the first triple to be tested and replace it with each entity in the entity dictionary in turn to obtain multiple second damaged triples;

[0049] A test set construction module, configured to use the first to-be-tested triple, a plurality of the first damaged triples, and a plurality of the second damaged triples as a test set;

[0050] The first evaluation module is used to evaluate the weight-aware link prediction performance of a pre-trained weight-aware link prediction model using the test set, wherein the pre-trained weight-aware link prediction model is obtained by training with positive weighted triplets, negative weighted triplets, weight labels and positive and negative labels as training samples.

[0051] In a third aspect of an embodiment of the present application, an electronic device is proposed, comprising: a memory for storing one or more programs; a processor; when the one or more programs are executed by the processor, a weight-aware performance evaluation method for knowledge graph embedding as described in any one of the first aspects above is implemented.

[0052] The present application includes the following advantages: The present application embodiment provides a weight-aware performance evaluation method for knowledge graph embedding, by obtaining an undamaged weighted triple from a complete weighted knowledge graph as the first triple to be tested; deleting the head entity of the first triple to be tested and replacing it with each entity in the entity dictionary in turn, obtaining multiple first damaged triples; deleting the tail entity of the first triple to be tested and replacing it with each entity in the entity dictionary in turn, obtaining multiple second damaged triples; using the first triple to be tested, multiple first damaged triples and multiple second damaged triples as a test set; using the test set to evaluate the weight-aware link prediction performance of a pre-trained weight-aware link prediction model, the pre-trained weight-aware link prediction model is obtained by training with positive weighted triples, negative weighted triples, weight labels and positive and negative labels as training samples. By introducing weights into the weight-related model, the weight perception of triples and the learning ability of the knowledge graph embedding model are fully utilized, the information content encoded by the knowledge graph embedding model is improved, and the weight-aware link prediction model can learn the knowledge graph structure information more effectively. BRIEF DESCRIPTION OF THE DRAWINGS

[0053] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative labor.

[0054] Figure 1 It is a schematic diagram of the steps of a weight-aware performance evaluation method for knowledge graph embedding provided in an embodiment of the present application;

[0055] Figure 2 It is an architecture diagram of a knowledge graph embedded weight-aware performance evaluation system provided in an embodiment of the present application;

[0056] Figure 3 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0057] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0058] Knowledge graphs store real-world knowledge in the form of graphs, promoting the advancement of artificial intelligence. Facts encoded in knowledge graphs are mostly formalized as a set of triples (h, r, t), where h represents the head entity, t represents the tail entity, and r represents the relationship between h and t. The rapid development of knowledge graphs has led to their wide application in various real-world scenarios, such as information retrieval, question answering, recommendation systems, and specific domain tasks.

[0059] Knowledge graph embedding maps entities and relations in a knowledge graph into a continuous vector space to enable efficient and meaningful representation learning of structured information. Two basic tasks, link prediction and triple classification, are widely used to evaluate the effectiveness of knowledge graph embeddings. In link prediction, the goal is to predict missing entities in incomplete triples, such as (h, r, ?) or (?, r, t). By completing triples with missing entities, link prediction aims to infer previously unknown triples in the knowledge graph. On the other hand, triple classification involves determining the true value of an unseen triple (h, r, t). The task involves classifying whether the triple is valid and corresponds to real-world facts, or is wrong and does not hold in the real world. Both link prediction and triple classification are key evaluation tasks for the quality of knowledge graph embeddings because they evaluate the ability of embeddings to capture facts and meaningful patterns in the graph.

[0060] In the real world, facts are not discrete items that can be simply classified as true (1) or false (0). Instead, people assign different weights to different facts and pay more attention to those that are considered more important. However, deterministic knowledge graphs attach equal importance to all facts, limiting the expressive power of knowledge graph embedding models. For example, two triples about Mr. Jia: (Mr. Jia, identity, principal) and (Mr. Jia, book title, Ding XX). The former represents Mr. Jia’s identity as a principal and is more important in learning the embedding of “Mr. Jia” than the latter, which involves a book title that is irrelevant to most people. Although Mr. Jia’s book title is Ding XX, it has little significance in the context of knowledge graph representation learning.

[0061] Weighted knowledge graphs extend the concept of deterministic knowledge graphs by associating weights with each triple. This formalism is valuable in representing uncertainty, confidence scores, degree of relationships, edge importance, and even out-of-band knowledge in an increasing number of scenarios.

[0062] Currently used evaluation tasks, such as link prediction and triple classification, lack the ability to distinguish the weights of triples in weighted knowledge graphs, resulting in poor evaluation results. The reason is that these evaluation tasks consider all triples to be predicted to be equally important regardless of the weight of the triple to be tested, which may be disadvantageous in many real-world scenarios because the importance of triples varies significantly depending on their weights.

[0063] Based on this, this application proposes to distinguish triplets based on weights to accurately evaluate the performance of the knowledge graph embedding model.

[0064] In the first aspect of the embodiment of the present application, a weight-aware performance evaluation method for knowledge graph embedding is proposed, see Figure 1 , Figure 1 A flowchart of a method for weight-aware performance evaluation of knowledge graph embedding provided in an embodiment of the present application, the method comprising:

[0065] Step 101: Obtain an undamaged weighted triple from the complete weighted knowledge graph as the first triple to be tested;

[0066] Step 102: Delete the head entity of the first triple to be tested, and replace it with each entity in the entity dictionary in turn, to obtain multiple first damaged triples;

[0067] Step 103: deleting the tail entity of the first triple to be tested, and replacing it with each entity in the entity dictionary in turn, to obtain a plurality of second damaged triples;

[0068] Step 104: taking the first to-be-tested triple, a plurality of the first damaged triples, and a plurality of the second damaged triples as a test set;

[0069] Step 105: Use the test set to evaluate the weight-aware link prediction performance of a pre-trained weight-aware link prediction model, where the pre-trained weight-aware link prediction model is trained using positive weighted triplets, negative weighted triplets, weight labels, and positive and negative labels as training samples.

[0070] When step 101 is specifically implemented, an intact weighted triple is obtained from the complete weighted knowledge graph as the first triple to be tested for testing. The weighted knowledge graph WKG can be expressed as , u is the number of triples in the weighted knowledge graph, where , and are entity sets and relationship sets, respectively, i Represents the head entity, t i Represents the tail entity, r i Indicates h i and t i The relationship between i Represents the weight of the first triplet to be tested.

[0071] When step 102 is specifically implemented, for each first triple to be tested, the weight w i During the test, the first entity h of the first triple to be tested is omitted. i After deletion, it can be expressed as (?, r i , t i ), and then replace it with each entity e in the entity dictionary in turn, and obtain multiple possible first damaged triples, which can be expressed as (e, r i , t i ).

[0072] When step 103 is specifically implemented, the tail entity t of the first triple to be tested is i After deletion, it can be expressed as (h i , r i ,?), and then replaced with each entity e in the entity dictionary in turn, to obtain multiple possible second damaged triples, which can be expressed as (h i , r i , e).

[0073] When step 104 is specifically implemented, the first triple to be tested (h i , r i , t i ), a plurality of the first damaged triples (e, r i , t i ) and a plurality of said second damaged triplets (h i , r i , e) as a test set, and the constructed test set is used for subsequent performance evaluation of the weight-aware link prediction model.

[0074] When step 105 is specifically implemented, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is evaluated by the composed test set, and the pre-trained weight-aware link prediction model is obtained by training with positive weighted triples, negative weighted triples, weight labels and positive and negative labels as training samples. Triples include relationship triples and attribute triples. For example, (apple, is, fruit) is a relationship triple, in which "apple" and "fruit" are entities, and "is" is the relationship between them. (apple, color, red) is an attribute triple, in which "apple" is an entity, "color" is an attribute, and "red" is the value of the attribute. In an embodiment of the present application, a positive triple refers to a real relationship or attribute, and a negative triple refers to a non-existent relationship or attribute.

[0075] In an optional embodiment of the present application, the weight-aware link prediction model is trained according to the following steps:

[0076] Step 201: preprocessing the head entity, relationship and tail entity of the training sample to obtain the preprocessed head entity, relationship and tail entity;

[0077] Step 202: Calculate the preprocessed head entity, relationship and tail entity to obtain an interaction vector;

[0078] Step 203: performing link prediction on the interaction vector after taking the modulus of the base model to be trained to obtain a link prediction score of the training sample;

[0079] Step 204: Calculate the loss value of the base model to be trained based on the link prediction score of the training sample and the positive and negative labels;

[0080] Step 205: updating the loss value based on the weight label to obtain an updated loss value;

[0081] Step 206: Optimize the parameters of the base model to be trained using the updated loss value to obtain the weight-aware link prediction model.

[0082] When step 201 is specifically implemented, since the head entity h, relation r and tail entity t in the triple are not in the same plane, the above preprocessing aims to vertically project the head entity h, relation r and tail entity t so that the three are in the same plane.

[0083] When step 202 is specifically implemented, after preprocessing, the operations performed on the preprocessed head entity, relationship, and tail entity can be multiplied by a bilinear model to obtain an interaction vector, or can be added by a translation distance model to obtain an interaction vector. In an embodiment of the present application, the translation distance model can be TransH, a translation distance model TransE, and a semantic matching model DistMult or HolE can also be applied here.

[0084] When step 203 is specifically implemented, the interaction vector is modulo the base model to be trained to perform link prediction, and a link prediction score of the training sample is obtained. The link prediction score is a score assigned to different training samples by a scoring function in the base model. The link prediction score represents the rationality of the existence of the triple. The scoring function can be expressed as f(h, r, t), and the score corresponding to each training sample is obtained by the scoring function.

[0085] When step 204 is specifically implemented, the loss value of the base model to be trained is calculated through the link prediction score of each training sample and its corresponding positive and negative labels. The loss value is obtained for the training sample and does not include the loss of weight calculation.

[0086] When implementing step 205, the above loss value is updated by the weight label of each training sample to obtain the weighted loss value. This method is to calculate the weighted loss value through the activation function g() and weight it with the loss value of the base model to be trained to obtain the updated loss value. At this time, the scoring function in the base model to be trained is updated to a weight-aware scoring function, which can be expressed as f w (h, r, t, w), the calculation method of the updated scoring function can be expressed as f w (h, r, t, w) = g (w) · f (h, r, t).

[0087] When step 206 is specifically implemented, the parameters in the base model to be trained are optimized by the loss value after the weight update to obtain the weight-aware link prediction model. After the loss value is updated by the weight of the triple, the base model to be trained can use the weight of the triple during the training process, so that the weight-aware link prediction model obtained by the training can obtain accurate scores. By performing a weight-aware activation function on the non-weight-related knowledge graph embedding model, the weight of the triple and the learning ability of the knowledge graph embedding model are fully utilized, and the information content encoded in the knowledge graph embedding model is improved, so that it can be further applied to the weight-related model. At the same time, after the weight-aware link prediction model is obtained, in the stage of applying it, the weight-aware link prediction model, like the base model, does not require the weight of the unknown triple, and will not bring additional data burden.

[0088] In an optional embodiment of the present application, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is evaluated using the above test set, specifically comprising the following steps:

[0089] Step 301: obtaining a predicted ranking of the first triple to be tested, where the predicted ranking of the first triple to be tested is used to represent a ranking of a link prediction score of the first triple to be tested;

[0090] Step 302: adjusting the predicted ranking by using an activation function and the true weight of the first triplet to be tested to obtain a weight-aware ranking;

[0091] Step 303: Based on the performance evaluation index and the weight-aware ranking, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is evaluated.

[0092] When step 301 is specifically implemented, when the performance of the pre-trained weight-aware link prediction model is evaluated, the predicted ranking of the first triple to be tested is obtained. The predicted ranking gives the link prediction score of the first triple to be tested through the scoring function of the weight-aware link prediction model. The predicted ranking is the ranking of the link prediction score of the first triple to be tested, and can also be used to characterize the rationality of the existence of the first triple to be tested. When the link prediction score is closer to 1, the relationship of the corresponding first triple to be tested is more reasonable, and when the link prediction score is closer to 0, the relationship of the corresponding first triple to be tested is more unreasonable.

[0093] Specifically, the embodiment of the present application obtains the predicted ranking of the first triple to be tested in the following manner: first, a pre-trained weight-aware link prediction model is used to perform link prediction on the first triple to be tested and multiple first damaged triples, and the link prediction scores of the first triple to be tested and the multiple first damaged triples are obtained; then, the link prediction scores of the first triple to be tested and the multiple first damaged triples are sorted in ascending order, and the ranking of the first triple to be tested in the sorting of the multiple first damaged triples is recorded as the first ranking. Similarly, a pre-trained weight-aware link prediction model is used to perform link prediction on the first triple to be tested and multiple second damaged triples, and the link prediction scores of the first triple to be tested and the multiple second damaged triples are obtained; then, the link prediction scores of the first triple to be tested and the multiple second damaged triples are sorted in ascending order, and the ranking of the first triple to be tested in the sorting of the multiple second damaged triples is recorded as the second ranking. The above first ranking and second ranking are used as the predicted ranking of the first triple to be tested.

[0094] When specifically implementing step 302, after obtaining the predicted ranking of the first triple to be tested, in order to provide a higher evaluation for models that perform better on triples with higher weights, the activation function g() is applied to adjust the ranking according to the weight of the first test triple, thereby obtaining the weight-perceived ranking of the first test triple. The predicted ranking is adjusted using the activation function and the true weight of the first triple to be tested to obtain the weight-perceived ranking. It is assumed here that the activation function is monotonically increasing, that is, it is assumed that when the weight of the triple is higher, the corresponding importance of the triple is also higher. If the importance of the triple is higher, the corresponding loss value of the triple will also be greater, and the loss value of the triple will have a greater impact on the overall loss value. Weight-perceived ranking of the first test triple rk i w It can be expressed as:

[0095] ,in, rk i is the predicted ranking of the first triple to be tested, g() is the activation function, and w i is the weight of the i-th first triple to be tested.

[0096] When step 303 is specifically implemented, after the weight-aware ranking is obtained, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is evaluated using a performance evaluation index combined with the weight-aware ranking.

[0097] In the embodiment of the present application, the above-mentioned performance evaluation indicators include weight-aware average ranking, weight-aware average reciprocal ranking and weight-aware average proportion; wherein the weight-aware average ranking is used to calculate the average of the weight-aware rankings of all first triples to be tested, and the calculation formula of the weight-aware average ranking WaMR is as follows:

[0098] ,

[0099] The weight-aware average reciprocal ranking is used to calculate the average of the weight-aware reciprocal rankings of all first triples to be tested. The calculation formula of the weight-aware average reciprocal ranking WaMRR is as follows:

[0100] ,

[0101] The weighted-aware average proportion is used to calculate the proportion of the first triple to be tested in the top N. The calculation formula of the weighted-aware average proportion WaHits@N is as follows:

[0102] ,

[0103] Where i represents the i-th first triple to be tested, u represents the number of first triples to be tested, rk i w represents the weight-aware ranking of the i-th first triple to be tested, represents the indicator function, N represents the Nth ranking, represents the normalization factor, j represents the jth first triple to be tested, g() represents the activation function, w j represents the weight of the jth first triple to be tested.

[0104] Compared with weight-independent performance evaluation indicators, the three weight-aware performance evaluation indicators introduced above can comprehensively evaluate the weight-aware link prediction performance of the weight-aware link prediction model according to the weights of the triples, and can evaluate the rationality of the existence of triples in the weighted knowledge graph.

[0105] The model gain to be evaluated in the weight-aware triplet classification task depends not only on the classification results of the model, but also on the weights associated with the real triples in the test. The task involves a set of triplets, which include positive and negative triplets. The goal of the task is to classify positive and negative triplets taking into account the weights of the positive triples. The gain of classifying the triplets is calculated based on the classification results and the weights of the triples.

[0106] Specifically, the test evaluation method of the above-mentioned weight-aware triple classification task includes:

[0107] In an optional embodiment of the present application, the method further includes classifying weight-aware triples using a pre-trained weight-aware link prediction model, and the test evaluation method for the weight-aware link prediction model includes the following steps:

[0108] Step 401: Obtaining an intact weighted triple from the complete weighted knowledge graph as a second triple to be tested;

[0109] Step 402: Delete the head entity of the second triple to be tested, and replace it with each entity in the entity dictionary in turn, to obtain a plurality of third damaged triples;

[0110] Step 403: Mix the second to-be-tested triplet and a plurality of the third damaged triples to form a mixed set of damaged head triples;

[0111] Step 404: Delete the tail entity of the second triple to be tested, and replace it with each entity in the entity dictionary in turn, to obtain a plurality of fourth damaged triples;

[0112] Step 405: Mix the second to-be-tested triple and a plurality of the fourth damaged triples to form a mixed set of tail damaged triples;

[0113] Step 406: Use the mixed set of damaged head triplets and the mixed set of damaged tail triplets as mixed test sets, and evaluate the weight-aware link prediction performance of the pre-trained weight-aware link prediction model based on the mixed test sets.

[0114] The specific method for implementing step 401 is the same as the method for implementing step 101 , and the details can be found in the implementation of the above step 101 .

[0115] Specifically, when step 402 is implemented, the head entity h of the second triple to be tested is i The damaged triples with damaged heads are deleted and the head entities of the damaged triples with damaged heads are replaced by k entities in the dictionary, thereby generating multiple third damaged triples with damaged heads, which can be expressed as (φ, r i , t i ).

[0116] When step 403 is specifically implemented, in order to obtain a mixed set of head-damaged triples, the second triple to be tested and the corresponding plurality of third damaged triples with damaged heads are mixed to obtain a mixed set of head-damaged triples.

[0117] When step 404 is implemented, the tail entity t of the second triple to be tested is i The damaged triples with damaged tails are deleted and the tail entities of the damaged triples with damaged tails are replaced by k entities in the dictionary, thereby generating multiple fourth damaged triples with damaged tails, which can be expressed as (h i ,ri,φ).

[0118] Specifically, when step 405 is implemented, in order to obtain a mixed set of triples with damaged tails, the second triple to be tested and a plurality of corresponding fourth damaged triples with damaged tails are mixed to obtain a mixed set of triples with damaged tails.

[0119] When step 406 is specifically implemented, a mixed set of head damaged triples and a mixed set of tail damaged triples are mixed as a mixed test set, and based on the constructed mixed test set, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is evaluated.

[0120] Specifically, based on a mixed test set, a method for evaluating the weight-aware link prediction performance of a pre-trained weight-aware link prediction model comprises: first, using a pre-trained weight-aware link prediction model to perform link prediction on each triple in the mixed test set, and obtaining a link prediction score for each triple in the mixed test set. Then, based on the link prediction score of each triple and a preset threshold, the triples in the mixed test set are divided into a positive set and a negative set to obtain a prediction classification result of the triple to be tested. Exemplarily, the preset threshold is represented as τ, which can be set to 0.8. When the link prediction score of the triple is greater than or equal to 0.8, the triple is divided into a positive set, indicating that the triple is predicted to be true, and when the link prediction score of the triple is less than 0.8, the triple is divided into a negative set, indicating that the triple is predicted to be false.

[0121] Then, each triplet is divided into a true positive set, a false positive set, and a false negative set according to the predicted classification results and positive and negative labels of each triplet. TP ) is composed of true triplets classified as positive, that is, when the predicted classification result of the triplet is true (positive set) and the positive and negative labels of the triplet are positive, the triplet is classified as true positive (TP) and is divided into the true positive set. FP ) is composed of false triplets classified as positive, that is, when the predicted classification result of the triple is true (positive set) and the positive and negative labels of the triple are negative, the triple is classified as a false positive (false positive, FP) and is divided into the false positive set. FN ) is composed of true triplets classified as negative, that is, when the predicted classification result of the triplet is false (negative set) and the positive and negative labels of the triplet are positive, the triplet is classified as a false negative (FN) and is divided into the false negative set.

[0122] The predicted classification result of predicting a triple as true or false may not match the actual positive and negative labels of the triple, that is, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is inaccurate. Only when the predicted classification result of the triple has a high degree of match with the actual positive and negative labels of the triple, it is considered that the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is more accurate, and the pre-trained weight-aware link prediction model has the ability to infer the authenticity of the triple according to the triple weight. Therefore, based on the true positive set, the false positive set and the false negative set, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model can be evaluated using classification performance evaluation indicators.

[0123] The classification performance evaluation indicators used in the above evaluation include: weight-aware precision, weight-aware recall and weight-aware F1 score; the weight-aware precision is used to calculate the precision of the second test triplet of each category in the corresponding category. w prec The calculation formula is as follows:

[0124] ,

[0125] The weight-aware recall rate is used to calculate the recall rate of the second test triplet of each category in the corresponding category. w recall The calculation formula is as follows:

[0126] ,

[0127] The weight-aware F1 score is used to comprehensively measure the weight-aware precision and the weight-aware recall, and the weight-aware F1 score can provide a greater reward for a weight-aware link prediction model that correctly classifies triples with more high weights. The calculation formula of the weight-aware F1 score WaF1 is as follows:

[0128] ,

[0129] Where i represents the i-th second triple to be tested, (h i , r i , t i ) represents the i-th second triple to be tested, 〈(h i , r i , t i ),?〉∈TP means that the second triple to be tested belongs to the true positive set, 〈(h i , r i , t i ),?〉∈FP indicates that the second triple to be tested belongs to the false positive set, 〈(h i , r i , t i ),?〉∈FN indicates that the second triple to be tested belongs to the false negative set, c represents the normalization factor, g() represents the activation function, and w i represents the weight of the i-th second triple to be tested.

[0130] The embodiment of the present application provides a weight-aware performance evaluation method for knowledge graph embedding, by obtaining an undamaged weighted triple from a complete weighted knowledge graph as the first triple to be tested; deleting the head entity of the first triple to be tested, and replacing it with each entity in the entity dictionary in turn, to obtain multiple first damaged triples; deleting the tail entity of the first triple to be tested, and replacing it with each entity in the entity dictionary in turn, to obtain multiple second damaged triples; using the first triple to be tested, multiple first damaged triples and multiple second damaged triples as a test set; using the test set to evaluate the weight-aware link prediction performance of a pre-trained weight-aware link prediction model, the pre-trained weight-aware link prediction model is obtained by training with positive weighted triples, negative weighted triples, weight labels and positive and negative labels as training samples. By introducing weights into the weight-related model, the weight perception of triples and the learning ability of the knowledge graph embedding model are fully utilized, the information content encoded by the knowledge graph embedding model is improved, and the weight-aware link prediction model can learn the knowledge graph structure information more effectively.

[0131] In a second aspect of the present application, a weight-aware performance evaluation system for knowledge graph embedding is provided. Figure 2 , Figure 2 An architecture diagram of a knowledge graph embedded weight-aware performance evaluation system provided in an embodiment of the present application, the system comprising:

[0132] A first acquisition module 210, configured to acquire an intact weighted triple from the complete weighted knowledge graph as a first triple to be tested;

[0133] A first damaged triple construction module 220 is used to delete the head entity of the first triple to be tested and replace it with each entity in the entity dictionary in turn to obtain multiple first damaged triples;

[0134] A second damaged triple construction module 230 is used to delete the tail entity of the first triple to be tested and replace it with each entity in the entity dictionary in turn to obtain multiple second damaged triples;

[0135] A test set construction module 240, configured to use the first to-be-tested triple, a plurality of the first damaged triples, and a plurality of the second damaged triples as a test set;

[0136] The first evaluation module 250 is used to evaluate the weight-aware link prediction performance of a pre-trained weight-aware link prediction model using the test set, wherein the pre-trained weight-aware link prediction model is obtained by training with positive weighted triplets, negative weighted triplets, weight labels and positive and negative labels as training samples.

[0137] Wherein, the first evaluation module includes a training submodule, and the training submodule includes:

[0138] A preprocessing unit, used to preprocess the head entity, relationship and tail entity of the training sample to obtain the preprocessed head entity, relationship and tail entity;

[0139] A computing unit, used for computing the preprocessed head entity, relationship and tail entity to obtain an interaction vector;

[0140] A link prediction unit, configured to perform link prediction on the interaction vector by taking a modulus from a base model to be trained, and obtain a link prediction score for the training sample;

[0141] A calculation unit, used to calculate the loss value of the base model to be trained based on the link prediction score of the training sample and the positive and negative labels;

[0142] An updating unit, configured to update the loss value based on the weight label to obtain an updated loss value;

[0143] An optimization unit is used to optimize the parameters of the base model to be trained using the updated loss value to obtain the weight-aware link prediction model.

[0144] Wherein, the first evaluation module further includes:

[0145] A prediction ranking acquisition submodule, used to acquire the prediction ranking of the first triple to be tested, where the prediction ranking of the first triple to be tested is used to characterize the ranking of the link prediction score of the first triple to be tested;

[0146] A ranking adjustment submodule, used to adjust the predicted ranking by using an activation function and a true weight of the first triple to be tested to obtain a weight-aware ranking;

[0147] An evaluation submodule is used to evaluate the weight-aware link prediction performance of the pre-trained weight-aware link prediction model based on a performance evaluation index and the weight-aware ranking.

[0148] The predicted ranking acquisition submodule includes:

[0149] a first link prediction unit, configured to perform link prediction on the first to-be-tested triple and the plurality of first damaged triples by using the pre-trained weight-aware link prediction model, and obtain link prediction scores for the first to-be-tested triple and the plurality of first damaged triples;

[0150] A first recording unit, configured to sort the link prediction scores of the first triple to be tested and the plurality of the first damaged triples, and record a first ranking of the first triple to be tested;

[0151] A second link prediction unit, configured to perform link prediction on the first triple to be tested and the plurality of second damaged triples by using the pre-trained weight-aware link prediction model, and obtain link prediction scores for the first triple to be tested and the plurality of second damaged triples;

[0152] A second recording unit is used to sort the link prediction scores of the first triple to be tested and the plurality of the second damaged triples, and record a second ranking of the first triple to be tested;

[0153] A predicted ranking determining unit is used to use the first ranking and the second ranking as the predicted rankings of the triplet to be tested.

[0154] Among them, the performance evaluation indicators in the evaluation submodule include weight-aware average ranking, weight-aware average reciprocal ranking and weight-aware average proportion; the weight-aware average ranking is used to calculate the average value of the weight-aware rankings of all first triples to be tested, the weight-aware average reciprocal ranking is used to calculate the average value of the weight-aware reciprocal rankings of all first triples to be tested, and the weight-aware average proportion is used to calculate the proportion of the first triples to be tested ranked in the top N.

[0155] Wherein, the system further comprises:

[0156] A second acquisition module is used to acquire an undamaged weighted triple from the complete weighted knowledge graph as a second triple to be tested;

[0157] A third damaged triple construction module is used to delete the head entity of the second triple to be tested and replace it with each entity in the entity dictionary in turn to obtain multiple third damaged triples;

[0158] a head damaged triple mixed set building module, used for mixing the second triple to be tested and a plurality of the third damaged triples as a head damaged triple mixed set;

[0159] a fourth damaged triple construction module, configured to delete the tail entity of the second triple to be tested, and replace it with each entity in the entity dictionary in turn, to obtain a plurality of fourth damaged triples;

[0160] a tail damaged triple mixed set construction module, configured to mix the second triple to be tested and a plurality of the fourth damaged triples as a tail damaged triple mixed set;

[0161] The second evaluation module is used to use the mixed set of head damaged triples and the mixed set of tail damaged triples as mixed test sets, and evaluate the weight-aware link prediction performance of the pre-trained weight-aware link prediction model based on the mixed test sets.

[0162] Wherein, the second evaluation module comprises:

[0163] A link prediction score determination submodule, used to use a pre-trained weight-aware link prediction model to perform link prediction on each triple in the mixed test set to obtain a link prediction score for each triple in the mixed test set;

[0164] A prediction classification result determination submodule, used for dividing the triples in the mixed test set into a positive set and a negative set based on the link prediction score of each triple and a preset threshold, to obtain a prediction classification result of each triple;

[0165] A partitioning submodule, for partitioning each triple into a true positive set, a false positive set, and a false negative set based on the predicted classification result and the positive and negative labels of each triple;

[0166] The classification performance evaluation submodule is used to evaluate the weight-aware link prediction performance of the pre-trained weight-aware link prediction model using a classification performance evaluation index based on the true positive set, the false positive set and the false negative set.

[0167] Among them, the classification performance evaluation indicators in the classification performance evaluation submodule include: weight-aware precision, weight-aware recall rate and weight-aware F1 score; the weight-aware precision is used to calculate the precision of the second test triplet of each category in the corresponding category, the weight-aware recall rate is used to calculate the recall rate of the second test triplet of each category in the corresponding category, and the weight-aware F1 score is used to comprehensively measure the weight-aware precision and the weight-aware recall rate.

[0168] Based on the same application concept, the embodiment of the present application discloses an electronic device, Figure 3 A schematic diagram of an electronic device disclosed in an embodiment of the present application is shown. Figure 3 As shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory of the electronic device is not less than 12G, the main frequency of the processor is not less than 2.4GHz, the memory 110 and the processor 120 are connected through a bus communication, and a computer program is stored in the memory 110. The computer program can be run on the processor 120 to implement a weight-aware performance evaluation method for knowledge graph embedding disclosed in an embodiment of the present application.

[0169] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0170] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, devices, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or box in the flowchart and / or block diagram, as well as the combination of the processes and / or boxes in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0171] These computer program instructions may also be stored in a computer readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0172] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for implementing the process in the computer or other programmable terminal device. Figure 1 A process or multiple processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes.

[0173] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0174] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0175] The above is a detailed introduction to the weight-aware performance evaluation method for knowledge graph embedding provided by the present application. This article uses specific examples to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A weight-aware performance evaluation method for knowledge graph embedding, characterized in that: The method comprises: Obtaining an undamaged weighted triple from a complete weighted knowledge graph as a first triple to be tested, wherein the complete weighted knowledge graph stores real-world knowledge in the form of a graph, and the knowledge is related knowledge in an information retrieval system, a question-answering system, or a recommendation system, and when the weight of a triple in the complete weighted knowledge graph is higher, the corresponding importance of the triple is also higher; Deleting the head entity of the first triple to be tested, and replacing it with each entity in the entity dictionary in turn, to obtain multiple first damaged triples; The tail entity of the first triple to be tested is deleted, and replaced with each entity in the entity dictionary in turn, to obtain a plurality of second damaged triples; Taking the first to-be-tested triple, a plurality of the first damaged triples, and a plurality of the second damaged triples as a test set; Using the test set to evaluate the weight-aware link prediction performance of a pre-trained weight-aware link prediction model, the pre-trained weight-aware link prediction model is obtained by training with positive weighted triples, negative weighted triples, weight labels and positive and negative labels as training samples; The pre-trained weight-aware link prediction model is used to perform link prediction on the first to-be-tested triple and the plurality of first damaged triples, and obtain link prediction scores for the first to-be-tested triple and the plurality of first damaged triples; the closer the link prediction score of a triple is to 1, the more reasonable the relationship of the triple is, and the closer the link prediction score of a triple is to 0, the more unreasonable the relationship of the triple is; in link prediction, the goal is to predict missing entities in incomplete triples; The pre-trained weight-aware link prediction model is used to classify weight-aware triplets.

2. The weight-aware performance evaluation method for knowledge graph embedding according to claim 1, characterized in that: The weight-aware link prediction model is trained according to the following steps: Preprocessing the head entity, the relationship, and the tail entity of the training sample to obtain the preprocessed head entity, the relationship, and the tail entity; Performing operations on the preprocessed head entity, relationship, and tail entity to obtain an interaction vector; Performing link prediction on the interaction vector after taking the modulus of the interaction vector using the base model to be trained, and obtaining a link prediction score for the training sample; Calculating a loss value of the base model to be trained based on the link prediction score of the training sample and the positive and negative labels; Updating the loss value based on the weight label to obtain an updated loss value; The updated loss value is used to optimize the parameters of the base model to be trained to obtain the weight-aware link prediction model.

3. The weight-aware performance evaluation method for knowledge graph embedding according to claim 1, characterized in that: The step of evaluating the weight-aware link prediction performance of the pre-trained weight-aware link prediction model using the test set includes: Obtaining a predicted ranking of the first triple to be tested, where the predicted ranking of the first triple to be tested is used to represent a ranking of a link prediction score of the first triple to be tested; The predicted ranking is adjusted using an activation function and a true weight of the first triplet to be tested to obtain a weight-aware ranking; Based on the performance evaluation index and the weight-aware ranking, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is evaluated.

4. The weight-aware performance evaluation method for knowledge graph embedding according to claim 3 is characterized in that: The obtaining of the predicted ranking of the first triple to be tested includes: Using the pre-trained weight-aware link prediction model to perform link prediction on the first triple to be tested and the plurality of first damaged triples, to obtain link prediction scores for the first triple to be tested and the plurality of first damaged triples; sorting the link prediction scores of the first triple to be tested and the plurality of the first damaged triples, and recording the first ranking of the first triple to be tested; Using the pre-trained weight-aware link prediction model to perform link prediction on the first triple to be tested and the plurality of second damaged triples, to obtain link prediction scores for the first triple to be tested and the plurality of second damaged triples; sorting the link prediction scores of the first triple to be tested and the plurality of the second damaged triples, and recording the second ranking of the first triple to be tested; The first ranking and the second ranking are used as predicted rankings of the triplet to be tested.

5. The weight-aware performance evaluation method for knowledge graph embedding according to claim 3 is characterized in that: The performance evaluation indicators include: weight-perceived average ranking, weight-perceived average reciprocal ranking and weight-perceived average proportion; The weight-aware average ranking is used to calculate the average of the weight-aware rankings of all first triples to be tested, the weight-aware average reciprocal ranking is used to calculate the average of the weight-aware reciprocal rankings of all first triples to be tested, and the weight-aware average proportion is used to calculate the proportion of the first triples to be tested ranked in the top N.

6. The weight-aware performance evaluation method for knowledge graph embedding according to claim 1, characterized in that: The method further comprises: Obtaining an undamaged weighted triple from the complete weighted knowledge graph as the second triple to be tested; Deleting the head entity of the second triple to be tested, and replacing it with each entity in the entity dictionary in turn, to obtain a plurality of third damaged triples; Mixing the second triple to be tested and a plurality of the third damaged triples as a mixed set of head damaged triples; The tail entity of the second triple to be tested is deleted, and replaced with each entity in the entity dictionary in turn, to obtain a plurality of fourth damaged triples; Mixing the second triple to be tested and a plurality of the fourth damaged triples as a tail damaged triple mixed set; The mixed set of head damaged triplets and the mixed set of tail damaged triplets are used as mixed test sets, and based on the mixed test sets, the weight-aware link prediction performance of the pre-trained weight-aware link prediction model is evaluated.

7. The weight-aware performance evaluation method for knowledge graph embedding according to claim 6, characterized in that: The step of evaluating the weight-aware link prediction performance of the pre-trained weight-aware link prediction model based on the mixed test set includes: Using a pre-trained weight-aware link prediction model to perform link prediction on each triple in the mixed test set to obtain a link prediction score for each triple in the mixed test set; Based on the link prediction score of each triple and a preset threshold, the triples in the mixed test set are divided into a positive set and a negative set to obtain a prediction classification result of each triple; Based on the predicted classification results and positive and negative labels of each triplet, dividing each triplet into a true positive set, a false positive set and a false negative set; Based on the true positive set, the false positive set and the false negative set, a classification performance evaluation index is used to evaluate the weight-aware link prediction performance of the pre-trained weight-aware link prediction model.

8. The weight-aware performance evaluation method for knowledge graph embedding according to claim 7, characterized in that: The classification performance evaluation indicators include: weight-aware precision, weight-aware recall and weight-aware F1 score; The weight-aware precision is used to calculate the precision of the second triplet to be tested in each category on the corresponding category, the weight-aware recall is used to calculate the recall of the second triplet to be tested in each category on the corresponding category, and the weight-aware F1 score is used to comprehensively measure the weight-aware precision and the weight-aware recall.

9. A weight-aware performance evaluation system for knowledge graph embedding, characterized in that: The system comprises: A first acquisition module is used to acquire an intact weighted triple as a first triple to be tested from a complete weighted knowledge graph, wherein the complete weighted knowledge graph stores real-world knowledge in the form of a graph, and the knowledge is related knowledge in an information retrieval system, a question-answering system, or a recommendation system, and when the weight of a triple in the complete weighted knowledge graph is higher, the corresponding importance of the triple is also higher; A first damaged triple construction module is used to delete the head entity of the first triple to be tested and replace it with each entity in the entity dictionary in turn to obtain multiple first damaged triples; A second damaged triple construction module is used to delete the tail entity of the first triple to be tested and replace it with each entity in the entity dictionary in turn to obtain multiple second damaged triples; A test set construction module, configured to use the first to-be-tested triple, a plurality of the first damaged triples, and a plurality of the second damaged triples as a test set; The first evaluation module is used to evaluate the weight-aware link prediction performance of a pre-trained weight-aware link prediction model using the test set, wherein the pre-trained weight-aware link prediction model is obtained by training with positive weighted triples, negative weighted triples, weight labels and positive and negative labels as training samples; the pre-trained weight-aware link prediction model is used to perform link prediction on a first triple to be tested and multiple first damaged triples to obtain link prediction scores for the first triple to be tested and multiple first damaged triples; the closer the link prediction score of a triple is to 1, the more reasonable the relationship of the triple is, and the closer the link prediction score of a triple is to 0, the more unreasonable the relationship of the triple is; in link prediction, the goal is to predict the missing entities in the incomplete triples; the pre-trained weight-aware link prediction model is used to classify weight-aware triples.

10. An electronic device, characterized in that: include: A memory for storing one or more programs; processor; When the one or more programs are executed by the processor, the weight-aware performance evaluation method for knowledge graph embedding as described in any one of claims 1-8 is implemented.

Citation Information

Cited By

  • Evaluation method and system for generative AI model

    CN121705648A