Accumulative evaluation method, system and equipment of knowledge graph completion model and medium

Abstract

The invention provides an accumulative evaluation method, system and device for a knowledge graph completion model and a medium, and relates to the technical field of knowledge graphs, and the method comprises the steps: constructing a sparsest sub-graph, and gradually supplementing a triple which is predicted by a knowledge graph embedding model and possibly exists in a complete knowledge graph into the sparsest sub-graph, obtaining the incomplete knowledge graph of each scale so as to simulate the knowledge graph completion process based on the knowledge graph embedding model; training a knowledge graph embedding model by using the incomplete knowledge graph of each scale, and determining the knowledge graph integrity degree based on the knowledge graph embedding model by using the performance of the trained knowledge graph embedding model on a link prediction task; and evaluating the performance of the knowledge graph completion model by using the variation of the knowledge graph integrity degree based on the knowledge graph embedding model. According to the method, the quality change of the knowledge graph can be effectively monitored, and the performance of the knowledge graph completion model can be accurately evaluated.

Description

Technical Field

[0001] The present application relates to the field of knowledge graph technology, and in particular to a cumulative evaluation method, system, device and medium for a knowledge graph completion model. Background Art

[0002] Knowledge graph embedding maps entities and relations in a knowledge graph into a continuous vector space to achieve efficient and meaningful representation learning of structured information. Link prediction is a common task to evaluate the effectiveness of knowledge graph embedding in reasoning about knowledge graph structures.

[0003] However, due to the inherent difficulty in quantifying the completeness of knowledge graphs, triple classification tasks and link prediction tasks are often used in combination in the field of knowledge graph completion. The above tasks involve predicting missing triples and judging the correctness of predicted triples so that the missing triples can be added to the knowledge graph. However, the above tasks ignore the interdependence between link prediction and triple classification, as both tasks rely on the likelihood scores of triples. Current evaluation methods for knowledge graph completion models do not fully evaluate the effectiveness of the models in completing knowledge graphs.

[0004] The basic assumption of evaluating the performance of models on the knowledge graph completion task based on the number of correctly predicted triples is that each missing triple contributes equally to the overall completeness of the knowledge graph. This approach is reasonable given that the main evaluation criteria for evaluating the completeness of knowledge graphs currently revolve around the coverage of the target domain. However, the importance of each triple in the knowledge graph is different. Therefore, the above evaluation methods all involve the assignment uncertainty, confidence score, relevance, marginal importance, etc. of triples, resulting in inaccurate performance evaluation of knowledge graph completion.

[0005] Therefore, it is urgent to develop an evaluation method for knowledge graph completion models. Summary of the invention

[0006] The present application provides a cumulative evaluation method, system, device and medium for a knowledge graph completion model to solve the above-mentioned problems.

[0007] In a first aspect of an embodiment of the present application, a cumulative evaluation method for a knowledge graph completion model is proposed, the method comprising:

[0008] Constructing a sparsest subgraph, wherein the sparsest subgraph uses the least triples to represent all entities and relationships in the complete knowledge graph;

[0009] Step by step, the triplets predicted by the knowledge graph embedding model that may exist in the complete knowledge graph are added to the sparsest subgraph to obtain incomplete knowledge graphs of various scales based on the knowledge graph embedding model, thereby simulating the process of knowledge graph completion based on the knowledge graph embedding model;

[0010] In the process of simulating knowledge graph completion based on the knowledge graph embedding model, the knowledge graph embedding model is trained using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, wherein the trained knowledge graph embedding model is used to: determine triples that may exist in the complete knowledge graph for the next step of completion;

[0011] The performance of the knowledge graph completion model is evaluated by using the change in the completeness of the knowledge graph based on the knowledge graph embedding model.

[0012] In an optional embodiment of the present application, the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, including:

[0013] In the process of simulating knowledge graph completion based on the knowledge graph embedding model, an incomplete knowledge graph based on the knowledge graph embedding model of a certain scale obtained after each step of completion is used to train the knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is recorded to obtain the link prediction performance based on the knowledge graph embedding model, wherein the link prediction performance indicators include the average reciprocal ranking, the average ranking or the top N hit rate;

[0014] Calculate an absolute performance improvement value based on the weighted function and the link prediction performance based on the knowledge graph embedding model in multiple steps;

[0015] The performance of the knowledge graph completion model is evaluated by using the change in the completeness of the knowledge graph based on the knowledge graph embedding model, including:

[0016] The absolute performance improvement value is used to evaluate the performance of the knowledge graph completion model.

[0017] In an optional embodiment of the present application, it also includes:

[0018] Gradually adding randomly determined triples that may exist in the complete knowledge graph into the sparsest subgraph to obtain random incomplete knowledge graphs of various scales, thereby simulating the process of random knowledge graph completion;

[0019] In the process of simulating random knowledge graph completion, the knowledge graph embedding model is trained using the random incomplete knowledge graphs of various sizes, and the completeness of the random knowledge graph is determined using the performance of the trained knowledge graph embedding model on the link prediction task;

[0020] At each scale, calculate the change between the completeness of the knowledge graph based on the knowledge graph embedding model and the completeness of the random knowledge graph to obtain a relative performance improvement value;

[0021] The performance of the knowledge graph completion model is evaluated by using the change in the completeness of the knowledge graph based on the knowledge graph embedding model, including:

[0022] The performance of the knowledge graph completion model is evaluated using the relative performance improvement value and the absolute performance improvement value.

[0023] In an optional embodiment of the present application, in the process of simulating random knowledge graph completion, the random incomplete knowledge graphs of various scales are used to train the knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the random knowledge graph, including:

[0024] Randomly select candidate triples to be added from the candidate triple set and add them to the sparsest subgraph to obtain a sparse subgraph;

[0025] After K random selections, K sparse subgraphs are obtained;

[0026] Based on the K sparse subgraphs, the knowledge graph embedding model is trained, and the link prediction performance of the knowledge graph embedding model on the K sparse subgraphs is calculated to obtain the completeness of the random knowledge graph;

[0027] Under the closed world assumption, all candidate triples in the candidate triple set are positive triples; under the open world assumption, the candidate triples in the candidate triple set are mixed positive and negative triples.

[0028] In an optional embodiment of the present application, in the process of simulating knowledge graph completion based on the knowledge graph embedding model, the knowledge graph embedding model is trained using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, including:

[0029] Based on the rationality of each candidate triple in the candidate triple set, gradually select candidate triples that may exist in the complete knowledge graph from the candidate triple set and add them into the sparsest subgraph to obtain a sparse subgraph;

[0030] After H times of rational selection based on candidate triple combinations by the knowledge graph embedding model, H sparse subgraphs are obtained;

[0031] Based on the H sparse subgraphs, the knowledge graph embedding model is trained, and the link prediction performance of the knowledge graph embedding model on the H sparse subgraphs is calculated to obtain the completeness of the knowledge graph based on the rational addition of the triple combination;

[0032] Under the closed world assumption, all candidate triples in the candidate triple set are positive triples; under the open world assumption, the candidate triples in the candidate triple set are mixed positive and negative triples.

[0033] In an optional embodiment of the present application, the construction process of the sparsest subgraph is as follows:

[0034] Initialize the sparsest subgraph to an empty set;

[0035] Constructing the entity set of the complete knowledge graph and the entity set of the sparsest subgraph respectively;

[0036] Based on the entity set of the complete knowledge graph and the entity set of the sparsest subgraph, determining the entities missing in the sparsest subgraph;

[0037] Traversing the complete knowledge graph, adding the triples containing the entities missing from the sparsest subgraph to the sparsest subgraph, updating the entity set of the sparsest subgraph, until all entities in the complete knowledge graph are added to the entity set of the sparsest subgraph, and obtaining the entity set of the sparsest subgraph containing all entities in the complete knowledge graph;

[0038] Constructing a relation set of the complete knowledge graph and a relation set of the sparsest subgraph respectively;

[0039] Based on the relationship set of the complete knowledge graph and the relationship set of the sparsest subgraph, determining the missing relationships in the sparsest subgraph;

[0040] The complete knowledge graph is traversed, and the triples containing the relations missing in the sparsest subgraph are added to the sparsest subgraph, and the relation set of the sparsest subgraph is updated, until all the relations in the complete knowledge graph are added to the relation set of the sparsest subgraph, thereby obtaining the sparsest subgraph.

[0041] In an optional embodiment of the present application, the process of simulating knowledge graph completion based on the knowledge graph embedding model includes:

[0042] Obtain the sparsest subgraph A of the complete knowledge graph, wherein the sparsest subgraph A contains all entities and relations in the complete knowledge graph, and obtain a mixed candidate triple set or a positive candidate triple set, wherein the positive candidate triple set includes: m positive triples among the remaining triples in the complete knowledge graph except the sparsest subgraph A, and the mixed candidate triple set includes: n negative triples generated based on each positive triple and the corresponding positive triples, and the remaining triples in the complete knowledge graph except the above-mentioned sparsest subgraph A and the positive candidate triple set are used as a test set;

[0043] Training the knowledge graph embedding model based on the sparsest subgraph A to obtain a trained knowledge graph embedding model;

[0044] Based on the test set, calculating an index value of the link prediction performance of the trained knowledge graph embedding model;

[0045] Based on the sparsest subgraph A, the process of simulating knowledge graph completion based on the knowledge graph embedding model is as follows:

[0046] Using the trained knowledge graph embedding model, embedding the sparsest subgraph A to obtain vector representations of entities and relationships;

[0047] Using the vector representations of the entities and relations, reasoning on the mixed candidate triple set or the positive candidate triple set, and sorting the triples in the mixed candidate triple set or the positive candidate triple set according to rationality;

[0048] Adding the top x% of triples to the sparsest subgraph A to obtain a sparse subgraph B;

[0049] Based on the sparse subgraph B, the trained knowledge graph embedding model is trained again;

[0050] Based on the test set, calculating the index value of the link prediction performance of the knowledge graph embedding model after retraining;

[0051] Repeat the above process Q times to perform Q knowledge graph completions based on the knowledge graph embedding model.

[0052] In a second aspect of an embodiment of the present application, a cumulative evaluation system for a knowledge graph completion model is proposed, the system comprising:

[0053] A construction module, used to construct a sparsest subgraph, wherein the sparsest subgraph uses the least triples to represent all entities and relations in the complete knowledge graph;

[0054] A simulation completion module is used to gradually complete the triples predicted by the knowledge graph embedding model that may exist in the complete knowledge graph into the sparsest subgraph, so as to obtain incomplete knowledge graphs of various scales based on the knowledge graph embedding model, thereby simulating the process of knowledge graph completion based on the knowledge graph embedding model;

[0055] A determination module is used to train the knowledge graph embedding model using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale in the process of simulating the completion of the knowledge graph based on the knowledge graph embedding model, and determine the completeness of the knowledge graph based on the knowledge graph embedding model using the performance of the trained knowledge graph embedding model on the link prediction task, wherein the trained knowledge graph embedding model is used to: determine the triples that may exist in the complete knowledge graph for the next completion;

[0056] The evaluation module is used to evaluate the performance of the knowledge graph completion model by utilizing the change in the completeness of the knowledge graph based on the knowledge graph embedding model.

[0057] In a third aspect of an embodiment of the present application, an electronic device is proposed, comprising a memory, a processor, and a computer program stored on the memory, wherein the processor executes the computer program to implement the cumulative evaluation method of the knowledge graph completion model described in any one of the first aspects above.

[0058] In a fourth aspect of an embodiment of the present application, a computer-readable storage medium is proposed, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, the cumulative evaluation method of the knowledge graph completion model described in any one of the first aspects above is implemented.

[0059] The present application includes the following advantages: An embodiment of the present application provides a cumulative evaluation method, system, device and medium for a knowledge graph completion model, by constructing a sparsest subgraph, wherein the sparsest subgraph uses the least triples to represent all entities and relationships in a complete knowledge graph; gradually filling in the triples predicted by the knowledge graph embedding model that may exist in the complete knowledge graph into the sparsest subgraph to obtain incomplete knowledge graphs of various scales based on the knowledge graph embedding model, thereby simulating the process of knowledge graph completion based on the knowledge graph embedding model; in the process of simulating knowledge graph completion based on the knowledge graph embedding model, the incomplete knowledge graphs of various scales based on the knowledge graph embedding model are used to train the knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, wherein the trained knowledge graph embedding model is used to: determine the triples that may exist in the complete knowledge graph for the next step of completion; and evaluate the performance of the knowledge graph completion model using the change in the completeness of the knowledge graph based on the knowledge graph embedding model. This method evaluates the performance of the knowledge graph completion model through relative performance improvement values ​​and absolute performance improvement values. It utilizes the performance changes of the knowledge graph embedding model in the link prediction task to evaluate the completeness of the embedded knowledge graph, and uses the triples predicted by the knowledge graph embedding model to expand the knowledge graph to be completed. The performance of the completion model is evaluated based on the quality changes. This method can effectively monitor the quality changes of the knowledge graph and accurately evaluate the performance of the knowledge graph completion model. BRIEF DESCRIPTION OF THE DRAWINGS

[0060] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the description of the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.

[0061] Figure 1 It is a flowchart of the steps of a cumulative evaluation method of a knowledge graph completion model provided in an embodiment of the present application;

[0062] Figure 2 It is an architecture diagram of a cumulative evaluation system of a knowledge graph completion model provided in an embodiment of the present application;

[0063] Figure 3 It is a schematic diagram of an electronic device provided in an embodiment of the present application. DETAILED DESCRIPTION

[0064] The following will be combined with the drawings in the embodiments of the present application to clearly and completely describe the technical solutions in the embodiments of the present application. Obviously, the described embodiments are part of the embodiments of the present application, not all of the embodiments. Based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of this application.

[0065] Knowledge graph embedding maps entities and relations in a knowledge graph into a continuous vector space to achieve efficient and meaningful representation learning of structured information. Link prediction is a common task to evaluate the effectiveness of knowledge graph embedding in reasoning about knowledge graph structures.

[0066] However, due to the inherent difficulty in quantifying the completeness of knowledge graphs, triple classification tasks and link prediction tasks are often used in combination in the field of knowledge graph completion. The above tasks involve predicting missing triples and judging the correctness of predicted triples so that the missing triples can be added to the knowledge graph. However, the above tasks ignore the interdependence between link prediction and triple classification, as both tasks rely on the likelihood scores of triples. Current evaluation methods for knowledge graph completion models do not fully evaluate the effectiveness of the models in completing knowledge graphs.

[0067] The basic assumption of evaluating the performance of models on the knowledge graph completion task based on the number of correctly predicted triples is that each missing triple contributes equally to the overall completeness of the knowledge graph. This approach is reasonable given that the main evaluation criteria for evaluating the completeness of knowledge graphs currently revolve around the coverage of the target domain. However, the importance of each triple in the knowledge graph is different. Therefore, the above evaluation methods all involve the assignment uncertainty, confidence score, relevance, marginal importance, etc. of triples, resulting in inaccurate performance evaluation of knowledge graph completion.

[0068] Based on this, this application proposes an evaluation method for a knowledge graph completion model, by randomly establishing knowledge graphs of different sizes under the open world hypothesis and the closed world hypothesis, selecting triples to construct knowledge graphs of different sizes under the open world hypothesis and the closed world hypothesis, training knowledge graph embedding on incomplete knowledge graphs of different sizes, and evaluating the trained model on the link prediction task, and proposing the absolute and relative performance improvements of the knowledge graph embedding model on the link prediction task to evaluate the quality enhancement of the knowledge graph in the completion process.

[0069] See also Figure 1 , Figure 1 This is a flowchart of a cumulative evaluation method for a knowledge graph completion model provided in an embodiment of the present application, and the method includes the following steps:

[0070] Step 101: construct a sparsest subgraph, where the sparsest subgraph uses the fewest triples to represent all entities and relationships in the complete knowledge graph;

[0071] Step 102: gradually add the triples predicted by the knowledge graph embedding model that may exist in the complete knowledge graph into the sparsest subgraph to obtain incomplete knowledge graphs of various scales based on the knowledge graph embedding model, thereby simulating the process of knowledge graph completion based on the knowledge graph embedding model;

[0072] Step 103: In the process of simulating knowledge graph completion based on the knowledge graph embedding model, the knowledge graph embedding model is trained using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, wherein the trained knowledge graph embedding model is used to: determine triples that may exist in the complete knowledge graph for the next step of completion;

[0073] Step 104: Evaluate the performance of the knowledge graph completion model using the change in the degree of completeness of the knowledge graph based on the knowledge graph embedding model.

[0074] When implementing step 101, in the evaluation method, a sparsest subgraph containing all entities and relations in the complete knowledge graph is first constructed, and the process of supplementing the triples in the candidate triples into the sparsest subgraph is used as a process of simulating the knowledge graph completion based on the knowledge graph embedding model. Therefore, the sparsest subgraph is first constructed, and in the sparsest subgraph, all entities and relations in the complete knowledge graph are represented by the least triples.

[0075] In an optional embodiment of the present application, the construction process of the above-mentioned sparsest subgraph is as follows: first, an empty set is initialized as the sparsest subgraph, at which time, there is no entity or relationship in the sparsest subgraph. Then, the entity set of the complete knowledge graph and the entity set of the sparsest subgraph are constructed respectively, and the entity set of the complete knowledge graph contains all the entities of the complete knowledge graph. Based on the entity set of the entire knowledge graph and the entity set of the sparsest subgraph, the entities missing in the sparsest subgraph are determined; all triples of the above-mentioned complete knowledge graph are traversed, and the triples containing the entities missing in the sparsest subgraph are added to the sparsest subgraph, and the entity set of the sparsest subgraph is updated until all entities in the complete knowledge graph are added to the entity set of the sparsest subgraph, and the entity set of the sparsest subgraph containing all entities in the complete knowledge graph is obtained; the relationship set of the above-mentioned complete knowledge graph and the relationship set of the sparsest subgraph are constructed respectively, and the relationship set of the complete knowledge graph contains all the relationships of the complete knowledge graph. Based on the relationship set of the complete knowledge graph and the relationship set of the sparsest subgraph, determine the relationship missing in the sparsest subgraph. Traverse the complete knowledge graph, add the triples containing the relationship missing in the sparsest subgraph to the sparsest subgraph, and update the relationship set of the sparsest subgraph until all the relationships in the complete knowledge graph are added to the relationship set of the sparsest subgraph to obtain the sparsest subgraph.

[0076] When implementing step 102, after constructing the sparsest subgraph, the triples predicted by the knowledge graph embedding model that may exist in the above-mentioned complete knowledge graph are gradually added to the sparsest subgraph to obtain incomplete knowledge graphs of various scales based on the knowledge graph embedding model, thereby simulating the process of completing the knowledge graph based on the knowledge graph embedding model. The triples predicted by the knowledge graph embedding model that may exist in the above-mentioned complete knowledge graph are predicted based on the rationality of the existence of the triples.

[0077] In an optional embodiment of the present application, the process of simulating knowledge graph completion based on the knowledge graph embedding model specifically includes: first obtaining the sparsest subgraph A of the above-mentioned complete knowledge graph, in which all entities and relations in the above-mentioned complete knowledge graph are included, and a mixed candidate triple set or a positive candidate triple set is obtained, wherein the positive candidate triple set includes: m positive triples in the remaining triples in the above-mentioned complete knowledge graph except the above-mentioned sparsest subgraph A, and the above-mentioned mixed candidate triple set includes: n negative triples generated based on each positive triple and the corresponding positive triples, and the remaining triples in the above-mentioned complete knowledge graph except the above-mentioned sparsest subgraph A and the positive candidate triple set are used as a test set. Then, the above-mentioned knowledge graph embedding model is trained based on the above-mentioned sparsest subgraph A to obtain the trained knowledge graph embedding model; based on the above-mentioned test set, the link prediction performance index value of the above-mentioned trained knowledge graph embedding model is calculated, and the corresponding link prediction performance index value can be the average reciprocal ranking, average ranking or top N hit rate value. Wherein, m and n are integers greater than 1, and N is an integer greater than 3 in the top N hit rates.

[0078] Furthermore, based on the sparsest subgraph A, the process of simulating knowledge graph completion based on the knowledge graph embedding model is as follows: using the trained knowledge graph embedding model, the sparsest subgraph A is embedded to obtain vector representations of entities and relationships; using the vector representations of entities and relationships, the mixed candidate triple set or the positive candidate triple set is inferred, and the triples in the mixed candidate triple set or the positive candidate triple set are sorted according to rationality; the top x% of triples are added to the sparsest subgraph A to obtain a sparse subgraph B; based on the sparse subgraph B , retrain the knowledge graph embedding model after the training; and based on the test set, calculate the index value of the link prediction performance of the knowledge graph embedding model after the retraining, and the corresponding index value of the link prediction performance can be the average reciprocal ranking, average ranking or the value of the top N hit rate; repeat the above process Q times to perform Q times of knowledge graph completion based on the knowledge graph embedding model. In this application, the value of Q is a pre-set value, which is determined according to the number of triples in the above mixed candidate triple set or the above positive candidate triple set that need to be supplemented in the sparsest subgraph, and Q is an integer greater than 1. The top x% is determined by the number of triples that need to be supplemented in the sparsest subgraph and the Q value, and x can be 5, 10 or other values.

[0079] When implementing step 103 specifically, in the process of simulating the completion of the knowledge graph based on the knowledge graph embedding model, multiple incomplete knowledge graphs of different sizes can be obtained after multiple steps of filling in the sparsest subgraph, and the knowledge graph embedding model is trained using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, wherein the trained knowledge graph embedding model is used to determine the triples that may exist in the complete knowledge graph for the next step of completion.

[0080] In an optional embodiment of the present application, the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, specifically including: in the process of simulating the completion of the knowledge graph based on the knowledge graph embedding model, a large-scale incomplete knowledge graph based on the knowledge graph embedding model obtained after each step of completion is used to train the above-mentioned knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is recorded to obtain the link prediction performance based on the knowledge graph embedding model, and the indicators of the above-mentioned link prediction performance include the average reciprocal ranking, the average ranking or the top N hit rate; then, based on the weighted function and multiple steps of the above-mentioned link prediction performance based on the knowledge graph embedding model, the absolute performance improvement value is calculated, and the absolute performance improvement value is the performance value obtained based on the absolute improvement of the knowledge graph embedding model.

[0081] In an optional embodiment of the present application, in addition to adding the triples predicted by the knowledge graph embedding model to the sparsest subgraph to simulate the knowledge graph completion process, the knowledge graph completion process can also be simulated by randomly selecting triples and adding them to the sparsest subgraph. Specifically, the randomly determined triples that may exist in the above-mentioned complete knowledge graph are gradually added to the above-mentioned sparsest subgraph to obtain random incomplete knowledge graphs of various scales, thereby simulating the random knowledge graph completion process; in the process of simulating random knowledge graph completion, the random incomplete knowledge graphs of various scales are used to train the knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the random knowledge graph; at each scale, the change in the completeness of the knowledge graph based on the knowledge graph embedding model and the random knowledge graph completeness is calculated to obtain a relative performance improvement value, which is a performance value obtained based on the relative improvement of the knowledge graph embedding model.

[0082] In an optional embodiment of the present application, in the process of simulating random knowledge graph completion, the random incomplete knowledge graphs of various scales are used to train the knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the random knowledge graph, which specifically includes the following processes: randomly selecting candidate triples to be added from the candidate triple set to fill in the sparsest subgraph to obtain a sparse subgraph; after K random selections, K sparse subgraphs are obtained; based on the above K sparse subgraphs, the above knowledge graph embedding model is trained, and the link prediction performance of the above knowledge graph embedding model on the above K sparse subgraphs is calculated to obtain the completeness of the random knowledge graph. In the present application, the value of K is a pre-set value, which is set according to the number of triples in the above candidate triple set that need to be added to the sparsest subgraph, and K is an integer greater than 1. Under the closed world assumption, all candidate triplets in the above candidate triple set are positive triples; under the open world assumption, the candidate triplets in the above candidate triple set are mixed positive and negative triples.

[0083] In the embodiment of the present application, the closed world assumption refers to the assumption that all candidate triples in the candidate triple set are positive triples, and triples not in the candidate triple set are negative, and the open world assumption refers to the assumption that all candidate triples in the candidate triple set may be positive or negative, and candidate triples not in the candidate triple set may also be positive. A positive triple means that the triple is true, and a negative triple means that the triple is false.

[0084] In an optional embodiment of the present application, in the process of simulating knowledge graph completion based on the knowledge graph embedding model, the knowledge graph embedding model is trained using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, specifically including: based on the rationality of each candidate triple in the candidate triple set, gradually selecting triples that may exist in the above-mentioned complete knowledge graph from the above-mentioned candidate triple set and filling them into the most sparse subgraph to obtain a sparse subgraph; after H times of rational selection based on candidate triple combinations by the knowledge graph embedding model, H sparse subgraphs are obtained; based on the above-mentioned H sparse subgraphs, the above-mentioned knowledge graph embedding model is trained, and the link prediction performance of the above-mentioned knowledge graph embedding model on the above-mentioned H sparse subgraphs is calculated to obtain the completeness of the knowledge graph based on the rational filling of triple combinations, that is, the completeness of the knowledge graph based on the knowledge graph embedding model. In the present application, the value of H is a preset value, which is determined according to the number of triples in the candidate triple set that need to be added to the sparsest subgraph, and H is an integer greater than 1. Under the closed world assumption, all candidate triples in the candidate triple set are positive triples; under the open world assumption, the candidate triples in the candidate triple set are mixed positive and negative triples.

[0085] When step 104 is specifically implemented, after multiple steps of filling in the sparsest subgraph, the knowledge graph embedding model is trained using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale, and the performance of the trained knowledge graph embedding model on the link prediction task is used to obtain the change in the completeness of the knowledge graph based on the knowledge graph embedding model, which is an absolute performance improvement value and a relative performance improvement value. The performance of the knowledge graph completion model is evaluated using the absolute performance improvement value and the relative performance improvement value.

[0086] In an optional embodiment of the present application, the performance of the knowledge graph completion model is evaluated using the absolute performance improvement value as the change in the degree of completeness of the knowledge graph based on the knowledge graph embedding model, including: using the above absolute performance improvement value to evaluate the performance of the knowledge graph completion model. Since the absolute performance improvement value is an absolute improvement in the link prediction performance based on the knowledge graph embedding model, it is more accurate to use it to evaluate the performance of the knowledge graph completion model.

[0087] In an optional embodiment of the present application, the absolute performance improvement value can be calculated by the following formula:

[0088]

[0089] Where g(i) is the weighting function, P i+1 and P iAn incomplete knowledge graph based on a knowledge graph embedding model of a certain scale is obtained after completion in the i+1th step and the ith step respectively. The knowledge graph embedding model is trained to obtain the performance of the trained knowledge graph embedding model on the link prediction task.

[0090] In an optional embodiment of the present application, the performance of the knowledge graph completion model is evaluated using absolute performance improvement values ​​and relative performance improvement values ​​as changes in the degree of completeness of the knowledge graph based on the knowledge graph embedding model, including: using the above relative performance improvement values ​​and the absolute performance improvement values ​​to evaluate the performance of the knowledge graph completion model. The performance of the knowledge graph completion model is evaluated jointly by relative performance improvement values ​​and absolute performance improvement values, the performance changes of the knowledge graph embedding model in the link prediction task are used to evaluate the integrity of the embedded knowledge graph, and the triples predicted by the knowledge graph embedding model are used to expand the knowledge graph to be completed, and the performance of the completion model is evaluated based on quality changes. This method can effectively monitor the quality changes of the knowledge graph and accurately evaluate the performance of the knowledge graph completion model.

[0091] In an optional embodiment of the present application, the relative performance improvement value can be calculated by the following formula:

[0092]

[0093] In the formula, The incomplete knowledge graph obtained by filling the sparsest subgraph with the triples predicted by the knowledge graph embedding model in the i-th step is trained with a knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is used, P i 0 Train a knowledge graph embedding model for the random incomplete knowledge graph obtained in the i-th step, and use the performance of the trained knowledge graph embedding model on the link prediction task.

[0094] An embodiment of the present application provides a cumulative evaluation method for a knowledge graph completion model, by constructing a sparsest subgraph, wherein the sparsest subgraph uses the least triples to represent all entities and relationships in a complete knowledge graph; gradually filling in the triples predicted by the knowledge graph embedding model that may exist in the complete knowledge graph into the sparsest subgraph to obtain incomplete knowledge graphs of various scales based on the knowledge graph embedding model, thereby simulating the process of knowledge graph completion based on the knowledge graph embedding model; in the process of simulating knowledge graph completion based on the knowledge graph embedding model, the incomplete knowledge graphs of various scales based on the knowledge graph embedding model are used to train the knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, wherein the trained knowledge graph embedding model is used to: determine the triples that may exist in the complete knowledge graph for the next step of completion; and evaluate the performance of the knowledge graph completion model using the change in the completeness of the knowledge graph based on the knowledge graph embedding model. This method evaluates the performance of the knowledge graph completion model through relative performance improvement values ​​and absolute performance improvement values. It utilizes the performance changes of the knowledge graph embedding model in the link prediction task to evaluate the completeness of the embedded knowledge graph, and uses the triples predicted by the knowledge graph embedding model to expand the knowledge graph to be completed. The performance of the completion model is evaluated based on the quality changes. This method can effectively monitor the quality changes of the knowledge graph and accurately evaluate the performance of the knowledge graph completion model.

[0095] In a second aspect of the present application, a cumulative evaluation system for a knowledge graph completion model is provided. Figure 2 , Figure 2 : is an architecture diagram of a cumulative evaluation system for a knowledge graph completion model provided in an embodiment of the present application, the system comprising:

[0096] A construction module 201 is used to construct a sparsest subgraph, wherein the sparsest subgraph uses the least triples to represent all entities and relationships in the complete knowledge graph;

[0097] A simulation completion module 202 is used to gradually complete the triples predicted by the knowledge graph embedding model that may exist in the complete knowledge graph into the sparsest subgraph, so as to obtain incomplete knowledge graphs of various scales based on the knowledge graph embedding model, thereby simulating the process of knowledge graph completion based on the knowledge graph embedding model;

[0098] The determination module 203 is used to train the knowledge graph embedding model using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale in the process of simulating the knowledge graph completion based on the knowledge graph embedding model, and determine the completeness of the knowledge graph based on the knowledge graph embedding model using the performance of the trained knowledge graph embedding model on the link prediction task, wherein the trained knowledge graph embedding model is used to: determine the triples that may exist in the complete knowledge graph for the next step of completion;

[0099] The evaluation module 204 is used to evaluate the performance of the knowledge graph completion model by utilizing the change in the degree of completeness of the knowledge graph based on the knowledge graph embedding model.

[0100] Wherein, the determination module includes:

[0101] A link prediction performance acquisition submodule is used to train the knowledge graph embedding model by using an incomplete knowledge graph based on the knowledge graph embedding model of a certain scale obtained after each step of completion in the process of simulating knowledge graph completion based on the knowledge graph embedding model, and record the performance of the trained knowledge graph embedding model on the link prediction task to obtain the link prediction performance based on the knowledge graph embedding model, wherein the link prediction performance indicators include the average reciprocal ranking, the average ranking or the top N hit rate;

[0102] The absolute performance improvement value calculation submodule is used to calculate the absolute performance improvement value based on the weighted function and the link prediction performance based on the multi-step knowledge graph embedding model.

[0103] Wherein, the evaluation module includes:

[0104] The first evaluation submodule is used to evaluate the performance of the knowledge graph completion model using the absolute performance improvement value.

[0105] Wherein, the determination module further includes:

[0106] A random completion submodule, used to gradually fill randomly determined triples that may exist in the complete knowledge graph into the sparsest subgraph to obtain random incomplete knowledge graphs of various scales, thereby simulating the process of random knowledge graph completion;

[0107] A knowledge graph completeness determination submodule is used to train a knowledge graph embedding model using random incomplete knowledge graphs of various sizes in the process of simulating random knowledge graph completion, and determine the completeness of the random knowledge graph using the performance of the trained knowledge graph embedding model on the link prediction task;

[0108] A relative performance improvement value calculation submodule is used to calculate the change in the completeness of the knowledge graph based on the knowledge graph embedding model and the completeness of the random knowledge graph at various scales to obtain a relative performance improvement value;

[0109] The evaluation module also includes:

[0110] The second evaluation submodule is used to evaluate the performance of the knowledge graph completion model using the relative performance improvement value and the absolute performance improvement value.

[0111] Wherein, the random completion submodule includes:

[0112] A random filling unit, used for randomly selecting a candidate triple to be added from the candidate triple set and filling it into the sparsest subgraph to obtain a sparse subgraph;

[0113] K sparse subgraph acquisition units, used to obtain K sparse subgraphs through K random selections;

[0114] A random knowledge graph completeness acquisition unit is used to train the knowledge graph embedding model based on the K sparse subgraphs, and calculate the link prediction performance of the knowledge graph embedding model on the K sparse subgraphs to obtain the random knowledge graph completeness; under the closed world assumption, all candidate triples in the candidate triple set are positive triples; under the open world assumption, the candidate triples in the candidate triple set are mixed positive and negative triples.

[0115] Wherein, the determination module further includes:

[0116] A step-by-step selection submodule is used to gradually select candidate triples that may exist in the complete knowledge graph from the candidate triple set based on the rationality of each candidate triple in the candidate triple set and add them into the sparsest subgraph to obtain a sparse subgraph;

[0117] H sparse subgraph determination submodules are used to obtain H sparse subgraphs through H times of knowledge graph embedding model based on rational selection of candidate triple combinations;

[0118] The knowledge graph completeness acquisition submodule is used to train the knowledge graph embedding model based on the H sparse subgraphs, and calculate the link prediction performance of the knowledge graph embedding model on the H sparse subgraphs, so as to obtain the completeness of the knowledge graph based on the rational supplementation of triple combinations; under the closed world assumption, all candidate triples in the candidate triple set are positive triples; under the open world assumption, the candidate triples in the candidate triple set are mixed positive and negative triples.

[0119] Wherein, the building blocks include:

[0120] Initialization submodule, used to initialize the sparsest subgraph to an empty set;

[0121] An entity set construction submodule, used to respectively construct the entity set of the complete knowledge graph and the entity set of the sparsest subgraph;

[0122] A missing entity determination submodule, used to determine the missing entities in the sparsest subgraph based on the entity set of the complete knowledge graph and the entity set of the sparsest subgraph;

[0123] An entity update submodule, used to traverse the complete knowledge graph, add the triples containing the entities missing in the sparsest subgraph to the sparsest subgraph, and update the entity set of the sparsest subgraph until all entities in the complete knowledge graph are added to the entity set of the sparsest subgraph, thereby obtaining the entity set of the sparsest subgraph containing all entities in the complete knowledge graph;

[0124] A relation set construction submodule, used to respectively construct the relation set of the complete knowledge graph and the relation set of the sparsest subgraph;

[0125] A missing relationship determination submodule, used to determine the missing relationship in the sparsest subgraph based on the relationship set of the complete knowledge graph and the relationship set of the sparsest subgraph;

[0126] The sparsest subgraph construction submodule is used to traverse the complete knowledge graph, add triples containing relations missing in the sparsest subgraph to the sparsest subgraph, and update the relationship set of the sparsest subgraph until all relations in the complete knowledge graph are added to the relationship set of the sparsest subgraph to obtain the sparsest subgraph.

[0127] Wherein, the simulation completion module includes:

[0128] The sparsest subgraph A acquisition submodule is used to obtain the sparsest subgraph A of the complete knowledge graph, wherein the sparsest subgraph A contains all entities and relations in the complete knowledge graph, and obtain a mixed candidate triple set or a positive candidate triple set, wherein the positive candidate triple set includes: m positive triples among the remaining triples in the complete knowledge graph except the sparsest subgraph A, and the mixed candidate triple set includes: n negative triples generated based on each positive triple and the corresponding positive triples, and the remaining triples in the complete knowledge graph except the above-mentioned sparsest subgraph A and the positive candidate triple set are used as a test set;

[0129] A training submodule, used for training the knowledge graph embedding model based on the sparsest subgraph A to obtain a trained knowledge graph embedding model;

[0130] An index value calculation submodule, used to calculate the index value of the link prediction performance of the trained knowledge graph embedding model based on the test set;

[0131] The simulation completion module also includes a completion submodule, and the completion submodule includes:

[0132] An embedding unit, used to embed the sparsest subgraph A using the trained knowledge graph embedding model to obtain vector representations of entities and relationships;

[0133] A reasoning unit, configured to use the vector representations of the entities and relations to reason on the mixed candidate triple set or the positive candidate triple set, and to sort the triples in the mixed candidate triple set or the positive candidate triple set according to rationality;

[0134] An adding unit, used for adding the top x% of triples to the sparsest subgraph A to obtain a sparse subgraph B;

[0135] A retraining unit, used for retraining the trained knowledge graph embedding model based on the sparse subgraph B;

[0136] An index value calculation unit, used to calculate the index value of the link prediction performance of the knowledge graph embedding model after retraining based on the test set;

[0137] The repeating completion unit is used to repeat the above process Q times to perform Q times of knowledge graph completion based on the knowledge graph embedding model.

[0138] Based on the same application concept, the embodiment of the present application discloses an electronic device in a third aspect. Figure 3 A schematic diagram of an electronic device disclosed in an embodiment of the present application is shown. Figure 3 As shown, the electronic device 100 includes: a memory 110 and a processor 120. The memory of the electronic device is not less than 12G, the main frequency of the processor is not less than 2.4GHz, the memory 110 and the processor 120 are connected via a bus communication, and a computer program is stored in the memory 110. The computer program can be run on the processor 120 to implement a cumulative evaluation method for a knowledge graph completion model disclosed in an embodiment of the present application.

[0139] Based on the same application concept, the fourth aspect of the embodiment of the present application discloses a computer-readable storage medium, on which a computer program / instruction is stored. When the computer program / instruction is executed by a processor, a cumulative evaluation method for a knowledge graph completion model disclosed in the embodiment of the present application is implemented.

[0140] The various embodiments in this specification are described in a progressive manner, and each embodiment focuses on the differences from other embodiments. The same or similar parts between the various embodiments can be referenced to each other.

[0141] The embodiments of the present application are described with reference to the flowcharts and / or block diagrams of the methods, apparatuses, electronic devices, and computer program products according to the embodiments of the present application. It should be understood that each process and / or block in the flowchart and / or block diagram, as well as the combination of the processes and / or blocks in the flowchart and / or block diagram, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing terminal device to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing terminal device generate instructions for implementing the processes in the flowchart and / or block diagram. Figure 1 A process or multiple processes and / or boxes Figure 1 A device that provides the functions specified in a block or multiple blocks.

[0142] These computer program instructions may also be stored in a computer-readable memory capable of directing a computer or other programmable data processing terminal device to operate in a specific manner, so that the instructions stored in the computer-readable memory produce a manufactured product including an instruction device, which implements the process Figure 1 A process or multiple processes and / or boxes Figure 1 A function specified in one or more boxes.

[0143] These computer program instructions can also be loaded onto a computer or other programmable data processing terminal device so that a series of operating steps are executed on the computer or other programmable terminal device to produce a computer-implemented process, thereby providing instructions for executing on the computer or other programmable terminal device to implement the process. Figure 1 A process or multiple processes and / or boxes Figure 1 A step that specifies a function in one or more boxes.

[0144] Although the preferred embodiments of the present application have been described, those skilled in the art may make additional changes and modifications to these embodiments once they have learned the basic creative concept. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the embodiments of the present application.

[0145] Finally, it should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or terminal device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or terminal device. In the absence of further restrictions, the elements defined by the sentence "including one..." do not exclude the existence of other identical elements in the process, method, article or terminal device including the elements.

[0146] The above is a detailed introduction to the cumulative evaluation method, system, device and medium of a knowledge graph completion model provided by the present application. Specific examples are used in this article to illustrate the principles and implementation methods of the present application. The description of the above embodiments is only used to help understand the method of the present application and its core idea; at the same time, for general technical personnel in this field, according to the ideas of the present application, there will be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as a limitation on the present application.

Claims

1. A cumulative evaluation method for knowledge graph completion model, It is characterized in that The method comprises: Constructing a sparsest subgraph, wherein the sparsest subgraph uses the least triples to represent all entities and relationships in the complete knowledge graph; Step by step, the triplets predicted by the knowledge graph embedding model that may exist in the complete knowledge graph are added to the sparsest subgraph to obtain incomplete knowledge graphs of various scales based on the knowledge graph embedding model, thereby simulating the process of knowledge graph completion based on the knowledge graph embedding model; In the process of simulating knowledge graph completion based on the knowledge graph embedding model, the knowledge graph embedding model is trained using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, wherein the trained knowledge graph embedding model is used to: determine triples that may exist in the complete knowledge graph for the next step of completion; The performance of the knowledge graph completion model is evaluated by using the change in the completeness of the knowledge graph based on the knowledge graph embedding model.

2. The cumulative evaluation method of the knowledge graph completion model according to claim 1, It is characterized in that The performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, including: In the process of simulating knowledge graph completion based on the knowledge graph embedding model, an incomplete knowledge graph based on the knowledge graph embedding model of a certain scale obtained after each step of completion is used to train the knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is recorded to obtain the link prediction performance based on the knowledge graph embedding model, wherein the link prediction performance indicators include the average reciprocal ranking, the average ranking or the top N hit rate; Calculate an absolute performance improvement value based on the weighted function and the link prediction performance based on the knowledge graph embedding model in multiple steps; The performance of the knowledge graph completion model is evaluated by using the change in the completeness of the knowledge graph based on the knowledge graph embedding model, including: The absolute performance improvement value is used to evaluate the performance of the knowledge graph completion model.

3. The cumulative evaluation method of the knowledge graph completion model according to claim 2, It is characterized in that Also includes: Gradually adding randomly determined triples that may exist in the complete knowledge graph into the sparsest subgraph to obtain random incomplete knowledge graphs of various scales, thereby simulating the process of random knowledge graph completion; In the process of simulating random knowledge graph completion, the knowledge graph embedding model is trained using the random incomplete knowledge graphs of various sizes, and the completeness of the random knowledge graph is determined using the performance of the trained knowledge graph embedding model on the link prediction task; At each scale, calculate the change between the completeness of the knowledge graph based on the knowledge graph embedding model and the completeness of the random knowledge graph to obtain a relative performance improvement value; The performance of the knowledge graph completion model is evaluated by using the change in the completeness of the knowledge graph based on the knowledge graph embedding model, including: The performance of the knowledge graph completion model is evaluated using the relative performance improvement value and the absolute performance improvement value.

4. The cumulative evaluation method of the knowledge graph completion model according to claim 3, It is characterized in that In the process of simulating random knowledge graph completion, the random incomplete knowledge graphs of various scales are used to train the knowledge graph embedding model, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the random knowledge graph, including: Randomly select candidate triples to be added from the candidate triple set and add them to the sparsest subgraph to obtain a sparse subgraph; After K random selections, K sparse subgraphs are obtained; Based on the K sparse subgraphs, the knowledge graph embedding model is trained, and the link prediction performance of the knowledge graph embedding model on the K sparse subgraphs is calculated to obtain the completeness of the random knowledge graph; Under the closed world assumption, all candidate triples in the candidate triple set are positive triples; under the open world assumption, the candidate triples in the candidate triple set are mixed positive and negative triples.

5. The cumulative evaluation method of the knowledge graph completion model according to claim 1, It is characterized in that In the process of simulating knowledge graph completion based on the knowledge graph embedding model, the knowledge graph embedding model is trained using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale, and the performance of the trained knowledge graph embedding model on the link prediction task is used to determine the completeness of the knowledge graph based on the knowledge graph embedding model, including: Based on the rationality of each candidate triple in the candidate triple set, gradually select candidate triples that may exist in the complete knowledge graph from the candidate triple set and add them into the sparsest subgraph to obtain a sparse subgraph; After H times of rational selection based on candidate triple combinations by the knowledge graph embedding model, H sparse subgraphs are obtained; Based on the H sparse subgraphs, the knowledge graph embedding model is trained, and the link prediction performance of the knowledge graph embedding model on the H sparse subgraphs is calculated to obtain the completeness of the knowledge graph based on the rational addition of the triple combination; Under the closed world assumption, all candidate triples in the candidate triple set are positive triples; under the open world assumption, the candidate triples in the candidate triple set are mixed positive and negative triples.

6. The cumulative evaluation method of the knowledge graph completion model according to claim 1, It is characterized in that The construction process of the sparsest subgraph is as follows: Initialize the sparsest subgraph to an empty set; Constructing the entity set of the complete knowledge graph and the entity set of the sparsest subgraph respectively; Based on the entity set of the complete knowledge graph and the entity set of the sparsest subgraph, determining the entities missing in the sparsest subgraph; Traversing the complete knowledge graph, adding the triples containing the entities missing from the sparsest subgraph to the sparsest subgraph, updating the entity set of the sparsest subgraph, until all entities in the complete knowledge graph are added to the entity set of the sparsest subgraph, and obtaining the entity set of the sparsest subgraph containing all entities in the complete knowledge graph; Constructing a relation set of the complete knowledge graph and a relation set of the sparsest subgraph respectively; Based on the relationship set of the complete knowledge graph and the relationship set of the sparsest subgraph, determining the missing relationships in the sparsest subgraph; The complete knowledge graph is traversed, and the triples containing the relations missing in the sparsest subgraph are added to the sparsest subgraph, and the relation set of the sparsest subgraph is updated, until all the relations in the complete knowledge graph are added to the relation set of the sparsest subgraph, thereby obtaining the sparsest subgraph.

7. The cumulative evaluation method of the knowledge graph completion model according to claim 1, It is characterized in that Simulate the process of knowledge graph completion based on the knowledge graph embedding model, including: Obtain the sparsest subgraph A of the complete knowledge graph, wherein the sparsest subgraph A contains all entities and relations in the complete knowledge graph, and obtain a mixed candidate triple set or a positive candidate triple set, wherein the positive candidate triple set includes: m positive triples among the remaining triples in the complete knowledge graph except the sparsest subgraph A, and the mixed candidate triple set includes: n negative triples generated based on each positive triple and the corresponding positive triples, and the remaining triples in the complete knowledge graph except the above-mentioned sparsest subgraph A and the positive candidate triple set are used as a test set; Training the knowledge graph embedding model based on the sparsest subgraph A to obtain a trained knowledge graph embedding model; Based on the test set, calculating an index value of the link prediction performance of the trained knowledge graph embedding model; Based on the sparsest subgraph A, the process of simulating knowledge graph completion based on the knowledge graph embedding model is as follows: Using the trained knowledge graph embedding model, embedding the sparsest subgraph A to obtain vector representations of entities and relationships; Using the vector representations of the entities and relations, reasoning on the mixed candidate triple set or the positive candidate triple set, and sorting the triples in the mixed candidate triple set or the positive candidate triple set according to rationality; Adding the top x% of triples to the sparsest subgraph A to obtain a sparse subgraph B; Based on the sparse subgraph B, the trained knowledge graph embedding model is trained again; Based on the test set, calculating the index value of the link prediction performance of the knowledge graph embedding model after retraining; Repeat the above process Q times to perform Q knowledge graph completions based on the knowledge graph embedding model.

8. A cumulative evaluation system for knowledge graph completion models, It is characterized in that The system comprises: A construction module, used to construct a sparsest subgraph, wherein the sparsest subgraph uses the least triples to represent all entities and relations in the complete knowledge graph; A simulation completion module is used to gradually complete the triples predicted by the knowledge graph embedding model that may exist in the complete knowledge graph into the sparsest subgraph, so as to obtain incomplete knowledge graphs of various scales based on the knowledge graph embedding model, thereby simulating the process of knowledge graph completion based on the knowledge graph embedding model; A determination module is used to train the knowledge graph embedding model using the incomplete knowledge graphs based on the knowledge graph embedding model of each scale in the process of simulating the completion of the knowledge graph based on the knowledge graph embedding model, and determine the completeness of the knowledge graph based on the knowledge graph embedding model using the performance of the trained knowledge graph embedding model on the link prediction task, wherein the trained knowledge graph embedding model is used to: determine the triples that may exist in the complete knowledge graph for the next completion; The evaluation module is used to evaluate the performance of the knowledge graph completion model by utilizing the change in the completeness of the knowledge graph based on the knowledge graph embedding model.

9. An electronic device comprising a memory, a processor and a computer program stored in the memory, It is characterized in that The processor executes the computer program to implement the cumulative evaluation method of the knowledge graph completion model described in any one of claims 1-7.

10. A computer-readable storage medium, It is characterized in that A computer program / instruction is stored thereon, which, when executed by a processor, implements the cumulative evaluation method of the knowledge graph completion model described in any one of claims 1-7.

Images