Knowledge graph entity alignment algorithm automatic test method and system based on metamorphic relationship

Abstract

The invention discloses an automatic test method and system for a knowledge graph entity alignment algorithm based on a metamorphic relation, and the method comprises the steps: firstly providing a general framework for standardizing the input and output and basic feature description of different algorithms, setting a corresponding metamorphic rule to simulate the change of a data set, and evaluating the response of the algorithm; and then, according to the selected data set and entity alignment algorithm, result stability test and adaptability test are carried out, and different indexes are respectively adopted to evaluate the adaptability and accuracy of the algorithm to data change. Test experiment results show that the performance of the entity alignment algorithm can be effectively evaluated, and powerful technical support is provided for optimization of the algorithm.

Description

Technical Field

[0001] The present invention belongs to the field of knowledge graphs, and particularly relates to an automatic testing method for entity alignment algorithms of knowledge graphs based on metamorphic relations. Background Art

[0002] Entity Alignment is a key link in Knowledge Graph technology, which involves discovering and establishing corresponding relationships of entities between different knowledge graphs. This process is of great significance for realizing knowledge integration, avoiding redundancy and ambiguity, and improving the accuracy and reliability of knowledge. The importance of entity alignment lies in that it can not only improve search quality, enhance recommendation systems, but also promote cross-domain applications and improve decision-making support. With the growth of the scale and complexity of knowledge graphs, developing effective entity alignment algorithms and testing methods has become the research focus in this field.

[0003] Early entity alignment methods mainly relied on logical reasoning and lexical matching techniques, which usually highly depended on specific heuristic rules such as string similarity, dictionary matching, etc. However, these traditional methods have limitations in dealing with large-scale and heterogeneous data because they often ignore the complex relationships and attribute information between entities.

[0004] In recent years, embedding-based entity alignment methods have become a research hotspot. These methods learn low-dimensional vector representations of entities and relationships in knowledge graphs, and map entities and relationships to a common vector space. In this space, semantically similar entities will have closer vector representations, thus facilitating the discovery of corresponding entities between different knowledge graphs. The TransE model is a typical representative of this type of method, which learns embedding representations by regarding relationships as "translation" operations between entities. TransE and its variants (such as TransH, TransR, etc.) have made remarkable progress in dealing with complex relationships and large-scale knowledge graphs, but they still have limitations in expressive power, especially when dealing with one-to-many, many-to-one, many-to-many relationships and reflexive relationships.

[0005] To solve these problems, researchers have proposed entity alignment methods based on Graph Neural Networks (GNNs). These methods utilize the powerful capabilities of GNNs to capture the structural information in knowledge graphs and learn the weights of different types of relationships through attention mechanisms. For example, the GCN-Align model uses a multi-layer graph convolutional network to learn the embedding representations of entities, while the MuGNN model uses an attention mechanism to learn the weights of different types of relationships. These GNN-based methods have shown advantages in dealing with local subgraph-level consistency and relationship information.

[0006] In addition, some research work has begun to explore incorporating additional external information into the entity alignment process, such as image information, text information, and neighborhood information. This information, as weak supervision signals, helps improve the performance and robustness of entity alignment algorithms. For example, the EVA model incorporates visual information in the images associated with entities into entity embeddings, while the JEANS model utilizes rich external corpus information to improve performance.

[0007] Entity alignment technology plays a crucial role in the construction and application of knowledge graphs. With the growth of the scale and complexity of knowledge graphs, developing effective entity alignment algorithms and testing methods has become the research focus in this field.

[0008] In terms of testing entity alignment algorithms, due to the lack of effective test oracles (i.e., expected outputs), relatively little research has been done on automated testing methods. Existing testing methods mainly rely on manually constructed datasets, which require a large amount of human effort. Future research will further explore how to utilize advanced machine learning techniques and external information to improve the performance and robustness of entity alignment algorithms. At the same time, the research on automated testing methods will also provide important support for the evaluation and optimization of entity alignment algorithms. Summary of the Invention

[0009] Aiming at the above problems, the purpose of the present invention is to provide an automatic testing method and system for knowledge graph entity alignment algorithms based on metamorphic relations, which evaluate the effectiveness, accuracy, and scalability of entity alignment algorithms when processing knowledge graphs by designing metamorphic rules.

[0010] The specific technical solution to achieve the purpose of the present invention is as follows:

[0011] An automatic testing method for knowledge graph entity alignment algorithms based on metamorphic relations, comprising the following steps:

[0012] Step 1, set the general framework of the knowledge graph entity alignment algorithm;

[0013] Step 2, set the mutation rules of the knowledge graph alignment algorithm;

[0014] Step 3, select the dataset and specific algorithm of the alignment algorithm;

[0015] Step 4, based on the selected dataset and knowledge graph entity alignment algorithm, test the result stability of the knowledge graph entity alignment algorithm;

[0016] Step 5, based on the selected dataset and knowledge graph entity alignment algorithm, test the adaptability of the knowledge graph entity alignment algorithm.

[0017] The present invention also provides an automatic test system for a knowledge graph entity alignment algorithm based on metamorphic relations, including the following modules:

[0018] Framework construction module: used to set the general framework of the knowledge graph entity alignment algorithm;

[0019] Rule setting module: used to set the mutation rules of the knowledge graph alignment algorithm;

[0020] Algorithm determination module: used to select the dataset and specific algorithm of the alignment algorithm;

[0021] Stability test module: used to test the result stability of the knowledge graph entity alignment algorithm based on the selected dataset and knowledge graph entity alignment algorithm;

[0022] Adaptability test module: used to test the adaptability of the knowledge graph entity alignment algorithm based on the selected dataset and knowledge graph entity alignment algorithm.

[0023] Compared with the prior art, the beneficial effects of the present invention are as follows:

[0024] (1) The solution of the present invention proposes a general framework for the knowledge graph entity alignment algorithm, which is used to describe the input, output and basic features of various entity alignment algorithms, making the method applicable to various different types of datasets and algorithms. Eight metamorphic rules are set during the test, which are divided into four main categories: structure, attribute, text, and reasoning. These rules reflect the necessary or expected attributes of the entity alignment algorithm when the dataset changes. Assuming that after applying such metamorphic rules to the knowledge graph, the results of the entity alignment algorithm should not change. By applying these metamorphic rules to the knowledge graph, test cases can be generated to verify the robustness and correctness of the entity alignment algorithm under various modifications of the input data.

[0025] (2) The solution of the present invention selects multiple widely used datasets and multiple popular entity alignment algorithms for comparison. By using multiple datasets, it aims to provide a comprehensive evaluation of the performance of the entity alignment algorithm in different scenarios, including same-language to cross-language alignment, different professional fields, and from small-scale to large-scale knowledge graphs. To conduct a comprehensive comparison, a series of representative entity alignment algorithms are selected to be tested together with the algorithm to be tested, including different implementation technologies such as embedding-based, rule-based, attribute matching, and graph matching, different embedding models, and different design considerations, etc.

[0026] (3) The solution of the present invention includes result stability testing: asking about the changes in the test results of different entity alignment algorithms under different metamorphic relationships, and using a set of entity alignment algorithms to conduct test experiments; applying various transformations that preserve the essence of the alignment task to the input dataset, measuring the Jaccard similarity index of the alignment results obtained by each algorithm and transformation, and analyzing the changes in their results by comparing the similarities obtained by different algorithms and transformations;

[0027] (4) Algorithm adaptability testing: Focusing on the effectiveness of the results of entity alignment algorithms based on different metamorphic relationships, using metrics such as Hits@k, Mean Rank (MR), and Mean Reciprocal Rank (MRR) commonly used in information retrieval and ranking tasks to measure the efficiency of the results by considering the ranking of the correct answers; in the context of entity alignment, considering the correct alignment as the true alignment, if available, or the alignment obtained by a high-performance baseline algorithm, and analyzing how the quality of the alignment changes under different metamorphic relationships by calculating the metrics of each algorithm and transformation;

[0028] (5) The test experimental results show that the metamorphic rules proposed by the present invention can effectively evaluate the performance of entity alignment algorithms, and can discover potential defects and deficiencies of the algorithms; through the method of the present invention, the performance of entity alignment algorithms in different scenarios can be comprehensively evaluated, providing important technical support for the development and optimization of entity alignment algorithms, and improving their reliability and efficiency in practical applications.

[0029] The following further describes the present invention in conjunction with specific embodiments. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 It is a schematic flow chart of the automatic testing method for knowledge graph entity alignment algorithm based on metamorphic relationship of the present invention.

[0031] Figure 2 It is a schematic diagram of the test system architecture applying this method in the embodiment of the present invention.

[0032] Figure 3 It is a schematic diagram of the background technology analysis of the automatic testing of knowledge graph entity alignment algorithm of the present invention.

[0033] Figure 4 It is a schematic diagram of four categories and eight kinds of metamorphic rules in the embodiment of the present invention.

[0034] Figure 5 It is a schematic diagram of the test status in the embodiment of the present invention.

[0035] Figure 6 It is a schematic diagram of the result of the stability test of entity alignment in the embodiment of the present invention.

[0036] Figure 7Schematic diagram of the adaptability test results of entity alignment in the embodiments of the present invention. Detailed implementation manners

[0037] Embodiment

[0038] Next, the technical solutions in the embodiments of the present invention will be clearly and completely described in conjunction with the accompanying drawings in the embodiments of the present invention. The described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative efforts shall fall within the protection scope of the present invention.

[0039] As shown in this application and the claims, unless the context clearly indicates an exceptional situation, the words "a", "an", "one" and / or "the" are not specifically singular, but may also include the plural. Generally speaking, the terms "include" and "comprise" only indicate the inclusion of the clearly identified steps and elements, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.

[0040] Unless otherwise specifically stated, the relative arrangements, numerical expressions and values of the components and steps described in these embodiments do not limit the scope of this application. At the same time, it should be understood that for the convenience of description, the sizes of the various parts shown in the drawings are not drawn according to the actual proportional relationship. Technologies, methods and devices known to those of ordinary skill in the relevant fields may not be discussed in detail, but in appropriate cases, the said technologies, methods and devices should be regarded as part of the authorization specification. In all the examples shown and discussed here, any specific value should be interpreted as merely exemplary, rather than as a limitation. Therefore, other examples of the exemplary embodiments may have different values. It should be noted that: similar reference numerals and letters indicate similar items in the following drawings. Therefore, once an item is defined in one drawing, it does not need to be further discussed in the subsequent drawings.

[0041] Combined with Figures 1 to 3 , an automatic test method for a knowledge graph entity alignment algorithm based on metamorphic relations, adopting the idea of metamorphic testing and realizing automatic testing by designing metamorphic rules, including the following steps:

[0042] Step 1: Set the general framework of the knowledge graph entity alignment algorithm;

[0043] Specifically, in the alignment algorithm, given two knowledge graphs, KG1 = <E1, P1, V1, R1, A1> and KG2 = <E2, P2, V2, R2, A2>;

[0044] Among them, $E$ represents the set of entity names in the knowledge graph, $P$ represents the set of attribute names that include the relationships and attributes between the entities defined above in the knowledge graph, $V$ represents the set of numerical or literal values associated with these attributes in the knowledge graph, $R$ represents the set of relationship triples in the form of $\langle e_1, p, e_2\rangle$, where $e_1, e_2\in E$ and $p\in P$, and $A$ represents the set of attribute triples in the form of $\langle e, a, v\rangle$, where $e\in E$, $a\in P$, and $v\in V$;

[0045] The goal of the entity alignment algorithm is to predict unknown entity alignment relationships, that is, to identify each pair of entities $\langle e_1, e_2\rangle$ such that $e_1\in E_1$ and $e_2\in E_2$, and $e_1$ and $e_2$ represent the same entity. This prediction is made possible by learning from the training set, which plays a central role in this process. The input and output forms of the entity alignment algorithm are given here:

[0046] Input:

[0047] KG1: The first knowledge graph containing entities, relationships, and attributes.

[0048] KG2: The second knowledge graph containing entities, relationships, and attributes.

[0049] Training set: The set of known entity alignment relationships between KG1 and KG2, used to train the algorithm.

[0050] Validation set: The set of known entity alignment relationships used to verify the performance of the algorithm, but not used for training.

[0051] Select a dataset and the knowledge graph entity alignment algorithm, and train the selected knowledge graph entity alignment algorithm;

[0052] After the training of the knowledge graph entity alignment algorithm is completed, input two knowledge graphs;

[0053] Output:

[0054] The predicted entity alignment relationships between the knowledge graphs KG1 and KG2, that is, the entity alignment link $L_{12}=\{\langle e_{i1}, e_{i2}\rangle,...\}$, which is usually sorted according to similarity, distance, or confidence scores.

[0055] Step 2: Set the mutation rules for the knowledge graph alignment algorithm. These rules are classified into four major categories in the context of entity alignment, and it is expected that the entity alignment algorithm should satisfy these rules. These eight rules represent the necessary or expected properties of the entity alignment algorithm. Specifically, it includes the following two parts:

[0056] 1. The processing differences of different types of triples in various entity alignment algorithms are significant. Therefore, two sets of metamorphosis rules are designed to describe the constraints of the entity alignment algorithm on relationship triples and attribute triples.

[0057] 2. Based on the text information and reasoning in the knowledge graph, two additional sets of metamorphic rules were designed. The hypothesis is that "text changes (including language translation and synonym replacement) and reasoning (including adding and deleting triples that can be inferred from existing triples) do not change the semantics."

[0058] In a knowledge graph KGi (i ∈ {1, 2}), its triples can be divided into two main categories: relation triples in Ri and attribute triples in Ai, each playing a different role in defining the graph structure and literal content.

[0059] Combined with Figure 4 , the mutation rules in this embodiment include four categories:

[0060] MR1: Metamorphic rules for relation triples in the knowledge graph, including:

[0061] (1) MR1.1: Adding irrelevant relation triples, that is, introducing new, random, or external relation triples into the knowledge graph KG, which are not connected to the existing relation triples of the entities involved in the entity alignment result. These added relation triples form disconnected components in the knowledge graph structure;

[0062] (2) MR1.2: Deleting irrelevant relation triples, that is, randomly removing from the knowledge graph KG the triples that have no path relation to the entities involved in the entity alignment result, that is, identified as textually irrelevant and path-disconnected; this ensures that the graph structure around these entities remains unchanged.

[0063] MR2: Metamorphic rules for attribute triples in the knowledge graph, including:

[0064] (1) MR2.1: Adding irrelevant attribute triples, that is, randomly assigning new attribute triples to entities that do not appear in the entity alignment result (which can be obtained from the existing entity alignment result data), and these new attributes do not contribute to the alignment process;

[0065] (2) MR2.2: Deleting irrelevant attribute triples, that is, randomly removing attribute triples from entities that are not part of or related to the entity alignment result, and this modification does not affect the attributes related to the alignment task.

[0066] MR3: Metamorphic rules for text information in the knowledge graph, including:

[0067] (1) MR3.1: Translating text values, that is, translating all text values in the knowledge graph from one language to another with equivalent meaning (for example, from English to German). This conversion should preserve the semantic meaning of the text;

[0068] (2) MR3.2: Replace the text value with a synonym, that is, randomly replace the available text value in the knowledge graph with a synonym. If the text is available, randomly replace it with their synonyms.

[0069] MR4: For the metamorphic rules of reasoning in the knowledge graph, including:

[0070] (1) MR4.1: Add the inferred triples, that is, randomly add new triples to the knowledge graph that can be inferred from the existing triples using RDF rules (Resource Description Framework, a knowledge representation language based on triples that can perform simple reasoning tasks and is a W3C recommendation standard).

[0071] (2) MR4.2: Delete the inferred triples. That is, randomly remove from the knowledge graph the triples that can be inferred from other triples using RDF rules.

[0072] Step 3. Combine Figure 5 , select the dataset and specific algorithm for the alignment algorithm;

[0073] In this embodiment, three widely used datasets are selected, including DBP-WD15K, DBP-YG15K, and En-Fr15K. In actual use, it is not limited to these databases, and the method of this solution can be applied to any database; each dataset represents a unique entity alignment scenario: alignment between knowledge graphs within the same domain, alignment between knowledge graphs in different domains, and alignment between knowledge graphs in the same domain but in different languages;

[0074] Selecting these datasets not only conforms to the current research trend but also enables a comprehensive exploration of the nuances of entity alignment under different knowledge graph backgrounds. The three datasets studied are described in detail as follows:

[0075] DW15K: An entity alignment dataset between the DBpedia and Wikidata knowledge graphs. This dataset serves as a benchmark for evaluating entity alignment between two popular and comprehensive knowledge bases.

[0076] DY15K: An entity alignment dataset focusing on entity alignment between the DBpedia and Yago3 knowledge graphs. The diversity of these two sources provides a challenging but valuable test platform for entity alignment algorithms.

[0077] EF15K: An entity alignment dataset designed to align entities between the English version of DBpedia and its French counterpart. This dataset highlights the challenges of cross-lingual entity alignment and evaluates the robustness of algorithms to handle language variations.

[0078] By using these datasets, it aims to provide a comprehensive evaluation of the performance of entity alignment algorithms in different scenarios, ranging from within-language to cross-language alignment, and from small-scale to large-scale knowledge graphs.

[0079] For comparison, a series of representative entity alignment algorithms are selected, including embedding-based, rule-based, attribute matching-based, and graph matching-based techniques. Specifically, the alignment algorithms selected in this embodiment include MTransE, GCN-Align, AttrE, SEA, and IMUSE algorithms.

[0080] Among them:

[0081] MTransE: MTransE is the first translation-based embedded entity alignment model. It uses an embedding module to embed entities and relation predicates in each knowledge graph (KG) into different embedding spaces. The alignment module plays a key role in this process. By minimizing the sum of the alignment score functions of all aligned seed relation triples, it learns cross-KG transformations and unifies these embeddings into a single space.

[0082] GCN-Align: GCN-Align is the first entity alignment research based on graph neural networks (GNNs). It learns entity embeddings from the structural information of entities and utilizes them by treating attribute triples as relation triples.

[0083] AttrE: AttrE is the first technique to use attribute values. Its embedding module uses TransE to learn KG structure embeddings. The main innovation of AttrE is its ability to encode the semantics of attribute values.

[0084] SEA: SEA proposes a semi-supervised method that combines KG embeddings while taking into account the degree differences of entities.

[0085] IMUSE: IMUSE is an unsupervised entity alignment method that utilizes attribute triples and relation triples for alignment without labeled data.

[0086] Step 4. Test the result stability of the knowledge graph entity alignment algorithm based on the selected datasets and knowledge graph entity alignment algorithms:

[0087] Step 4-1. Perform entity alignment operations according to the selected datasets and knowledge graph entity alignment algorithms, and output the entity alignment results;

[0088] Step 4-2. Process the selected datasets based on the set mutation rules, that is, apply four types of mutation rules to the original datasets to generate mutated datasets;

[0089] That is, continuously apply the same type of metamorphic rules to the original dataset until the number of metamorphosed data exceeds a certain proportion of the data, generating a new mutated dataset;

[0090] Based on four types of mutation rules, four new mutated datasets are generated;

[0091] Specifically in this embodiment, four groups of eight metamorphic rules are applied to the original dataset X (X ∈ {DW15K, DY15K, EF15K}). For each group of two metamorphic rules, the metamorphic dataset is established in the following way:

[0092] Continuously apply two metamorphic rules of the same category to the original dataset until the number of deleted and added triples exceeds 50% of the size of the original triple set. For example, under the first group of metamorphic rules, MR1.1 and MR1.2, alternately use MR1.1 and MR1.2, that is, randomly add and delete irrelevant triples until the number of deleted and added triples exceeds 50% of the original size. Under the fourth group of metamorphic rules regarding reasoning, if the number of triples deleted and added based on reasoning cannot exceed 50% of the size of the original triple set, then randomly add and delete irrelevant triples to reach 50%. For the application of the four groups of metamorphic rules, the four new datasets generated are denoted as XMR1, XMR2, XMR3, and XMR4 (that is, data X plus the MR group number, and X is the initial three benchmark datasets).

[0093] And based on the processed dataset, perform entity alignment operations using the knowledge graph entity alignment algorithm, and output the entity alignment result;

[0094] That is, for the selected five algorithms MTransE, GCN - Align, SEA, Imuse, and AttrE, for each benchmark dataset, execute each algorithm on the five datasets (that is, XMR0, XMR1, XMR2, XMR3, and XMR4, where XMR0 represents no change), ensuring that its performance is evaluated under both the original and transformed conditions.

[0095] Step 4 - 3: After obtaining the alignment results of each algorithm from all datasets, calculate the Jaccard similarity coefficient between the alignments generated from the original and metamorphosed datasets, that is, use the Jaccard similarity metric to compare the results obtained by the same algorithm when executed on the original dataset and its metamorphosed version after applying the metamorphic relationship (metamorphic rules). Use the Jaccard similarity metric to compare the results obtained by the same knowledge graph entity alignment algorithm when executed on the original dataset and its metamorphosed version dataset after applying the mutation rules, and complete the stability test:

[0096] J(A,B) = |A ∩ B| / |A ∪ B|

[0097] Among them, A ∩ B represents the intersection of the alignment result A obtained based on the original dataset and the alignment result B obtained based on the mutated dataset, A ∪ B represents the union of A and B, and || represents the cardinality of the set.

[0098] Figure 6 Shows the Jaccard similarity values of five entity alignment algorithms between the original and transformed datasets under four sets of metamorphic rules. There are three subgraphs, and each subgraph shows the results on DW15K, DY15K, and EF15K respectively. The following observations are made:

[0099] GCN - Align shows an abnormally high average similarity on the DY15K variant, indicating that it can effectively utilize the graph structure even under structural changes (MR1) and is robust to text changes introduced by MR2 and MR3.

[0100] Imuse also performs well on the DY15K variant, indicating that its multimodal method is practical and less sensitive to changes introduced by MR1 - MR4.

[0101] MTransE and AttrE perform well on the DY15K dataset but have relatively low average similarity on most variants. This indicates that they may need to be improved to better handle structural and text changes.

[0102] SEA performs well on DY15K but degrades on the EF15K variant, indicating that although it may handle structural changes well, it may be more affected by text changes and may need to adjust its natural language processing components.

[0103] The test experiment results show that: driven by the changes of four sets of metamorphic rules in the dataset, the output results of the algorithms will be different. MR4 seems to be the most challenging for all algorithms, probably because it requires advanced reasoning capabilities. At the same time, if the decline in the original performance is small, the impact of MR3 is the smallest. This indicates that the algorithms are resilient to language translations and synonym replacements that do not change the meaning.

[0104] Step 5: Based on the selected dataset and knowledge graph entity alignment algorithm, test the adaptability of the knowledge graph entity alignment algorithm:

[0105] The goal of the algorithm adaptability test is to answer how the effectiveness of the entity alignment algorithm changes under different metamorphic relationships during the test process. The steps in the present invention aim to provide a comprehensive evaluation framework to guide the selection of entity alignment algorithms based on their effectiveness under different metamorphic transformations.

[0106] Step 5 - 1: Perform entity alignment operations according to the selected dataset and knowledge graph entity alignment algorithm, and output the entity alignment result;

[0107] Step 5-2: Process the selected data set based on the set mutation rules, that is: apply the four types of mutation rules to the original data set to generate a mutated data set;

[0108] That is, continuously apply the same type of metamorphosis rule to the original data set until the number of metamorphosed data exceeds a certain proportion of the data, generating a new mutated data set;

[0109] Generate four new mutated data sets based on the four types of mutation rules, labeled MR1 to MR4;

[0110] Based on the processed data set, perform entity alignment operations using the knowledge graph entity alignment algorithm and output the entity alignment results;

[0111] Step 5-3: For each algorithm and data set variant, multiple metrics are recorded, including three key metrics, namely Hits@k (H@1, H@5, H@10), mean rank, and mean reciprocal rank, to evaluate the results obtained by executing the same knowledge graph entity alignment algorithm on the original data set and its metamorphosed version data set after applying the mutation rules, so as to evaluate the accuracy and effectiveness of entity alignment and complete the algorithm adaptability test.

[0112] Hits@k: Hits@k is a metric that measures the probability of hitting among the top k results. In the context of entity alignment, Hits@k focuses on the proportion of correctly aligned entities among the given k candidate entities. For example, if k = 1, Hits@1 indicates whether the first candidate entity is correctly aligned. Hits@k reflects the ability of the algorithm to quickly find the correctly aligned entity in a limited candidate set. In this embodiment, k = 1, 5, 10, 50 are set.

[0113] MR (mean rank): MR represents the mean rank, which measures the average rank of the correctly aligned entities among all candidate entities. In the entity alignment task, for each entity to be aligned, the algorithm generates a list of candidate entities according to a specific criterion (such as similarity score). The MR metric calculates the average position of the correctly aligned entity in this list. A lower MR value means that the correctly aligned entity usually appears at the top of the list, indicating that the algorithm can more accurately identify the aligned entity.

[0114] MRR (Mean Reciprocal Rank): MRR stands for Mean Reciprocal Rank and is another metric for evaluating ranking performance. Similar to MR, MRR also focuses on the ranking of the correctly aligned entity among all candidate entities, but the calculation method is slightly different. For each query (entity to be aligned), MRR calculates the reciprocal rank of the correctly aligned entity, and then takes the average of the reciprocal ranks of all queries. Compared with MR, MRR is more sensitive to ranking, especially when the correctly aligned entity has a high rank. Therefore, MRR can usually more comprehensively reflect the performance of the algorithm in the entity alignment task.

[0115] Figure 7 The results show that each algorithm responds differently to the transformations applied to the dataset:

[0116] The performance of MTransE shows a slight decline from MR0 to MR1, indicating a certain sensitivity to structural changes. However, it maintains relatively stable performance in MR2 and MR3, indicating robustness to text changes. A more significant decline is observed in MR4, highlighting the challenges of inferring transformations.

[0117] GCN - Align shows high resilience to transformations, with minimal performance degradation in all MR variants. This indicates that GCN - Align can maintain the accuracy of entity alignment even under structural and text changes.

[0118] The performance of SEA decreases as the transformation becomes more complex, with MR4 showing the greatest impact. This indicates that SEA may have difficulty coping with the inference capabilities required by MR4.

[0119] The performance of AttrE drops significantly from MR0 to MR1, indicating high sensitivity to structural changes. It continues to show performance degradation in subsequent MR variants, indicating the need for improvement to handle various types of data changes.

[0120] iMuse maintains high performance in all MR variants, with only a slight decline in MR4. This indicates that iMuse is resilient to the tested transformations and can effectively align entities even in the face of complex changes.

[0121] This step - by - step analysis reveals that the impact of dataset transformations on entity alignment algorithms is significantly different. While some algorithms like GCN - Align and iMuse show high resilience across all transformation types, other algorithms such as SEA and AttrE exhibit performance degradation, especially in inference changes (MR4). AttrE shows high sensitivity to structural changes (MR1), and SEA is affected by the increasing complexity of transformations, indicating the need for better adaptation mechanisms. MR4 poses the greatest challenge to all algorithms and requires advanced inference capabilities for effective entity alignment.

[0122] Analysis in combination with algorithm attributes indicates that the choice of algorithm should be determined according to specific metamorphic rules. GCN - Align and AttrE, due to their ability to integrate additional structural and attribute information, may be more suitable for datasets with rich connections and attributes between entities, which can help eliminate entity ambiguity in transformations. On the other hand, simpler algorithms like MTransE and SEA that do not use predicates or attributes may be more effective for less complex datasets.

[0123] In summary, the present invention uses the method of metamorphic testing to achieve automated testing of entity alignment algorithms. First, eight metamorphic rules are designed, and these rules are classified into four major categories in the context of entity alignment, indicating the necessary or expected attributes that entity alignment algorithms should possess. The test experimental results show that under different metamorphic relationships (metamorphic rules), the output and effectiveness of entity alignment algorithms will vary. Research shows that in the field of entity alignment algorithms, applying metamorphic testing by designing metamorphic rules can effectively solve the problem of the lack of test oracles. It can identify a large number of real - world errors through metamorphic rules, thereby helping to evaluate and enhance current entity alignment algorithms.

[0124] The above - described embodiments merely represent several implementation manners of the present application. Their descriptions are relatively specific and detailed, but should not be construed as limiting the scope of the invention patent. It should be noted that for those of ordinary skill in the art, without departing from the concept of the present application, several modifications and improvements can still be made, and these all belong to the protection scope of the present application. Therefore, the protection scope of the patent of the present application shall be subject to the appended claims.

Claims

1. An automatic testing method for a knowledge graph entity alignment algorithm based on metamorphic relations, characterized in that It includes the following steps: Step 1: Set the general framework of the knowledge graph entity alignment algorithm; Step 2: Set the mutation rules of the knowledge graph alignment algorithm; Step 3: Select the dataset and specific algorithm of the alignment algorithm; Step 4: Based on the selected dataset and knowledge graph entity alignment algorithm, test the result stability of the knowledge graph entity alignment algorithm; Step 5: Based on the selected dataset and knowledge graph entity alignment algorithm, test the adaptability of the knowledge graph entity alignment algorithm.

2. The automatic testing method for the knowledge graph entity alignment algorithm based on the metamorphic relationship according to claim 1, characterized in that In the general framework of the knowledge graph entity alignment algorithm in Step 1, given two knowledge graphs, KG1 = <E1, P1, V1, R1, A1> and KG2 = <E2, P2, V2, R2, A2>; Among them, E represents the set of entity names in the knowledge graph, P represents the set of attribute names including the defined relationships and attributes between entities in the knowledge graph, V represents the set of numerical or literal values associated with these attributes in the knowledge graph, R represents the set of relationship triples in the form <e1, p, e2>, where e1, e2 ∈ E, p ∈ P, and A represents the set of attribute triples in the form <e, a, v>, where e ∈ E, a ∈ P, v ∈ V; Select the dataset and the knowledge graph entity alignment algorithm, and train the selected knowledge graph entity alignment algorithm; After the training of the knowledge graph entity alignment algorithm is completed, input two knowledge graphs, and output the predicted entity alignment relationship between the knowledge graphs KG1 and KG2, that is, the entity alignment link L12 = {<ei1, ei2>,...}.

3. The automatic test method for the knowledge graph entity alignment algorithm based on the metamorphic relationship according to claim 2, wherein, The mutation rules of the knowledge graph alignment algorithm in Step 2 include four categories: The metamorphosis rule for relationship triples in the knowledge graph; The metamorphosis rule for attribute triples in the knowledge graph; The metamorphosis rule for text information in the knowledge graph; The metamorphosis rule for reasoning in the knowledge graph.

4. The automatic testing method for the knowledge graph entity alignment algorithm based on the metamorphic relationship according to claim 3, characterized in that The metamorphosis rule for relationship triples in the knowledge graph includes: (1) Add irrelevant relationship triples, that is, introduce new, random or external relationship triples into the knowledge graph, and these triples are not connected to the existing relationship triples of the entities involved in the entity alignment result. These added relationship triples form disconnected components in the knowledge graph structure; (2) Delete irrelevant relationship triples, that is, randomly remove the relationship triples that have no path to the entities involved in the entity alignment result from the knowledge graph; The metamorphosis rule for attribute triples in the knowledge graph includes: (1) Add irrelevant attribute triples, that is, randomly assign new attribute triples to entities that do not appear in the entity alignment result in the knowledge graph, and these new attributes do not contribute to the alignment process; (2) Delete irrelevant attribute triples, that is, randomly remove attribute triples from entities that are not part of or related to the entity alignment result; The metamorphosis rule for text information in the knowledge graph includes: (1) Translate text values, that is, translate all text values in the knowledge graph from one language into another language with equivalent meaning; (2) Replace the text values with synonyms, that is, randomly replace the available text values in the knowledge graph with synonyms; The metamorphic rules for reasoning in the knowledge graph include: (1) Add inferred triples, that is, randomly add new triples that can be inferred from existing triples using RDF rules to the knowledge graph; (2) Delete inferred triples. That is, randomly remove from the knowledge graph the triples that can be inferred from other triples using RDF rules.

5. The automatic test method for the knowledge graph entity alignment algorithm based on metamorphic relations according to claim 1, wherein The datasets in step 3 include DBP-WD15K, DBP-YG15K, and En-Fr15K; The alignment algorithms include MTransE, GCN-Align, AttrE, SEA, and IMUSE algorithms.

6. The automatic testing method for the knowledge graph entity alignment algorithm based on the metamorphic relationship according to claim 3, characterized in that, Test the result stability of the knowledge graph entity alignment algorithm in step 4: Step 4-1: Perform entity alignment operations according to the selected dataset and the knowledge graph entity alignment algorithm, and output the entity alignment result; Step 4-2: Process the selected dataset based on the set mutation rules, and perform entity alignment operations using the knowledge graph entity alignment algorithm based on the processed dataset, and output the entity alignment result; Step 4-3: Use the Jaccard similarity index to compare the results obtained by the same knowledge graph entity alignment algorithm on the original dataset and its metamorphic version dataset after applying the mutation rules to complete the stability test: J(A,B) = |A∩B| / |A∪B| where A∩B represents the intersection of the alignment result A obtained based on the original dataset and the alignment result B obtained based on the mutated dataset, A∪B represents the union of A and B, and || represents the cardinality of the set.

7. The automatic testing method for the knowledge graph entity alignment algorithm based on metamorphic relations according to claim 6, characterized in that, The processing of the selected dataset based on the set mutation rules is specifically as follows: Apply four types of mutation rules to the original dataset to generate a mutated dataset; That is, continuously apply the same type of metamorphic rule to the original dataset until the number of metamorphosed data exceeds a certain proportion of the data to generate a new mutated dataset; Based on the four types of mutation rules, generate four new mutated datasets.

8. The automatic testing method for the knowledge graph entity alignment algorithm based on metamorphic relations according to claim 3, characterized in that The adaptability test of the knowledge graph entity alignment algorithm in step 5 is specifically as follows: Step 5-1: Perform entity alignment operations according to the selected dataset and the knowledge graph entity alignment algorithm, and output the entity alignment result; Step 5-2: Process the selected dataset based on the set mutation rules, and perform entity alignment operations using the knowledge graph entity alignment algorithm based on the processed dataset, and output the entity alignment result; Step 5-3: Use three key metrics, namely Hits@k, average rank, and average reciprocal rank, to evaluate the results obtained by the same knowledge graph entity alignment algorithm on the original dataset and its metamorphic version dataset after applying the mutation rules to complete the algorithm adaptability test.

9. The automatic test method for the knowledge graph entity alignment algorithm based on the metamorphic relationship according to claim 6, characterized in that The processing of the selected dataset based on the set mutation rules is specifically as follows: Apply four types of mutation rules to the original dataset to generate a mutated dataset; That is, continuously apply the same type of metamorphic rule to the original dataset until the number of metamorphosed data exceeds a certain proportion of the data to generate a new mutated dataset; Four new mutation datasets are generated based on four types of mutation rules.

10. An automatic test system for a knowledge graph entity alignment algorithm based on metamorphic relations, characterized in that, It includes the following modules: Framework construction module: used to set the general framework of the knowledge graph entity alignment algorithm; Rule setting module: used to set the mutation rules of the knowledge graph alignment algorithm; Algorithm determination module: used to select the dataset and specific algorithm of the alignment algorithm; Stability test module: used to test the result stability of the knowledge graph entity alignment algorithm based on the selected dataset and knowledge graph entity alignment algorithm; Adaptability test module: used to test the adaptability of the knowledge graph entity alignment algorithm based on the selected dataset and knowledge graph entity alignment algorithm.

Images