Knowledge graph entity alignment method and system based on large model labeling and active learning

Abstract

The invention provides a knowledge graph entity alignment method and system based on large model labeling and active learning. The method comprises the following steps: an expert searches corresponding entities according to entity characteristics of entities in a representative entity set to obtain positive example entity pairs; based on the positive example entity pair, utilizing a negative sampling strategy and a dynamic deletion strategy to obtain a negative example entity pair and a difference entity pair, and constructing the positive example entity pair, the negative example entity pair and the difference entity pair into a sample pool; labeling the entities and the entity pairs in the sample pool by using a large model to obtain a labeling result; evaluating the labeling quality of the labeling result through an evaluation index; and generating a candidate set by using a KNN algorithm based on the sample pool and the labeling quality. According to the knowledge graph entity alignment method and system based on large model labeling and active learning, the problem that an entity alignment model depends on manual labeling is solved, and the accuracy and reliability of entity alignment are remarkably improved.

Description

Technical Field

[0001] This invention relates to the field of knowledge graph technology, and more specifically, to a method and system for aligning knowledge graph entities based on large model annotation and active learning. Background Technology A knowledge graph is a data model used to structurally describe entities, concepts, attributes, and the relationships between them. It is commonly used to represent knowledge in the real world, providing a foundation for machine understanding and reasoning. As an effective knowledge representation and management tool, knowledge graphs can transform large amounts of unstructured or semi-structured data into structured knowledge, supporting machine learning and artificial intelligence applications.

[0002] The accuracy of a knowledge graph depends on the quality of the data behind it. If the data contains errors or is incomplete, the accuracy of the knowledge graph will be affected. For example, a knowledge graph built by automatically extracting information from the web may contain a large number of errors and inconsistencies.

[0003] Entity alignment is a crucial step in knowledge graph data quality management. Its purpose is to identify and merge multiple different representations of the same real-world entity from different knowledge graphs (i.e., entity disambiguation). Entity alignment ensures consistency of data from different sources within the knowledge graph, avoiding duplication and conflicting information. It also enriches the information in the knowledge graph, integrating entity attributes and relationships scattered across different graphs and improving data integrity. When constructing large-scale knowledge graphs, multiple knowledge graphs from different domains and sources need to be merged. Entity alignment is a key technology for achieving this goal.

[0004] Currently, most mainstream knowledge graph entity alignment methods are based on embedding. These methods typically rely on a large amount of pre-aligned seed data, using neural networks to capture semantic information of entities and achieving matching based on embedding similarity. However, in practical applications, pre-aligned training data is often lacking, making it unavoidable to reduce annotation costs. Furthermore, existing technologies may not fully utilize the correlation information between entities when facing complex entity alignment tasks, thus affecting alignment accuracy. With the emergence of large-scale models, knowledge graph entity alignment has seen new approaches. Automated annotation using entity knowledge from large-scale models can reduce the manual cost of annotation data. However, large-scale models can only generate text sequences and cannot generate candidate sample sets for alignment, and errors may occur during the annotation process, resulting in low-quality annotation data. Therefore, controlling sample quality during the automatic annotation process of large-scale models and achieving high-quality, low-cost training data annotation is key to solving the problem of reliance on manual annotation for entity alignment.

[0005] On the other hand, current active learning-based entity alignment methods all neglect the discussion of annotators, making them unsuitable for entity alignment based on large model annotations. Active learning is a learning method that selects the most valuable data for annotation through interaction with experts. In entity alignment tasks, active learning can help select the most valuable entity pairs for annotation, thereby improving alignment accuracy. However, current research on active learning-based entity alignment is relatively limited and mainly focuses on specific domains or small-scale datasets. How to effectively apply active learning in large-scale, cross-domain entity alignment tasks remains a pressing issue. Summary of the Invention

[0006] To address the aforementioned issues, this invention provides a knowledge graph entity alignment method and system based on large-model annotation and active learning. The large model acts as the active learner and annotator, providing training data to smaller models, thus alleviating the difficulties of deploying large models and the data dependency issues of smaller models. This invention evaluates and filters the quality of labeled data by combining active learning algorithms with large-model annotation, while effectively resisting noise-robust entity alignment interference through a noise-robust entity alignment model, ultimately improving the accuracy and efficiency of entity alignment. Specifically, this invention achieves low-cost, high-quality automated annotation of entity alignment data through the synergistic effect of active learning strategies, noise-robust models, and expert sample pools.

[0007] In summary, this invention effectively reduces the reliance on manual annotation in entity alignment tasks, providing high-quality labeled data by leveraging the world knowledge of large models and low-cost access methods. By designing an active learning strategy and a noise-robust model training method, this invention not only solves the errors that may occur during the annotation process of large models but also significantly improves the training performance of entity alignment models. Ultimately, this invention will significantly improve the accuracy and reliability of knowledge graph entity alignment, promoting the widespread application of knowledge graphs in large-scale, cross-domain entity alignment tasks.

[0008] In a first aspect, the present invention provides a knowledge graph entity alignment method based on large model annotation and active learning, characterized in that the method includes: Experts search for corresponding entities based on the entity characteristics of entities in a representative entity set to obtain positive entity pairs; Based on the positive entity pairs, negative entity pairs and differential entity pairs are obtained using a negative sampling strategy and a dynamic deletion strategy, and the positive entity pairs, the negative entity pairs, and the differential entity pairs are constructed into a sample pool; The entities and entity pairs in the sample pool are labeled using a large model to obtain the labeling results. The quality of the annotation results is evaluated using assessment metrics. Based on the sample pool and the annotation quality, a candidate set is generated using the KNN algorithm; By calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entity, the uncertainty of the target entity in the large model is obtained. Training data is selected from the target entities based on the uncertainty and entity value, and a statistical model is trained using this data to predict the labeling accuracy of the large model. Based on the annotation accuracy, the high-quality annotation types of the large model are obtained; The large model selects entities to be labeled based on the high-quality labeling type to obtain labeled data.

[0009] Secondly, the present invention also provides a knowledge graph entity alignment system based on large model annotation and active learning, characterized in that the system comprises: The lookup device is used by experts to find the corresponding entities based on the entity characteristics of entities in a representative entity set, and to obtain positive entity pairs. A construction apparatus is used to obtain negative entity pairs and differential entity pairs based on the positive entity pairs using a negative sampling strategy and a dynamic deletion strategy, and to construct the positive entity pairs, the negative entity pairs and the differential entity pairs into a sample pool; Apparatus for labeling entities and entity pairs in the sample pool using a large model to obtain labeling results; Apparatus for evaluating the annotation quality of the annotation results using evaluation metrics; A generation device for generating a candidate set based on the sample pool and the annotation quality using the KNN algorithm; A computing device for obtaining the uncertainty of a large model for a target entity by calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entity. A prediction device for selecting training data from the target entity based on the uncertainty and entity value, and using it to train a statistical model to predict the labeling accuracy of a large model; Apparatus for obtaining high-quality annotation types for the large model based on the annotation accuracy; An apparatus for selecting entities to be labeled based on the high-quality annotation type in the large model to obtain labeled data.

[0010] This invention provides a knowledge graph entity alignment method and system based on large-model annotation and active learning: First, by using a large-model-driven annotation method, the need for manual annotation is reduced, labor costs are lowered, and the automation level of entity alignment tasks is improved. Second, by using a large-model annotation quality evaluation method based on an expert sample pool, the annotation capabilities of large models are classified and evaluated, improving the adaptability of the annotation framework and making the annotation results of large models more reliable. Third, by combining a candidate set generation strategy based on KNN and heuristic rules, an active learning strategy, and an entity feature modeling method, the generation of entity alignment candidate sets is made more accurate, improving the final matching effect of the entity alignment task. Fourth, by using pre-trained language models and knowledge graph embedding technology to reduce computational overhead, and by using negative sampling and dynamic deletion strategies to expand the sample pool, more efficient annotation and model training are achieved, reducing overall computational resource consumption. Fifth, an active learning algorithm based on a large model is adopted to comprehensively measure the value and difficulty of sample labeling, achieving high-quality, high-value data labeling and improving the accuracy and reliability of training data. Sixth, through noise label learning technology and isolated node identification technology, the entity alignment model can maintain good training performance even with a small number of noise labels, improving the model's stability and generalization ability. Seventh, multiple entity features (structural information, attribute information, etc.) are used for similarity calculation, and multi-hop graph neural networks and graph attention networks (GATs) are used for feature fusion, making the entity alignment method of this invention applicable to different types of knowledge graphs and improving its versatility. Eighth, by improving the accuracy and reliability of entity alignment, the widespread application of knowledge graphs in fields such as artificial intelligence, natural language processing, and recommendation systems is promoted, driving the development and practical application of knowledge graph technology. Attached Figure Description

[0011] To more clearly illustrate the technical solutions in the embodiments of the present invention, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0012] Figure 1 This is a flowchart of the knowledge graph entity alignment method based on large model annotation and active learning provided in the embodiments of the present invention; Figure 2 This is a block diagram of a knowledge graph entity alignment system based on large model annotation and active learning provided in an embodiment of the present invention. Detailed Implementation

[0013] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Invention Overview As mentioned above, this invention provides a knowledge graph entity alignment method and system based on large model annotation and active learning, which solves the problem of entity alignment model dependence on manual annotation and significantly improves the accuracy and reliability of entity alignment.

[0015] Exemplary methods Figure 1 This is a flowchart of a knowledge graph entity alignment method based on large model annotation and active learning provided in an embodiment of the present invention. This embodiment includes the following steps: S101: Experts search for the corresponding entities based on the entity characteristics of entities in the representative entity set to obtain positive entity pairs.

[0016] The entity features include entity category, entity relationship, node centrality, PageRank score, and / or number of attributes.

[0017] Prior to S101, the method further includes: Using diversity sampling techniques, a representative set of entities is selected from the knowledge graph based on the entity features using heuristic rules, thus obtaining a representative entity set. This approach enables diverse entity sampling. Compared to randomly selecting entities, choosing a representative set of entities more effectively captures the entirety of the knowledge graph, reduces the amount of data required, and improves model training efficiency. By analyzing entity features, it ensures that the selected entity set covers various types and patterns in the knowledge graph, avoiding bias towards entities of a particular category or feature. This ensures that large models receive sufficiently diverse information without requiring large amounts of data, thereby improving the generalization ability and performance of large models.

[0018] Experts then group entities based on representative entities. Find the corresponding entity based on the entity features. And construct a positive entity alignment set Entities and their corresponding entities are different representations of the same real-world entity from different knowledge graphs, and positive example entity pairs are matching entity pairs. Generally, experts can easily find the corresponding entities; if it is difficult to obtain the corresponding entities for certain entities in the representative entity set, they are ignored. By labeling only positive examples, the number of times experts need to label can be effectively reduced, saving costs.

[0019] S102: Based on the positive entity pairs, negative entity pairs and differential entity pairs are obtained using a negative sampling strategy and a dynamic deletion strategy, and the positive entity pairs, the negative entity pairs, and the differential entity pairs are constructed into a sample pool.

[0020] To evaluate the performance of large models in labeling multiple entity types and multiple entity pair types, this invention uses two strategies: negative sampling strategy and dynamic deletion strategy, to expand the sample data.

[0021] The negative sampling strategy includes: randomly replacing one entity in the positive entity pair, such as replacing a given positive entity pair with another entity. Replace with negative example entity pairs and Negative example entity pairs are non-matching entity pairs. To further increase the deceptiveness of negative example entity pairs, one entity in the positive example entity pair is replaced with its k-neighbor node, which is an entity similar to it in the knowledge graph. The k-neighbor node is selected based on a similarity matrix (such as a mean matrix) and is usually the top-k nearest neighbor of the entity. And / or combined with traditional embedded training methods, multiple negative example samplings are added to more comprehensively cover the negative example space, improving the robustness and generalization ability of large models. Increasing the number of negative example samples can accelerate the model's convergence process.

[0022] The dynamic deletion strategy involves constructing the differential entity pairs by deleting and / or modifying at least one of the entity features of the entities in the positive entity pairs, in order to uncover entity features that play a key role in large model entity alignment inference.

[0023] The difference entity pairs include information content difference entity pairs, structural difference entity pairs, attribute difference entity pairs, and / or attribute value difference entity pairs.

[0024] Entity pairs with varying information content are constructed by randomly deleting portions of descriptive information, resulting in pairs with different levels of information completeness. For example, one entity may have a complete textual description, while another entity may retain only a partial description.

[0025] Structurally differentiated entity pairs are constructed by deleting some of the associated nodes or edges of an entity in the knowledge graph, creating entity pairs with different topological structures. For example, one entity's neighbor nodes are deleted, while the other entity retains its complete structural information.

[0026] Attribute-differentiated entity pairs are created by selectively removing certain attribute fields from entities, resulting in entity pairs with different attribute information. For example, one entity has "date of birth" and "occupation," while another entity only retains "occupation."

[0027] Attribute value difference entity pairs involve modifying the values ​​of the same attribute (e.g., partially missing values, partially perturbed values) to examine the sensitivity of the large model to fine-grained attribute information. For example, one entity's "nationality" is "China", while another entity's "nationality" is "not provided".

[0028] Through these different levels of information deletion strategies, various different types of differential entity pairs can be formed, and then the annotation capabilities of the large model in different information loss scenarios can be deeply analyzed, providing data support for optimizing the annotation capabilities of the large model.

[0029] S103: Use the large model to annotate the entities and entity pairs in the sample pool to obtain an annotation result.

[0030] That is, the large model annotates various entity types (such as person names, place names, and organizations, etc.) and various entity pair types (such as whether they match). For example, it is annotated that "Beijing" and "北京" match, and "Apple (company)" and "Apple (fruit)" do not match.

[0031] S104: Evaluate the annotation quality of the annotation result through evaluation metrics.

[0032] The evaluation metrics include accuracy, recall rate, false positive rate, false negative rate, and / or F value.

[0033] Accuracy (Match Precision) is the proportion of correct ones in the matching results.

[0034] Recall rate (Match Recall) is the proportion of positive examples that can be correctly matched.

[0035] False positive rate (False Positive Rate, FPR) is the probability of misjudging a non-matching entity as a matching one.

[0036] False negative rate (False Negative Rate, FNR) is the probability of misjudging a matching entity as a non-matching one.

[0037] F1 value (F1-score) is a balanced metric that combines accuracy and recall rate.

[0038] S105: Generate a candidate set using the KNN algorithm based on the sample pool and the annotation quality.

[0039] Specifically, according to the assumption that matching entity pairs have similar structural information and attribute information, the present invention generates a candidate set based on KNN and heuristic rules. Use a pre-trained language model (such as BERT, RoBERTa, etc.) to obtain the embedded vector representation of entity attributes, and then use vector cosine to obtain the similarity matrix of entity attributes ; Similarly, use a classic knowledge graph embedding model (such as TransE, RotatE, ConvE, etc.) to obtain the vector representation of entity structural information, and calculate the similarity matrix of entity structural information If the entity name exists, use a pre-trained language model to obtain the vector representation of the entity name and calculate the similarity matrix of the entity names. Using the aforementioned sample pool as a bridge between two knowledge graphs, the embedding vector representations of entity attributes, entity structure information, and entity names are fine-tuned. Considering the limitations of the embedding model, the similarity representation of entity vectors cannot fully represent the similarity of entities. Three types of heuristic features are used: node features (such as node centrality and entity type), graph structure features (such as PageRank score), and attribute features (such as attribute name and keywords). Basic distance calculation methods such as Euclidean distance and edit distance are used on these three types of features to calculate the similarity of each feature and form a similarity matrix. Finally, based on the mean matrix The KNN algorithm is used to select the top-k nearest neighbors for each entity, generating the candidate set E. These neighbors include positive entity pairs, negative entity pairs, and dissimilar entity pairs.

[0040] The mean matrix integrates multiple features such as attributes, structure, and name, capturing different aspects of entities. This multi-dimensional similarity calculation allows KNN to select neighbors that may be similar in some features but dissimilar in others, covering different feature combinations, scenarios, and situations. For example, consider entity pairs A and B with similar names but dissimilar attributes and structures: Entity A: Name "Apple", Attribute "Technology Company", Structure "Associated with iPhone"; Entity B: Name "Apple", Attribute "Fruit", Structure "Associated with Health Food".

[0041] Although the candidate set generation strategy of this invention is relatively complex, it boasts high accuracy, thus reducing annotation costs. Furthermore, through data preprocessing and offline representation learning, a mean matrix is ​​pre-generated. This reduces the time cost of active learning. Furthermore, both positive and negative examples in the candidate set can be used to train the statistical model, helping to improve its accuracy without wasting resources. Finally, as the amount of training data increases, the performance of the entity alignment model improves, allowing for a reduction in the candidate set size and saving on annotation costs.

[0042] Then, a greedy algorithm is used to select the highest value sample from the candidate set E. As the target entity.

[0043] S106: By calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entity, the uncertainty of the target entity in the large model is obtained.

[0044] This invention considers the uncertainty of samples, i.e., sample difficulty. In traditional active learning frameworks, models typically assume the existence of an omniscient and omnipotent annotator (i.e., an "oracle") capable of accurately labeling selected samples. In this case, the primary goal of active learning is to improve model performance by selecting high-value samples, without considering the labeling difficulty. However, with the advent of large-scale pre-trained models, the labeling process has become more complex, and the capabilities and reliability of the annotator may be limited, making it impossible to guarantee accurate labeling of all samples.

[0045] Therefore, it is necessary to extend the traditional active learning framework to take annotation difficulty into account. This extended active learning framework needs to evaluate the annotation difficulty of each sample in order to make more informed decisions when selecting training data. By considering annotation difficulty, it is possible to avoid selecting samples that are too difficult or error-prone, thereby improving the training efficiency and performance of the model.

[0046] In active learning, this invention prioritizes the annotation of instances with high model uncertainty, effectively improving the model's learning efficiency and performance. By focusing on entities with high uncertainty, resources are allocated more effectively, reducing annotation costs. By identifying and processing highly uncertain entities, the model can better learn complex or fuzzy patterns, thereby improving overall performance and alignment accuracy.

[0047] Specifically, the following formula is used to calculate the target entity in the large model. Uncertainty :

[0048] in Represents the target entity The The nearest neighbor entities; align is a preset alignment function (such as the SVM function) used to determine whether entity pairs are similar, and will return a value between 0 and 1. The higher the value, the more similar the entity pairs are.

[0049] When the model has strong confidence in its judgments, the entity With corresponding entity Alignment It will be much larger than other nearby entities. Alignment Uncertainty The higher the value, the lower the ability of the large model to distinguish between the corresponding entity and its nearest neighbor entities.

[0050] This invention not only calculates the alignment between a target entity and its corresponding entity, but also considers the alignment between the target entity and its nearest neighbor entities, providing a more comprehensive evaluation of the large model's understanding and processing capabilities for target entities. Calculating only the alignment between the target entity and its corresponding entity might overlook the model's performance when handling other similar entities. By introducing the alignment of nearest neighbor entities, deviations in the model across different entity types can be identified, avoiding over-reliance on the annotation results of single entity pairs. Furthermore, considering nearest neighbor entities helps capture semantic relationships and contextual information between entities. This approach can improve the model's performance in complex scenarios, especially when dealing with ambiguous or uncertain entities. Additionally, by comparing the alignment between the target entity and its corresponding and nearest neighbor entities, weaknesses in the model regarding specific domains or entity types can be identified. This facilitates targeted selection of training data and improves the model's performance in key domains.

[0051] S107: Select training data from the target entity based on the uncertainty and entity value, and use it to train a statistical model to predict the annotation accuracy of the large model.

[0052] Computational model for entities Uncertainty While it's possible to select difficult entities, this can also lead to the selection of structurally unimportant entities. Therefore, this invention incorporates graph structure into the entity value measurement standard; specifically, it uses entities... PageRank Score ( The importance of nodes is measured by the product of the uncertainty of the target entity and the PageRank Score of the target entity. Select training data from the target entity.

[0053] In summary, this invention combines the uncertainty (difficulty) of a sample with its importance (value) within the graph structure, enabling more effective selection of training data. By using the product of the target sample's uncertainty and its PageRank Score as a comprehensive indicator of sample value, it ensures that the selected samples are both challenging and occupy an important position in the network structure, thereby improving the overall performance of the model. That is, unlike existing technologies that only focus on sample value, this invention considers both the value and difficulty of a sample during the training data selection process. By comprehensively evaluating these two factors, a more effective training set can be constructed, improving the model's predictive ability and generalization performance.

[0054] The statistical model (such as SVM) is trained using machine learning. For the true labels of entity pairs, The labels assigned to the large model indicate the accuracy of the large model's labeling. for:

[0055] in for The feature vector includes a discrete feature vector and an embedded similarity feature vector. The discrete features include out-degree, in-degree, and / or PageRank. The embedded similarity features include entity attributes, entity structure, and / or entity name.

[0056] Additionally, an information deletion strategy can be used, which involves removing certain entity features while ensuring the large model is correctly labeled, until the model malfunctions. This method can uncover key relationship and attribute information that is helpful for the large model's judgment and incorporate them into discrete feature vectors.

[0057] S108: Obtain the high-quality annotation type of the large model based on the annotation accuracy.

[0058] The accuracy of large-scale model annotation across various entity types and entity pairs is statistically analyzed. By comparing these accuracies, we can identify which types of entities or entity pairs the large model performs better, i.e., high-quality annotation types. This clarifies the strengths of the large model, allowing it to be utilized in subsequent big data annotation. For example, large models perform differently across different domains and with varying levels of information completeness; they have higher accuracy in annotating celebrities but lower accuracy in annotating satellites, thus celebrity annotations are considered high-quality. Large models also have high accuracy in annotating entities with complete information such as names and dates of birth, but are prone to errors when annotating entities with only nicknames.

[0059] S109: The large model selects entities to be labeled according to the high-quality labeling type to obtain labeled data.

[0060] After identifying high-quality annotation types, the large model selects entities to be labeled based on these types. This selection strategy aims to fully leverage the large model's strengths in high-quality annotation types, ensuring that the obtained labeled data is of high quality and reliability. By focusing on annotation types that the large model excels at, this invention can improve overall annotation efficiency, ensure data quality, and provide a solid foundation for subsequent training and application of small entity alignment models.

[0061] Even after an active learning strategy based on large model annotation is determined, the resulting labeled data inevitably contains a small amount of noise. To overcome the impact of noisy labels on entity alignment of small models, the method may further include: Based on the labeled data, a small entity alignment model is trained by constructing a model using an isolated node detection mechanism, a noise label learning algorithm, and a multi-hop graph neural network.

[0062] Specifically, Dropout and cross-entropy loss are used to filter out noiseless data from the labeled data; a feedforward neural network predictor is trained to identify isolated nodes in the noiseless data, and margin-based loss is used to maintain the relatively independent representation of the isolated nodes in the vector space to obtain high-quality data; a multi-hop graph neural network is used to extract the structural and attribute information of entities in the high-quality data to obtain the entity embedding representation; a negative sampling strategy and margin-based loss function are used to train the entity alignment mini-model, and the entities from the two knowledge graphs are mapped to a unified vector space based on the embedding representation to obtain vector representation; the similarity between entities is calculated based on the vector representation, and the most matching entity pairs are output according to the similarity ranking.

[0063] For example, by estimating epistemic uncertainty in the model through Dropout and filtering out noiseless data using the cross-entropy loss function, only minor adjustments to the model structure and training framework are needed, with no significant additional overhead. Considering the potential for orphan nodes in knowledge graphs, an orphan node detection mechanism is introduced. Orphan nodes are considered noisy nodes in entity alignment tasks. Therefore, to ensure orphan nodes have independent representations in the knowledge graph embedding space and obtain high-quality data—that is, to maintain a certain distance between orphan nodes and their surrounding embedded nodes—a margin-based loss function is used to train an orphan node predictor based on feedforward neural networks (FNNs). This margin-based loss function ensures that orphan nodes maintain relatively independent representations in the vector space, effectively improving data quality and ensuring that orphan nodes maintain an appropriate distance from their surrounding embedded nodes. During training, the loss function of the orphan node predictor and the loss function of the entity alignment mini-model are jointly trained. A multi-hop graph neural network is used to build the model, and entity alignment training is performed in a unified vector space. The multi-hop neural network mechanism can effectively capture the structural relationships between entities and achieve attribute aggregation. Unlike traditional single-hop neighbor attention mechanisms, multi-hop attention mechanisms can capture remote interactions between nodes (interaction information between multi-hop neighbors) at each layer, achieving effective remote message passing. This allows the model to capture a wider range of structural information at each layer, improving representation learning performance. To reduce parameter overfitting, convolutional graph neural networks (GCNs) are used to handle message passing and updates between entity attributes. Regarding entity attribute information, graph attention networks (GATs) are used to assign different weights to the relationships between entity attributes, achieving aggregated entity representations. This results in embedded entity representations. For model training, two knowledge graphs are embedded into a unified vector space, and a negative sampling strategy is used to achieve end-to-end model training based on labeled training data. An entity alignment training method based on the unified vector space is used. Specifically, for two different knowledge graphs, the aforementioned entity attribute encoding method is used to obtain vector representations of entities in the unified space. After obtaining all entity encodings, seed data alignment is performed using pre-given seed entity pairs; a margin-based loss function and a negative sampling strategy are used to achieve end-to-end model parameter training. Margin-based loss functions introduce adaptive margins, enabling the model to flexibly adapt to different data distributions and improving its robustness and generalization ability. Through training and optimization based on seed data, a unified spatial vector representation of all entities is ultimately obtained. Based on this, the similarity between entities is calculated, and the top-ranked entity pairs are output according to their similarity scores.

[0064] In summary, the present invention improves the performance of small entity alignment models through the above steps, enhances their robustness to noisy data, and effectively captures complex structural and attribute relationships between entities.

[0065] Exemplary System Accordingly, embodiments of the present invention also provide a knowledge graph entity alignment system based on large model annotation and active learning. Figure 2 This is a block diagram of a knowledge graph entity alignment system based on large model annotation and active learning provided in an embodiment of the present invention, such as... Figure 2 As shown, the system 100 provided in this embodiment includes: The lookup device 101 is used by experts to find the corresponding entity based on the entity characteristics of the entity in the representative entity set, and obtain positive entity pairs. The construction device 102 is used to obtain negative entity pairs and differential entity pairs based on the positive entity pairs using a negative sampling strategy and a dynamic deletion strategy, and to construct the positive entity pairs, the negative entity pairs and the differential entity pairs into a sample pool. Apparatus for labeling entities and entity pairs in the sample pool using a large model to obtain labeling results; Apparatus for evaluating the annotation quality of the annotation results using evaluation metrics; Generation device 103 is used to generate a candidate set based on the sample pool and the annotation quality using the KNN algorithm; The computing device 104 is used to obtain the uncertainty of the target entity in the large model by calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entity. Prediction device 105 is used to select training data from the target entity based on the uncertainty and entity value, and use it to train a statistical model to predict the labeling accuracy of a large model. Apparatus for obtaining high-quality annotation types for the large model based on the annotation accuracy; An apparatus for selecting entities to be labeled based on the high-quality annotation type in the large model to obtain labeled data.

[0066] The entity features include entity category, entity relationship, node centrality, PageRank score, and / or number of attributes.

[0067] The system 100 also includes: An apparatus for selecting a representative set of entities from a knowledge graph based on the entity features using heuristic rules, thereby obtaining a representative set of entities.

[0068] The evaluation metrics include accuracy, recall, false positive rate, false negative rate, and / or F-score.

[0069] The negative sampling strategy includes: Randomly replace one of the entities in the positive example entity pair; Replace one of the entities in the positive entity pair with its k-neighbor node; and / or By combining traditional embedded training methods, multiple negative sample samplings are added.

[0070] The dynamic deletion strategy is to construct the differential entity pair by deleting and / or modifying at least one of the entity features of the entities in the positive entity pair; The difference entity pairs include information content difference entity pairs, structural difference entity pairs, attribute difference entity pairs, and / or attribute value difference entity pairs.

[0071] The search device 101 is also used to ignore entities if it is difficult to obtain the corresponding entities of certain entities in the representative entity set.

[0072] The generating device 103 includes: This is used to obtain embedding vector representations of entity attributes using a pre-trained language model, and to obtain the similarity matrix of entity attributes using vector cosine. Units; This is used to obtain vector representations of entity structure information using classic knowledge graph embedding models, and to calculate the similarity matrix of entity structure information. Units; If an entity name exists, this method uses a pre-trained language model to obtain the vector representation of the entity name and calculate the similarity matrix of the entity names. Units; A unit for fine-tuning the embedded vector representation of entity attributes, the vector representation of entity structure information, and the vector representation of entity name using the sample pool; This method uses basic distance calculation to compute the similarity of each feature on node features, graph structure features, and attribute features, and then constructs a similarity matrix. Units; Used for mean matrix The KNN algorithm is used to select the top-k nearest neighbors of each entity to generate the candidate set of units.

[0073] The computing device 104 is also used to calculate the target entity using the large model according to the following formula. Uncertainty :

[0074] in Represents the target entity The The nearest neighbor entities, where align is a preset alignment function.

[0075] The preset alignment function is the SVM function; The statistical model is SVM.

[0076] The system 100 also includes: A means for selecting the target entity from the candidate set using a greedy algorithm.

[0077] The prediction device 105 is also used to train the statistical model using machine learning, assuming... For the true labels of entity pairs, The labels assigned to the large model indicate the accuracy of the large model's labeling. for:

[0078] in for The feature vector includes a discrete feature vector and an embedded similarity feature vector. The discrete features include out-degree, in-degree, and / or PageRank. The embedded similarity features include entity attributes, entity structure, and / or entity name.

[0079] The entity value is the PageRank Score of the target entity.

[0080] The system 100 also includes: The denoising device 106 is used to train a small entity alignment model based on the labeled data through an isolated node detection mechanism, a noise label learning algorithm, and a multi-hop graph neural network.

[0081] The noise reduction device 106 further includes: A unit for filtering out noise-free data from the labeled data using Dropout and cross-entropy loss; This is used to train a feedforward neural network predictor to identify isolated nodes in the noiseless data, and to use margin-based loss to maintain the relatively independent representation of the isolated nodes in the vector space, thereby obtaining units of high-quality data. A unit used to extract the structural and attribute information of entities in the high-quality data using a multi-hop graph neural network, and to obtain the embedded representation of the entities; This is used to train the entity alignment small model using a negative sampling strategy and a margin-based loss function, and to map entities from two knowledge graphs to a unified vector space based on the embedding representation to obtain units of vector representation; A unit used to calculate the similarity between entities based on the vector representation, sort them according to the similarity, and output the most matching entity pair.

[0082] It should be noted that although the operations of the knowledge graph entity alignment method based on large model annotation and active learning of the present invention are described in a specific order in the accompanying drawings, this does not require or imply that these operations must be performed in that specific order, or that all of the operations shown must be performed to achieve the desired result. Additionally or alternatively, certain steps may be omitted, multiple steps may be combined into one step, and / or one step may be broken down into multiple steps.

[0083] Furthermore, although several devices, units, or modules of a knowledge graph entity alignment system based on large model annotation and active learning have been mentioned in the detailed description above, this division is merely exemplary and not mandatory. In fact, according to embodiments of the present invention, the features and functions of two or more modules described above can be embodied in one module. Conversely, the features and functions of one module described above can be further divided and embodied by multiple modules.

[0084] While the spirit and principles of the invention have been described with reference to several specific embodiments, it should be understood that the invention is not limited to the disclosed specific embodiments, and the division of aspects does not imply that features in these aspects cannot be combined for benefit; such division is merely for ease of description. The invention is intended to cover various modifications and equivalent arrangements included within the spirit and scope of the appended claims.

[0085] This invention provides: 1. A knowledge graph entity alignment method based on large model annotation and active learning, characterized in that the method includes: Experts search for corresponding entities based on the entity characteristics of entities in a representative entity set to obtain positive entity pairs; Based on the positive entity pairs, negative entity pairs and differential entity pairs are obtained using a negative sampling strategy and a dynamic deletion strategy, and the positive entity pairs, the negative entity pairs, and the differential entity pairs are constructed into a sample pool; The entities and entity pairs in the sample pool are labeled using a large model to obtain the labeling results. The quality of the annotation results is evaluated using assessment metrics. Based on the sample pool and the annotation quality, a candidate set is generated using the KNN algorithm; By calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entity, the uncertainty of the target entity in the large model is obtained. Training data is selected from the target entities based on the uncertainty and entity value, and a statistical model is trained using this data to predict the labeling accuracy of the large model. Based on the annotation accuracy, the high-quality annotation types of the large model are obtained; The large model selects entities to be labeled based on the high-quality labeling type to obtain labeled data.

[0086] 2. The knowledge graph entity alignment method based on large model annotation and active learning as described in item 1, characterized in that the entity features include entity category, entity relationship, node centrality, PageRank score and / or number of attributes.

[0087] 3. The knowledge graph entity alignment method based on large model annotation and active learning as described in item 2, characterized in that, before the step of the expert finding the corresponding entity based on the entity features of the entity in the representative entity set to obtain positive example entity pairs, the method further includes: Based on the entity features, a representative entity set is obtained by selecting a representative entity set from the knowledge graph using heuristic rules.

[0088] 4. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of items 1-3, characterized in that the evaluation index includes accuracy, recall, false positive rate, false negative rate and / or F-score.

[0089] 5. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of items 1-3, characterized in that the negative sampling strategy includes: Randomly replace one of the entities in the positive example entity pair; Replace one of the entities in the positive entity pair with its k-neighbor node; and / or By combining traditional embedded training methods, multiple negative sample samplings are added.

[0090] 6. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of items 1-3, characterized in that the dynamic deletion strategy is to construct the differential entity pair by deleting and / or modifying at least one of the entity features of the entities in the positive entity pair; The difference entity pairs include information content difference entity pairs, structural difference entity pairs, attribute difference entity pairs, and / or attribute value difference entity pairs.

[0091] 7. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of items 1-3, characterized in that the step of experts finding the corresponding entity based on the entity features of entities in a representative entity set to obtain positive example entity pairs specifically further includes: If it is difficult to obtain the corresponding entity for some entities in the representative entity set, then they are ignored.

[0092] 8. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of items 1-3, characterized in that the step of generating a candidate set using the KNN algorithm based on the sample pool and the annotation quality specifically includes: The embedding vector representations of entity attributes are obtained using a pre-trained language model, and the similarity matrix of entity attributes is obtained using vector cosine. ; We use a classic knowledge graph embedding model to obtain vector representations of entity structure information and calculate the similarity matrix of entity structure information. ; If entity names exist, use a pre-trained language model to obtain the vector representation of the entity names and calculate the similarity matrix of the entity names. ; The sample pool is used to fine-tune the embedding vector representation of entity attributes, the vector representation of entity structure information, and the vector representation of entity name; The basic distance calculation method is used on node features, graph structure features, and attribute features to calculate the similarity of each feature and form a similarity matrix. ; Based on the mean matrix The KNN algorithm is used to select the top-k nearest neighbors of each entity to generate the candidate set.

[0093] 9. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of items 1-3, characterized in that the step of obtaining the uncertainty of the large model for the target entity by calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entities, specifically comprises: The following formula is used to calculate the target entity in the large model. Uncertainty :

[0094] in Represents the target entity The The nearest neighbor entities, where align is a preset alignment function.

[0095] 10. The knowledge graph entity alignment method based on large model annotation and active learning as described in item 9, characterized in that the preset alignment degree function align is an SVM function; The statistical model is SVM.

[0096] 11. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of items 1-3, characterized in that, before the step of obtaining the uncertainty of the large model for the target entity by calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entities, the method further includes: The target entity is selected from the candidate set using a greedy algorithm.

[0097] 12. The knowledge graph entity alignment method based on large model annotation and active learning as described in item 10, characterized in that the step of selecting training data from the target entities based on the uncertainty and entity value, and using it to train a statistical model to predict the annotation accuracy of the large model specifically includes: The statistical model is trained using machine learning, assuming... For the true labels of entity pairs, The labels assigned to the large model indicate the accuracy of the large model's labeling. for:

[0098] in for The feature vector includes a discrete feature vector and an embedded similarity feature vector. The discrete features include out-degree, in-degree, and / or PageRank. The embedded similarity features include entity attributes, entity structure, and / or entity name.

[0099] 13. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of items 1-3, wherein the entity value is the PageRank Score of the target entity.

[0100] 14. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of items 1-3, characterized in that, after the step of selecting entities to be annotated according to the high-quality annotation type and obtaining the annotation data, the method further includes: Based on the labeled data, a small entity alignment model is trained by constructing a model using an isolated node detection mechanism, a noise label learning algorithm, and a multi-hop graph neural network.

[0101] 15. The knowledge graph entity alignment method based on large model annotation and active learning as described in item 14, characterized in that the specific steps of constructing a model for entity alignment and training a small model based on the labeled data through an isolated node detection mechanism, a noise label learning algorithm, and a multi-hop graph neural network are as follows: Noise-free data was filtered out from the labeled data using Dropout and cross-entropy loss; A feedforward neural network predictor is trained to identify isolated nodes in the noiseless data, and a margin-based loss is used to maintain the relatively independent representation of the isolated nodes in the vector space, thereby obtaining high-quality data. A multi-hop graph neural network is used to extract the structural and attribute information of entities in the high-quality data to obtain the embedded representation of the entities; The entity alignment mini-model is trained using a negative sampling strategy and a margin-based loss function. Based on the embedding representation, entities from two knowledge graphs are mapped to a unified vector space to obtain vector representations. The similarity between entities is calculated based on the vector representation, and the entities are sorted according to the similarity to output the most matching entity pairs.

[0102] 16. A knowledge graph entity alignment system based on large model annotation and active learning, characterized in that the system comprises: The lookup device is used by experts to find the corresponding entities based on the entity characteristics of entities in a representative entity set, and to obtain positive entity pairs. A construction apparatus is used to obtain negative entity pairs and differential entity pairs based on the positive entity pairs using a negative sampling strategy and a dynamic deletion strategy, and to construct the positive entity pairs, the negative entity pairs and the differential entity pairs into a sample pool; Apparatus for labeling entities and entity pairs in the sample pool using a large model to obtain labeling results; Apparatus for evaluating the annotation quality of the annotation results using evaluation metrics; A generation device for generating a candidate set based on the sample pool and the annotation quality using the KNN algorithm; A computing device for obtaining the uncertainty of a large model for a target entity by calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entity. A prediction device for selecting training data from the target entity based on the uncertainty and entity value, and using it to train a statistical model to predict the labeling accuracy of a large model; Apparatus for obtaining high-quality annotation types for the large model based on the annotation accuracy; An apparatus for selecting entities to be labeled based on the high-quality annotation type in the large model to obtain labeled data.

[0103] 17. The knowledge graph entity alignment system based on large model annotation and active learning as described in item 16, wherein the entity features include entity category, entity relationship, node centrality, PageRank score and / or number of attributes.

[0104] 18. The knowledge graph entity alignment system based on large model annotation and active learning as described in item 17, characterized in that the system further includes: An apparatus for selecting a representative set of entities from a knowledge graph based on the entity features using heuristic rules, thereby obtaining a representative set of entities.

[0105] 19. The knowledge graph entity alignment system based on large model annotation and active learning according to any one of claims 16-18, characterized in that the evaluation metrics include accuracy, recall, false positive rate, false negative rate and / or F-score.

[0106] 20. The knowledge graph entity alignment system based on large model annotation and active learning according to any one of items 16-18, characterized in that the negative sampling strategy includes: Randomly replace one of the entities in the positive example entity pair; Replace one of the entities in the positive entity pair with its k-neighbor node; and / or By combining traditional embedded training methods, multiple negative sample samplings are added.

[0107] 21. The knowledge graph entity alignment system based on large model annotation and active learning according to any one of items 16-18, characterized in that the dynamic deletion strategy is to construct the differential entity pair by deleting and / or modifying at least one of the entity features of the entities in the positive entity pair; The difference entity pairs include information content difference entity pairs, structural difference entity pairs, attribute difference entity pairs, and / or attribute value difference entity pairs.

[0108] 22. The knowledge graph entity alignment system based on large model annotation and active learning according to any one of claims 16-18, characterized in that the lookup device is further configured to ignore certain entities in the representative entity set if it is difficult to obtain their corresponding entities.

[0109] 23. The knowledge graph entity alignment system based on large model annotation and active learning according to any one of claims 16-18, characterized in that the generation device comprises: This is used to obtain embedding vector representations of entity attributes using a pre-trained language model, and to obtain the similarity matrix of entity attributes using vector cosine. Units; This is used to obtain vector representations of entity structure information using classic knowledge graph embedding models, and to calculate the similarity matrix of entity structure information. Units; If an entity name exists, this method uses a pre-trained language model to obtain the vector representation of the entity name and calculate the similarity matrix of the entity names. Units; A unit for fine-tuning the embedded vector representation of entity attributes, the vector representation of entity structure information, and the vector representation of entity name using the sample pool; This method uses basic distance calculation to compute the similarity of each feature on node features, graph structure features, and attribute features, and then constructs a similarity matrix. Units; Used for mean matrix The KNN algorithm is used to select the top-k nearest neighbors of each entity to generate the candidate set of units.

[0110] 24. The knowledge graph entity alignment system based on large model annotation and active learning according to any one of claims 16-18, characterized in that the computing device is further configured to calculate the large model for the target entity using the following formula. Uncertainty :

[0111] in Represents the target entity The The nearest neighbor entities, where align is a preset alignment function.

[0112] 25. The knowledge graph entity alignment system based on large model annotation and active learning as described in item 24, characterized in that the preset alignment degree function align is an SVM function; The statistical model is SVM.

[0113] 26. The knowledge graph entity alignment system based on large model annotation and active learning according to any one of claims 16-18, characterized in that the system further comprises: A means for selecting the target entity from the candidate set using a greedy algorithm.

[0114] 27. The knowledge graph entity alignment system based on large model annotation and active learning as described in item 25, characterized in that the prediction device is further used to train the statistical model using machine learning, wherein... For the true labels of entity pairs, The labels assigned to the large model indicate the accuracy of the large model's labeling. for:

[0115] in for The feature vector includes a discrete feature vector and an embedded similarity feature vector. The discrete features include out-degree, in-degree, and / or PageRank. The embedded similarity features include entity attributes, entity structure, and / or entity name.

[0116] 28. The knowledge graph entity alignment system based on large model annotation and active learning according to any one of claims 16-18, wherein the entity value is the PageRank Score of the target entity.

[0117] 29. The knowledge graph entity alignment system based on large model annotation and active learning according to any one of claims 16-18, characterized in that the system further comprises: A denoising device is used to train a small entity alignment model based on the labeled data through an orphan node detection mechanism, a noise label learning algorithm, and a multi-hop graph neural network.

[0118] 30. The knowledge graph entity alignment system based on large model annotation and active learning as described in item 29, characterized in that the denoising device further includes: A unit for filtering out noise-free data from the labeled data using Dropout and cross-entropy loss; This is used to train a feedforward neural network predictor to identify isolated nodes in the noiseless data, and to use margin-based loss to maintain the relatively independent representation of the isolated nodes in the vector space, thereby obtaining units of high-quality data. A unit used to extract the structural and attribute information of entities in the high-quality data using a multi-hop graph neural network, and to obtain the embedded representation of the entities; This is used to train the entity alignment small model using a negative sampling strategy and a margin-based loss function, and to map entities from two knowledge graphs to a unified vector space based on the embedding representation to obtain units of vector representation; A unit used to calculate the similarity between entities based on the vector representation, sort them according to the similarity, and output the most matching entity pair.

Claims

1. A knowledge graph entity alignment method based on large model annotation and active learning, characterized in that, The method includes: Experts search for corresponding entities based on the entity characteristics of entities in a representative entity set to obtain positive entity pairs; Based on the positive entity pairs, negative entity pairs and differential entity pairs are obtained using a negative sampling strategy and a dynamic deletion strategy, and the positive entity pairs, the negative entity pairs, and the differential entity pairs are constructed into a sample pool; The entities and entity pairs in the sample pool are labeled using a large model to obtain the labeling results. The quality of the annotation results is evaluated using assessment metrics. Based on the sample pool and the annotation quality, a candidate set is generated using the KNN algorithm; By calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entity, the uncertainty of the target entity in the large model is obtained. Training data is selected from the target entities based on the uncertainty and entity value, and a statistical model is trained using this data to predict the labeling accuracy of the large model. Based on the annotation accuracy, the high-quality annotation types of the large model are obtained; The large model selects entities to be labeled based on the high-quality labeling type to obtain labeled data.

2. The knowledge graph entity alignment method based on large model annotation and active learning according to claim 1, characterized in that, The entity features include entity category, entity relationship, node centrality, PageRank score, and / or number of attributes.

3. The knowledge graph entity alignment method based on large model annotation and active learning according to claim 2, characterized in that, Before the step whereby the expert searches for the corresponding entity based on the entity characteristics of the entity in the representative entity set to obtain positive entity pairs, the method further includes: Based on the entity features, a representative entity set is obtained by selecting a representative entity set from the knowledge graph using heuristic rules.

4. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of claims 1-3, characterized in that, The evaluation metrics include accuracy, recall, false positive rate, false negative rate, and / or F-score.

5. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of claims 1-3, characterized in that, The negative sampling strategy includes: Randomly replace one of the entities in the positive example entity pair; Replace one of the entities in the positive entity pair with its k-neighbor node; and / or By combining traditional embedded training methods, multiple negative sample samplings are added.

6. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of claims 1-3, characterized in that, The dynamic deletion strategy is to construct the differential entity pair by deleting and / or modifying at least one of the entity features of the entities in the positive entity pair; The difference entity pairs include information content difference entity pairs, structural difference entity pairs, attribute difference entity pairs, and / or attribute value difference entity pairs.

7. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of claims 1-3, characterized in that, The steps for experts to find the corresponding entities based on the entity characteristics of entities in a representative entity set and obtain positive entity pairs also include: If it is difficult to obtain the corresponding entity for some entities in the representative entity set, then they are ignored.

8. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of claims 1-3, characterized in that, The steps for generating a candidate set using the KNN algorithm based on the sample pool and the annotation quality specifically include: The embedding vector representations of entity attributes are obtained using a pre-trained language model, and the similarity matrix of entity attributes is obtained using vector cosine. ; We use a classic knowledge graph embedding model to obtain vector representations of entity structure information and calculate the similarity matrix of entity structure information. ; If entity names exist, use a pre-trained language model to obtain the vector representation of the entity names and calculate the similarity matrix of the entity names. ; The sample pool is used to fine-tune the embedding vector representation of entity attributes, the vector representation of entity structure information, and the vector representation of entity name; The basic distance calculation method is used on node features, graph structure features, and attribute features to calculate the similarity of each feature and form a similarity matrix. ; Based on the mean matrix The KNN algorithm is used to select the top-k nearest neighbors of each entity to generate the candidate set.

9. The knowledge graph entity alignment method based on large model annotation and active learning according to any one of claims 1-3, characterized in that, The specific steps for obtaining the uncertainty of the target entity in the large model by calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entities, are as follows: The following formula is used to calculate the target entity in the large model. Uncertainty : in Represents the target entity The The nearest neighbor entities, where align is a preset alignment function.

10. A knowledge graph entity alignment system based on large model annotation and active learning, characterized in that, The system includes: The lookup device is used by experts to find the corresponding entities based on the entity characteristics of entities in a representative entity set, and to obtain positive entity pairs. A construction apparatus is used to obtain negative entity pairs and differential entity pairs based on the positive entity pairs using a negative sampling strategy and a dynamic deletion strategy, and to construct the positive entity pairs, the negative entity pairs and the differential entity pairs into a sample pool; Apparatus for labeling entities and entity pairs in the sample pool using a large model to obtain labeling results; Apparatus for evaluating the annotation quality of the annotation results using evaluation metrics; A generation device for generating a candidate set based on the sample pool and the annotation quality using the KNN algorithm; A computing device for obtaining the uncertainty of a large model for a target entity by calculating the alignment degree between the target entity and its corresponding entity in the candidate set, and the alignment degree between the target entity and its nearest neighbor entity. A prediction device for selecting training data from the target entity based on the uncertainty and entity value, and using it to train a statistical model to predict the labeling accuracy of a large model; Apparatus for obtaining high-quality annotation types for the large model based on the annotation accuracy; An apparatus for selecting entities to be labeled based on the high-quality annotation type in the large model to obtain labeled data.

Images