Graph neural network-based easily-confused entity linking method and system

By constructing the graph distillation operator of candidate entity relationship diagrams and graph neural networks, the subtle differences and semantic similarities of entity links in the prior art are solved, and the accuracy and robustness of entity links are improved, especially in the CCKS2020 Chinese short text task.

CN120493927APending Publication Date: 2025-08-15Shanxi Taihang Laboratory Co., Ltd.
View PDF 0 Cites 2 Cited by

Patent Information

Application Number
CN202510558860.5
Authority / Receiving Office
CN · China
Patent Type
Applications(China)
Current Assignee / Owner
Filing Date
2025-04-29
Publication Date
2025-08-15

AI Technical Summary

Technical Problem

When existing entity linking methods deal with candidate entities that are prone to confusing, it is difficult to capture subtle differences and distinguish semantic similar entities, resulting in limited disambiguation effects.

Method used

By constructing a relationship diagram between candidate entities, the graph distillation operator in the graph neural network is used to extract the distinguishable features described by candidate entities, and a high-dimensional semantic vector is generated by combining the pre-trained BERT model to perform feature fusion and similarity scoring to match the most suitable entity.

Benefits of technology

The accuracy and robustness of entity links have been significantly improved, especially in the case of confusion, and the disambiguation effect has been improved. The accuracy and F1 value in CCKS2020 Chinese short text entity link task have been significantly improved.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN120493927A_ABST
    Figure CN120493927A_ABST
Patent Text Reader

Abstract

The invention discloses an easily-confused entity linking method and system based on a graph neural network, and is applied to the technical field of natural language processing. The method comprises the following steps: encoding a short text and candidate entity description, generating a high-dimensional semantic vector, and providing basic features for subsequent entity disambiguation; respectively extracting features described by the short text and the candidate entities, and extracting distinguishable features in the features described by the candidate entities as candidate entity features by constructing a relation graph between the candidate entities and utilizing a graph distillation operator extracted from a graph neural network; the short text features and the candidate entity features are fused, and the most suitable entity is matched through similarity scoring. According to the method, the graph distillation operator of the graph neural network is introduced, so that the accuracy and robustness of entity linking are remarkably improved, and particularly, distinguishable features can be effectively extracted when easily confused candidate entities are processed, so that the disambiguation effect is improved.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present invention relates to the field of natural language processing technology, and more particularly to a method and system for linking easily confusing entities based on graph neural networks. Background Art

[0002] With the rapid development of deep learning technology, research on the entity linking (EL) task has undergone a significant shift from traditional feature engineering methods to automatic feature learning methods based on deep learning. In recent years, research on entity linking has focused on leveraging deep learning models to enhance semantic understanding of entities, improve the accuracy of entity disambiguation, and reduce computational complexity. In the early days of deep learning, entity linking primarily relied on traditional neural network models, such as feedforward neural networks and convolutional neural networks (CNNs). The core idea of these methods was to automatically learn the relevance features between input text and candidate entities through neural networks. Early deep learning entity linking methods typically vectorized entity descriptions and used neural networks for matching and disambiguation. However, these methods have significant limitations, particularly in handling complex contextual information and in capturing long-range dependencies, resulting in significant room for improvement in disambiguation accuracy.

[0003] With the rise of recurrent neural networks (RNNs) and long short-term memory (LSTM) networks, researchers have begun leveraging these models to better process the temporal nature of text. RNNs and LSTMs effectively capture the sequential information and contextual dependencies within text, addressing to some extent the inability of earlier neural network models to handle long-distance dependencies. However, while LSTMs excel at modeling contextual information, they remain inefficient when processing global dependencies within long texts due to their reliance on sequential information that is passed incrementally.

[0004] In recent years, entity linking methods based on deep learning have become a research hotspot, especially entity linking methods based on pre-trained language models (such as BERT) have made significant progress. However, existing methods still have certain limitations when dealing with easily confused candidate entities, specifically: 1. Coarse-grained features are difficult to capture subtle differences: Existing methods mainly rely on coarse-grained features, which make it difficult to accurately capture subtle differences between candidate entities, resulting in limited disambiguation effects. 2. Insufficient ability to distinguish semantically similar entities: When dealing with candidate entities that are semantically similar but point to different entities, the performance of existing methods still needs to be improved, especially the lack of disambiguation capabilities in complex contexts. Therefore, how to provide a method and system for easily confused entity linking based on graph neural networks is a problem that those skilled in the art urgently need to solve. Summary of the Invention

[0005] In view of this, the present invention provides a method and system for linking easily confused entities based on graph neural networks. By constructing a relationship graph between candidate entities, the features of entity descriptions are further refined, and distinguishable element words in the descriptions of candidate entities are extracted, thereby analyzing and distinguishing candidate entities at a finer-grained level, thereby improving the accuracy and robustness of entity linking.

[0006] In order to achieve the above object, the present invention provides the following technical solutions:

[0007] A method for linking easily confusing entities based on graph neural networks, comprising the following steps:

[0008] S1. Encode short text and candidate entity descriptions to generate high-dimensional semantic vectors, providing basic features for subsequent entity disambiguation;

[0009] S2. Extract features of short text and candidate entity descriptions respectively, build a relationship graph between candidate entities, and use the graph distillation operator extracted from the graph neural network to extract distinguishable features from the candidate entity description features as candidate entity features;

[0010] S3. Fuse the short text features with the candidate entity features and match the most appropriate entity through similarity scoring.

[0011] Optionally, S1 is:

[0012] S11. Concatenate the short text and the candidate entity description into a string in the format of [CLS] short text [SEP] candidate entity description [SEP];

[0013] S12. Use the pre-trained BERT model to encode the concatenated string and generate a feature vector;

[0014] S13. Extract the vector of the [CLS] tag from the output of the last layer of the BERT model as the global feature of the short text and the candidate entity description. Record the starting and ending positions of the text mention to be disambiguated in the short text. Extract the vectors of the corresponding positions and concatenate them to obtain the local feature representation.

[0015] S14, adding the global features and the local features to obtain a fused joint feature representation;

[0016] S15. Classify the fused joint feature representation through a fully connected layer with Sigmoid as the activation function to obtain the matching score between the text mention and the corresponding candidate entity in the knowledge base, and locate the easily confused clustering graph of the candidate entity.

[0017] Optionally, S2 is:

[0018] S21. Build a fully connected graph G for all candidate entities L *, where each pair of candidate entity descriptions L i ,L j The edge weights between ∈L are defined as their TF-IDF, which represents the TF-IDF i and TF-IDF j The cosine similarity between

[0019] S22, from the fully connected graph G * Delete the edges whose weight is less than the predefined threshold and get a new candidate entity graph There are several disconnected subgraphs g1,...,g M Composition, where each subgraph g i They all contain at least one description of a candidate entity;

[0020] S23, according to the graph distillation operator improved by the graph convolution operator, from each subgraph g i Extract distinguishable features.

[0021] Optionally, S23 specifically includes:

[0022] For the candidate entity description L i , the graph distillation operator uses a trainable weight matrix to capture the similarity information between it and its neighbors in the candidate entity graph G, and uses the matrix to extract the candidate entity description L i The effective semantic features of L, at each layer l≥0, the candidate entity description L i The aggregation of similar information between its neighbors is removed from its representation, i.e.:

[0023]

[0024] Where, Describe the candidate entity L in the lth graph distillation layer i The representation of N i is the neighbor set in the candidate entity graph G, b (l) is the deviation, Φ (l) ,Ψ (l) is a trainable self-weighting matrix;

[0025] Use a graph distillation operator with H layers to output the candidate entity description L at the last layer i Representation Contains rich distinguishable features, describing the candidate entity L i With the same subgraph g i to distinguish it from other candidate entities within the entity.

[0026] Optional, S3 specifically:

[0027] S31. Extract subgraph g from the embedded node feature matrixi The vectors of all candidate entities in , denoted as a∈[1,p], a single candidate entity vector is recorded as

[0028] S32, the entity reference item after fusion of all short text features is recorded as B = {t1, t2, ..., t q}, b∈[1,q], a single entity reference is denoted as t b ;

[0029] S33, will With t b Splicing to get:

[0030]

[0031] S34. Use the Sigmoid activation function through the fully connected layer to score the similarity of all candidate entities. The candidate entity with the highest score is the target entity.

[0032] A confusing entity linking system based on a graph neural network executes any of the above-mentioned confusing entity linking methods based on a graph neural network, including a basic coding layer, a distinguishable feature coding layer and a fully connected layer. The basic coding layer encodes the concatenated character strings based on a pre-trained BERT model to generate a high-dimensional semantic vector. The distinguishable feature coding layer extracts the features of short text and candidate entity descriptions respectively, and uses a graph distillation operator to extract distinguishable features from the candidate entity description features as candidate entity features. The fully connected layer fuses the short text features with the candidate entity features, and matches the most suitable entity through similarity scoring.

[0033] It can be seen from the above technical solution that compared with the existing technology, the present invention provides a method and system for easily confusing entity linking based on graph neural network, which has the following beneficial effects: The present invention significantly improves the accuracy and robustness of entity linking by introducing the graph distillation operator of graph neural network, especially when dealing with easily confusing candidate entities, it can effectively extract distinguishable features, thereby improving the disambiguation effect; in the CCKS2020 Chinese short text entity linking task, compared with the traditional BERT model, both the accuracy and F1 value are significantly improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0034] In order to more clearly illustrate the embodiments of the present invention or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments or the description of the prior art. Obviously, the drawings described below are merely embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on the provided drawings without paying any creative work.

[0035] Figure 1 Flowchart of the confusing entity linking method of the present invention;

[0036] Figure 2 Schematic diagram of the structure of the easily confusing entity linking system of the present invention. DETAILED DESCRIPTION

[0037] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0038] The embodiment of the present invention discloses a method for linking easily confused entities based on graph neural network, such as Figure 1 As shown, the following steps are included:

[0039] S1. Encode short text and candidate entity descriptions to generate high-dimensional semantic vectors, providing basic features for subsequent entity disambiguation;

[0040] S2. Extract features of short text and candidate entity descriptions respectively, build a relationship graph between candidate entities, and use the graph distillation operator extracted from the graph neural network to extract distinguishable features from the candidate entity description features as candidate entity features;

[0041] S3. Fuse the short text features with the candidate entity features and match the most appropriate entity through similarity scoring.

[0042] Furthermore, S1 is specifically:

[0043] S11. Concatenate the short text and the candidate entity description into a string in the format of [CLS] short text [SEP] candidate entity description [SEP]. [CLS] and [SEP] are special markers in the BERT model, indicating the start and separation of sentences, respectively.

[0044] S12. Use the pre-trained BERT model to encode the concatenated string and generate a feature vector. The BERT model captures the contextual information of the text through a bidirectional Transformer structure and outputs an embedded representation of each character.

[0045] S13. Extract the [CLS] token vector BERT(text) from the output of the last layer of the BERT model as the global feature of the short text and candidate entity description. This vector captures the global semantic information of the entire input sequence (including the short text and candidate entity description), records the start and end positions of the text mention to be disambiguated (i.e., the entity referent that needs to be linked to the knowledge base, such as ambiguous names or phrases in a sentence) in the short text, extracts the vectors of the corresponding positions and concatenates them to obtain the local feature representation Entity(begin,end);

[0046] text=[CLS]t1,t2,...,t n [SEP]d1,d2,...,d m [SEP]

[0047] Where t1, t2, ..., t n The short text context of the text mention to be disambiguated, d1,d2,...,d m is the description text of the corresponding candidate entity;

[0048] S14. Add the global features and local features to obtain the fused joint feature representation:

[0049] S link =BERT(text)+Entity(begin,end)

[0050] The fused joint feature representation S link Used to calculate the matching score between text mentions and candidate entities;

[0051] S15. Classify the fused joint feature representation through a fully connected layer with Sigmoid as the activation function to obtain the matching score between the text mention and the corresponding candidate entity in the knowledge base, and locate the easily confused clustering graph of the candidate entity.

[0052] Although the candidate entity disambiguation method using the base-layer BERT encoding method can disambiguate entities using coarse-grained correlation features, this method is primarily based on empirical results obtained through extensive data training. In some special cases, coarse-grained empirical information cannot express the subtle differences between two candidate entities. Therefore, the present invention employs a graph-based approach to identify subtle differences between the feature words in the descriptive text of the candidate entities.

[0053] Furthermore, S2 is specifically:

[0054] S21. For each text mention to be disambiguated, there may be multiple candidate entity descriptions. In order to find easily confused candidate entity descriptions, a fully connected graph G is constructed for all candidate entities L. *, where each pair of candidate entity descriptions L i ,L j The edge weights between ∈L are defined as their TF-IDF, which represents the TF-IDF i and TF-IDF j The cosine similarity between

[0055] S22, from the fully connected graph G * Delete the edges whose weight is less than the predefined threshold and get a new candidate entity graph There are several disconnected subgraphs g1,...,g M Composition, where each subgraph g i They all contain at least one description of a candidate entity;

[0056] In the embodiment of the present invention, the threshold is set to 0.2;

[0057] S23, according to the graph distillation operator improved by the graph convolution operator, from each subgraph g i Extract distinguishable features.

[0058] In the embodiment of the present invention, the weight and bias matrix trained by the basic coding layer BERT model are transferred to the new BERT model for initialization, and the entity reference item after the fusion of all candidate entity description features is obtained and recorded as A={t1,t2,...,t p}, a∈[1,p], where the candidate entity description L i The word feature vector in is t i .

[0059] Furthermore, S23 is specifically:

[0060] For the candidate entity description L i , the graph distillation operator uses a trainable weight matrix to capture the similarity information between it and its neighbors in the candidate entity graph G, and uses the matrix to extract the candidate entity description L i The effective semantic features of L, at each layer l≥0, the candidate entity description L i The aggregation of similar information between its neighbors is removed from its representation, i.e.:

[0061]

[0062] Where, Describe the candidate entity L in the lth graph distillation layer i The representation of N i is the neighbor set in the candidate entity graph G, b (l) is the deviation, Φ (l) ,Ψ (l) is a trainable self-weighting matrix;

[0063] Use a graph distillation operator with H layers to output the candidate entity description L at the last layer i Representation Contains rich distinguishable features, describing the candidate entity L i With the same subgraph g i to distinguish it from other candidate entities within the entity.

[0064] Furthermore, S3 is specifically:

[0065] S31. Extract subgraph g from the embedded node feature matrix i The vectors of all candidate entities in , denoted as a∈[1,p], a single candidate entity vector is recorded as

[0066] S32, the entity reference item after fusion of all short text features is recorded as B = {t1, t2, ..., t q}, b∈[1,q], a single entity reference is denoted as t b ;

[0067] S33, will With t b Splicing to get:

[0068]

[0069] S34. Use the Sigmoid activation function through the fully connected layer to score the similarity of all candidate entities. The candidate entity with the highest score is the target entity.

[0070] Furthermore, in one embodiment of the present invention, an entity linking experiment is conducted using a graph neural network-based easily confused entity linking method. The data set uses the data set of the CCKS2020 Chinese short text entity linking task. The average length of the short text in the data set is 21.73 Chinese characters, which mainly comes from microblogs, entries, encyclopedias, news videos and article titles on the Internet, as well as user conversation content, etc. The corpus consists of a training set and a verification set, wherein the training set includes 70,000 short text annotation data, and the test set includes 10,000 short text annotation data, and the training data set is divided into a training set and a verification set according to a 7-fold cross-validation. The knowledge base is semi-structured data, and each data has both triples and structured long texts, wherein the triples contain entities and relationships. Secondly, in order to prevent erroneous characters from affecting the downstream disambiguation results, the entities in the same field in the text in the data set and the knowledge base are unified in this embodiment to prevent problems that cannot be matched due to punctuation, capitalization, typos, etc. This division method ensures that the model can obtain sufficient and balanced data support in each stage of training, verification and testing, thereby improving the credibility of the experimental results.

[0071] In order to ensure the scientificity and repeatability of the experiment, model training was carried out in a unified hardware and software environment. In this embodiment, the basic coding layer uses the official Chinese BERT model as the encoder, and uses 8 NVIDIAA10040GB GPUs for accelerated calculation. Short texts and candidate entity descriptions are processed as character sequences, and the maximum input sequence length is set to 512. The Adam optimizer is used in the experiment, the learning rate is set to 2×10-5, the number of training rounds is set to 3, and the batch size is set to 32. The BERT model is fine-tuned based on the characteristics of the short text entity linking dataset to improve the experimental effect. The learning rate of the distinguishable feature encoding layer graph neural network (GNN) is set to 10-4, and the similarity threshold for removing similar features is 0.35.

[0072] Among them, BERT only allows a maximum of 512 symbols to be input, so the entity description text that is too long is truncated, that is, only 512 words are retained, and the excess is directly discarded. Since the default truncation operation will cause the loss of some entity information, this article assumes that the length of each triple text is proportional to the information it contains, and sorts each triple text in descending order according to length. Descending sorting can ensure as much as possible that the words discarded due to truncation belong to triples containing less information (shorter triple texts), ensuring that the truncated entity description text contains as much information as possible. Secondly, when the input text length is less than the maximum length, the input length can be increased to the maximum length by padding.

[0073] During the training process, a 7-fold cross-validation was used to divide the training and validation sets, and the models were saved according to the loss and F1 score, respectively. During the inference phase, the short text and candidate entity descriptions were input into the trained model, and the similarity scores were calculated through the fully connected layer. The candidate entity with the highest score was selected as the matching result.

[0074] In the CCKS2020 Chinese short text entity linking task, the present invention has significantly improved the F1 value compared with the traditional BERT model, verifying its application potential in the field of entity linking.

[0075] Technical terms in this embodiment:

[0076] BERT: Bidirectional Encoder Representations from Transformers, a pre-trained language model based on the Transformer architecture, widely used in natural language processing tasks;

[0077] Transformer: A deep learning model based on the self-attention mechanism that can process sequential data in parallel and capture long-distance dependencies. It is widely used in natural language processing tasks.

[0078] TF-IDF: A statistical method used to measure the importance of words in a document. It evaluates the discriminative power of words by calculating the inverse of the frequency of a word in a document and its frequency in the entire corpus.

[0079] GNN: Graph Neural Network, graph neural network, a deep learning model for processing graph structured data.

[0080] and Figure 1 Corresponding to the method, the embodiment of the present invention further discloses a confusing entity linking system based on a graph neural network, which executes any of the above-mentioned confusing entity linking methods based on a graph neural network, such as Figure 2 As shown in the figure, it includes a basic coding layer, a distinguishable feature coding layer and a fully connected layer. The basic coding layer encodes the concatenated character strings based on the pre-trained BERT model to generate a high-dimensional semantic vector. The distinguishable feature coding layer extracts the features of short text and candidate entity descriptions respectively, and uses the graph distillation operator to extract the distinguishable features in the candidate entity description features as candidate entity features. The fully connected layer fuses the short text features with the candidate entity features and matches the most suitable entity through similarity scoring.

[0081] The various embodiments in this specification are described in a progressive manner, with each embodiment focusing on the differences from other embodiments. Reference can be made to the common and similar parts between the various embodiments. For the systems disclosed in the embodiments, since they correspond to the methods disclosed in the embodiments, the description is relatively simple, and the relevant parts can be referred to the method description.

[0082] The above description of the disclosed embodiments is intended to enable one skilled in the art to implement or use the present invention. Various modifications to these embodiments will be readily apparent to one skilled in the art, and the general principles defined herein may be implemented in other embodiments without departing from the spirit or scope of the present invention. Therefore, the present invention is not limited to the embodiments shown herein but is intended to conform to the widest scope consistent with the principles and novel features disclosed herein.

Claims

1. A confusing entity linking method based on graph neural network, characterized in that: The following steps are involved: S1. Encode short text and candidate entity descriptions to generate high-dimensional semantic vectors, providing basic features for subsequent entity disambiguation; S2. Extract features of short text and candidate entity descriptions respectively, build a relationship graph between candidate entities, and use the graph distillation operator extracted from the graph neural network to extract distinguishable features from the candidate entity description features as candidate entity features; S3. Fuse the short text features with the candidate entity features and match the most appropriate entity through similarity scoring.

2. The method for linking easily confusing entities based on graph neural networks according to claim 1, characterized in that: S1 is specifically: S11. Concatenate the short text and the candidate entity description into a string in the format of [CLS] short text [SEP] candidate entity description [SEP]; S12. Use the pre-trained BERT model to encode the concatenated string and generate a feature vector; S13. Extract the vector of the [CLS] tag from the output of the last layer of the BERT model as the global feature of the short text and the candidate entity description. Record the starting and ending positions of the text mention to be disambiguated in the short text. Extract the vectors of the corresponding positions and concatenate them to obtain the local feature representation. S14, adding the global features and the local features to obtain a fused joint feature representation; S15. Classify the fused joint feature representation through a fully connected layer with Sigmoid as the activation function to obtain the matching score between the text mention and the corresponding candidate entity in the knowledge base, and locate the easily confused clustering graph of the candidate entity.

3. The method for linking easily confusing entities based on graph neural networks according to claim 1, characterized in that: S2 is specifically: S21. Build a fully connected graph G for all candidate entities L * , where each pair of candidate entity descriptions L i ,L j The edge weights between ∈L are defined as their TF-IDF, which represents the TF-IDF i and TF-IDF j The cosine similarity between S22, from the fully connected graph G * Delete the edges whose weight is less than the predefined threshold and get a new candidate entity graph There are several disconnected subgraphs g1,...,g M Composition, where each subgraph g i They all contain at least one description of a candidate entity; S23, according to the graph distillation operator improved by the graph convolution operator, from each subgraph g i Extract distinguishable features.

4. The method for linking easily confusing entities based on graph neural networks according to claim 3, characterized in that: S23 is specifically: For the candidate entity description L i , the graph distillation operator uses a trainable weight matrix to capture the similarity information between it and its neighbors in the candidate entity graph G, and uses the matrix to extract the candidate entity description L i The effective semantic features of L, at each layer l≥0, the candidate entity description L i The aggregation of similar information between its neighbors is removed from its representation, i.e.: Where, Describe the candidate entity L in the lth graph distillation layer i The representation of N i is the neighbor set in the candidate entity graph G, b (l) is the deviation, Φ (l) ,Ψ (l) is a trainable self-weighting matrix; Use a graph distillation operator with H layers to output the candidate entity description L at the last layer i Representation Contains rich distinguishable features, describing the candidate entity L i With the same subgraph g i to distinguish it from other candidate entities within the entity.

5. The method for linking easily confusing entities based on graph neural networks according to claim 4, characterized in that: S3 specifically: S31. Extract subgraph g from the embedded node feature matrix i The vectors of all candidate entities in , denoted as a∈[1,p], a single candidate entity vector is recorded as S32, the entity reference item after fusion of all short text features is recorded as B = {t1, t2, ..., t q }, b∈[1,q], a single entity reference is denoted as t b ; S33, will With t b Splicing to get: S34. Use the Sigmoid activation function through the fully connected layer to score the similarity of all candidate entities. The candidate entity with the highest score is the target entity.

6. A confusing entity linking system based on a graph neural network, executing a confusing entity linking method based on a graph neural network according to any one of claims 1 to 5, characterized in that: It includes a basic coding layer, a distinguishable feature coding layer and a fully connected layer. The basic coding layer encodes the concatenated character strings based on the pre-trained BERT model to generate a high-dimensional semantic vector. The distinguishable feature coding layer extracts the features of short text and candidate entity descriptions respectively, and uses the graph distillation operator to extract the distinguishable features in the candidate entity description features as candidate entity features. The fully connected layer fuses the short text features with the candidate entity features and matches the most suitable entity through similarity scoring.

Citation Information

Cited By

  • English knowledge confusion entity pair recognition and binding recommendation method based on answer modification behavior sequence

    CN122133657A

  • An english knowledge confusion entity pair identification and bundling recommendation method based on answer modification behavior sequence

    CN122133657B