Knowledge graph entity linking method and system for eliminating popularity deviation based on causal inference and medium

Abstract

The invention discloses a knowledge graph entity linking method and system for eliminating popularity deviation based on causal inference and a medium, an entity candidate set generation stage: processing an input text to generate a candidate entity set, and extracting semantic features, structural features and popularity features of each candidate entity; the problem that unpopular entities are neglected is effectively solved, and the robustness under complex contexts and noise data is remarkably enhanced; the effectiveness of the method in balancing popularity and semantic correlation is verified; therefore, the model can more comprehensively sense the context relationship of the entity in the knowledge graph, and the accuracy and robustness of entity linking are further improved. Ablation experiments show that introduction of the structural features contributes significantly to performance improvement of the long-tail entity.

Description

TECHNICAL FIELD

[0001] The present application relates to the field of natural language processing, knowledge graph and artificial intelligence, and in particular to a knowledge graph entity linking method based on causal inference to eliminate popularity bias, a system and a medium. BACKGROUND

[0002] With the development of the Internet and big data, knowledge graph has become an important supporting technology for core tasks such as text intelligent understanding and information retrieval. Entity linking (EL) is a key link in knowledge service applications, and its accuracy directly affects the effect of downstream tasks. Traditional entity linking methods mainly rely on semantic matching of text context and knowledge graph entities, and introduce the popularity characteristics (such as Wikipedia citation times, external link numbers, etc.) of entities to improve the linking accuracy.

[0003] However, the distribution of entity popularity is significantly unbalanced: popular entities are easily selected by the model due to high-frequency exposure, while long-tail entities are often ignored even if the text context has a clear direction. For example, when facing "What are Michael Jordan's academic achievements?", the model tends to link to the basketball "Michael Jordan" rather than the scholar in the field of machine learning. This "popularity bias" leads to a decrease in the accuracy of the recommendation system in long-tail entity identification, limiting the breadth and depth of knowledge graph applications.

[0004] In recent years, causal inference methods have shown great potential in eliminating confounding factors and revealing the true causal relationship between variables, and have been tried to correct bias in recommendation, advertising and other fields, but there are few systematic applications in knowledge graph entity linking tasks. How to ensure overall accuracy while stripping the confounding influence of popularity and improving the recall and identification ability of long-tail entities has become a core problem that needs to be solved in this field. SUMMARY

[0005] In order to overcome the shortcomings of the prior art, one of the purposes of the present application is to provide a knowledge graph entity linking method based on causal inference to eliminate popularity bias, a system and a medium.

[0006] One of the purposes of the present application is achieved by adopting the following technical solutions:

[0007] A knowledge graph entity linking method based on causal inference to eliminate popularity bias, comprising the following steps:

[0008] S1: Entity candidate set generation stage: processing the input text, generating a candidate entity set, and extracting the semantic features h T , structural features h g,e and popularity features Z of each candidate entity;

[0009] S2: Causal inference debiasing stage: Take "entity is linked" as the dependent variable Y, the context-entity semantic matching score T as the treatment variable, and the entity popularity Z as the confounding variable. Through causal inference modeling, the confounding effect of popularity is removed to obtain the causal association score P(Y=1|do(Z),T)=f(T,Z)-α·g(Z);

[0010] S3: Contrastive learning discrimination stage: Based on the entity features after causal correction, construct positive and negative sample pairs, and use the contrast loss function Train the model.

[0011] Further: the S1 entity candidate set generation stage includes:

[0012] Encode entity description h using a pre-trained language model e =Encoder LM (Prompt(e))

[0013] Aggregating neighbor information through graph neural networks:

[0014]

[0015] Furthermore: in S2, f(T, Z) is the semantic matching sub-model, g(Z) is the popularity influence function, and α is optimized by grid search;

[0016] The loss function is defined as: Where l is the cross entropy loss function.

[0017] Further:

[0018] Construct the first type of positive and negative sample pairs: the semantic embedding of the mention and the structurally enhanced embedding of the correct entity are used as positive samples, and the embedding of the incorrect entity is used as negative samples, and optimize the loss function:

[0019]

[0020] Construct the second type of positive and negative sample pairs: use the structurally enhanced embedding and semantic embedding of the same entity as positive samples, and the embeddings of different entities as negative samples, and optimize the loss function:

[0021]

[0022] The joint optimization goal is

[0023] Furthermore: the graph neural network is GCN or GraphSAGE, and the aggregation function AGGREGATE adopts mean aggregation, pooling aggregation or attention aggregation.

[0024] Furthermore: the popularity feature Z includes the number of Wikipedia links, the number of knowledge graph citations or the historical frequency of occurrence.

[0025] Furthermore: the similarity function sim is cosine similarity or Euclidean distance.

[0026] Furthermore, the pre-trained language model is BERT, ERNIE or BGE-M3.

[0027] A knowledge graph entity linking system based on causal inference to eliminate popularity bias, including:

[0028] Candidate generation module: used to generate candidate entity sets and extract features;

[0029] Causal debiasing module: used to perform causal inference modeling to eliminate popularity bias;

[0030] Contrastive discrimination module: used to implement entity linking decisions based on contrastive learning.

[0031] A computer-readable storage medium having a computer program stored thereon, wherein when the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

[0032] Compared with the prior art, the present invention has the following beneficial effects:

[0033] 1. Effectively improve the ability to identify long-tail entities: By removing the confounding factors of popularity through a causal inference model, the model focuses more on semantic relevance, effectively solving the problem of unpopular entities being masked by popularity bias. On the AIDA dataset, the F1 score of long-tail entities increased by 7.15 percentage points compared to the baseline method GENRE (82.32% → 89.47%); on the WIKI dataset, it increased by 7.45 percentage points (72.58% → 80.03%).

[0034] 2. Enhanced model feature representation and robustness: A contrastive learning mechanism for two types of positive and negative sample pairs was introduced, and the model's discriminative ability in complex contexts and noisy data was enhanced by aligning semantic embeddings with structural embeddings. Compared with the model using only causal inference, the F1 value of AIDA long-tail entities increased by 2.28 percentage points (87.19% → 89.47%) after the introduction of contrastive learning. On the AQUAINT dataset, the F1 value of the proposed method increased by 1.33 percentage points compared to SPEL (87.44% → 88.77%).

[0035] 3. Achieving a balance between performance for popular and long-tail entities: While maintaining performance for popular entities, the recognition capability for long-tail entities is improved. On the AIDA dataset, the F1 score for popular entities reached 94.02%, an improvement of 0.84 percentage points over the baseline ELQ method, and the overall F1 score of 93.41% surpassed all compared methods. On the WIKI dataset, the overall F1 score reached 83.12%, further demonstrating generalization capabilities.

[0036] The above description is only an overview of the technical solution of the present invention. In order to more clearly understand the technical means of the present invention, it can be implemented in accordance with the contents of the specification. In order to make the above and other purposes, features and advantages of the present invention more obvious and easy to understand, the following preferred embodiments are specifically cited and described in detail with reference to the accompanying drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 This is a flowchart of the causal inference entity linking disambiguation framework structure of the present invention. DETAILED DESCRIPTION

[0038] The present invention will be further described below in conjunction with the accompanying drawings and specific implementation methods. It should be noted that, under the premise of no conflict, the various embodiments or technical features described below can be arbitrarily combined to form new embodiments.

[0039] It should be noted that when a component is referred to as being "fixed to" another component, it may be directly on the other component or there may also be a central component. When a component is considered to be "connected to" another component, it may be directly connected to the other component or there may also be a central component. When a component is considered to be "set on" another component, it may be directly set on the other component or there may also be a central component. The terms "vertical", "horizontal", "left", "right" and similar expressions used herein are for illustrative purposes only.

[0040] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this invention pertains. The terms used in this specification of the present invention are for the purpose of describing specific embodiments only and are not intended to limit the present invention. The term "and / or" as used herein includes any and all combinations of one or more of the associated listed items.

[0041] Example: Entity Linking Method Process Based on Causal Inference

[0042] 1. Entity candidate set generation

[0043] (1) Data preprocessing

[0044] We constructed a knowledge graph using Chinese Wikipedia (note: we avoided using Freebase, which contains many redundant triples, to reduce data processing costs). The graph covers over 1 million entities. We segmented the input text "Michael Jordan's academic contributions" and identified mentions of "Michael Jordan."

[0045] (2) Semantic Embedding Generation

[0046] Use ERNIE-3.0 to encode the mention context and get h T =Encoder LM (Context(T)), where T is the mention and its context information, and the dimension is set to 1024 (to enhance semantic discrimination).

[0047] Encode the description text of the candidate entities "Michael Jordan (basketball player)" and "Michael Jordan (computer scientist)" to get h e .

[0048] (3) Graph structure feature extraction

[0049] With the candidate entity as the center, a 2-hop neighbor subgraph (containing about 50 related entities) is extracted. The GraphSAGE model is used for structural aggregation (instead of GCN to reduce computational complexity), with the number of layers L = 2 and the initial feature z e =Encoder LM (KG-DESC(e)).

[0050] Generate structural enhancement features through message passing:

[0051]

[0052] The aggregation function uses attention aggregation (instead of mean aggregation to highlight the influence of key neighbors), and the final output is

[0053] 2. Debiasing Causal Inference

[0054] (1) Feature extraction and problem definition

[0055] The semantic matching score T (cosine similarity) between the mention and the entity is calculated, and the entity popularity feature Z is not explicitly modeled. Due to the text encoding stage (for example, the bias in the BGE-M3 training data can easily cause the model to over-focus on popular entities such as "basketball player Jordan") and the graph GNN stage (entities with more neighboring edges tend to receive stronger attention), the model already contains interference factors related to popularity, which need to be eliminated through subsequent steps.

[0056] (2) Causal Modeling

[0057] The semantic matching sub-model f(T, Z) is constructed by point multiplication of graph encoding and text encoding, and the influence of popularity is stripped by causal graph modeling. The hyperparameter a is set by grid search optimization, and the loss function is defined as: where l is the cross-entropy loss function, g(Z i ) is the popularity influence function, and the confounding influence of popularity Z on semantic matching score T is stripped by causal inference, where h′ T is the text embedding of the corresponding mention without context T.

[0058] 3. Contrastive learning discrimination

[0059] After eliminating the influence of popularity to obtain causal correction features, the application uses a contrastive learning mechanism to further improve the discrimination ability of entity discrimination.

[0060] Positive and negative sample construction

[0061] In the contrastive learning phase of the application, the model makes full use of the entity embedding after the fusion of multiple knowledge graph information, and designs two types of positive and negative sample pairs to improve the entity discrimination ability and representation consistency.

[0062] First, for each mention to be linked, the model encodes its context into a semantic representation and pairs it with the structure-enhanced embedding of each candidate entity. For the correct entity, this constitutes a positive sample pair, while pairing with the wrong entity constitutes a negative sample pair. In this way, the model is trained to automatically reduce the distance between the mention and its correct entity, while increasing the distance from the wrong entity, thereby improving the accuracy of overall entity disambiguation. That is:

[0063]

[0064] Second, to further enhance the fusion effect of knowledge graph structure information and semantic information, the application also pairs the "structure-enhanced embedding" of the same entity with the "structure-aware semantic embedding" to form the second type of positive and negative sample pair. Here, the positive sample pair is composed of two different types of embeddings of the same entity, while the negative sample pair is composed of embeddings of different entities. Through this contrastive learning, the model can promote the alignment of structure information and semantic information in the embedding space, making the fused entity representation more discriminative and consistent.

[0065]

[0066] In summary, the final joint optimization objective we obtain is

[0067]

[0068] In this phase, the present invention employs a contrastive learning approach, taking causally corrected entity features as input to construct positive and negative sample pairs. By maximizing the representational similarity of positive samples and minimizing the similarity of negative samples, the training model automatically learns a highly discriminative representation space, enabling more accurate entity linking decisions. This contrastive learning mechanism fully leverages the high-quality features after causal debiasing, further improving the system's generalization capabilities for long-tail entities and complex scenarios.

[0069] Experimental process and data

[0070] 1. Experimental Setup

[0071] 1. Dataset

[0072] AIDA: An authoritative entity linking dataset with complex context and noisy data.

[0073] ISNBC: Focuses on entity links in the news field, covering entities in multiple fields.

[0074] AQUAINT: Contains a lot of long text and professional terms, and is relatively difficult.

[0075] WIKI: Based on Wikipedia, the entity popularity distribution is uneven. 2. Entity segmentation: Based on the frequency of occurrence, long-tail entities (frequency < 100) and popular entities (frequency ≥ 1000) are divided.

[0076] 3. Comparison method

[0077] GENRE: An autoregressive entity retrieval model.

[0078] BLINK: A zero-shot entity linking method.

[0079] ELQ: An Efficient End-to-End Entity Linking Model.

[0080] MMEL: Transformer-based entity linking model.

[0081] SPEL: Structured Predictive Entity Linking Model.

[0082] 4. Evaluation indicators: F1-Score (the harmonic mean of precision and recall) is used.

[0083] 2. Experimental Results

[0084] 1. Overall performance comparison

[0085]

[0086] 2. Performance comparison between long-tail entities and popular entities

[0087] method Dataset Popular Entity F1% Long-tail entity F1% GENRE AIDA 92.15 82.32 BLINK AIDA 92.83 83.75 ELQ AIDA 93.18 84.63 Method of the present invention AIDA 94.02 89.47 GENRE WIKI 88.42 72.58 BLINK WIKI 89.01 73.83 Method of the present invention WIKI 89.48 80.03

[0088] (3) Ablation experiment (AIDA dataset, correction module missing effect)

[0089]

[0090] The above embodiments are only preferred embodiments of the present invention and cannot be used to limit the scope of protection of the present invention. Any non-substantial changes and replacements made by technicians in this field on the basis of the present invention fall within the scope of protection required by the present invention.

Claims

1. A knowledge graph entity linking method based on causal inference to eliminate popularity bias, characterized by: The following steps are involved: S1: Entity candidate set generation stage: Process the input text, generate candidate entity sets, and extract the semantic features h of each candidate entity T , structural features h g,e and popularity feature Z; S2: Causal inference debiasing stage: Take "entity is linked" as the dependent variable T, the context-entity semantic matching score T as the treatment variable, and the entity popularity Z as the confounding variable. Through causal inference modeling, the confounding effect of popularity is removed to obtain the causal association score P(Y=1|do(Z),T)=f(T,Z)-α·g(Z); S3: Contrastive learning discrimination stage: Based on the entity features after causal correction, construct positive and negative sample pairs, and use the contrast loss function Train the model.

2. The knowledge graph entity linking method for eliminating popularity bias based on causal inference according to claim 1 is characterized by: The S1 entity candidate set generation stage includes: Encode entity description h using a pre-trained language model e =Encoder LM (Prompt(e)) Aggregating neighbor information through graph neural networks:

3. The knowledge graph entity linking method for eliminating popularity bias based on causal inference according to claim 1 is characterized by: In S2, f(T, Z) is the semantic matching sub-model, g(Z) is the popularity influence function, and α is optimized by grid search; The loss function is defined as: Where l is the cross entropy loss function.

4. The knowledge graph entity linking method for eliminating popularity bias based on causal inference according to claim 1 is characterized by: Construct the first type of positive and negative sample pairs: the semantic embedding of the mention and the structurally enhanced embedding of the correct entity are used as positive samples, and the embedding of the incorrect entity is used as negative samples, and optimize the loss function: Construct the second type of positive and negative sample pairs: use the structurally enhanced embedding and semantic embedding of the same entity as positive samples, and the embeddings of different entities as negative samples, and optimize the loss function: The joint optimization objective is L total =L 因果 +λ1L1+λ2L2.

5. The knowledge graph entity linking method for eliminating popularity bias based on causal inference according to claim 1 is characterized by: The graph neural network is GCN or GraphSAGE, and the aggregation function AGGREGATE adopts mean aggregation, pooling aggregation or attention aggregation.

6. The knowledge graph entity linking method for eliminating popularity bias based on causal inference according to claim 1 is characterized by: The popularity feature Z includes the number of Wikipedia links, the number of knowledge graph citations, or the historical frequency of occurrence.

7. The knowledge graph entity linking method for eliminating popularity bias based on causal inference according to claim 1 is characterized by: The similarity function sim is cosine similarity or Euclidean distance.

8. The method according to any one of claims 1 to 7, characterized in that: The pre-trained language model is BERT, ERNIE or BGE-M3.

9. A knowledge graph entity linking system based on causal inference to eliminate popularity bias, characterized by: include: Candidate generation module: used to generate candidate entity sets and extract features; Causal debiasing module: used to perform causal inference modeling to eliminate popularity bias; Contrastive discrimination module: used to implement entity linking decisions based on contrastive learning.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the method according to any one of claims 1 to 8 is implemented.

Images