An entity linking method and system based on large language model

Abstract

The present application discloses an entity linking method and system based on a large language model. The method includes: using a large language model to enhance the context of a given entity reference term in an entity document, generating a candidate entity list for the entity reference term, and generating description information for each candidate entity in the candidate entity list; constructing a question-answering task of a multiple-choice question for each entity reference term; constructing a reference map using the entity reference term and the candidate entity list to obtain the correlation between the entity reference terms; sorting all entity reference terms, selecting a preset number of question-answer pairs of entity reference terms with the highest correlation with the current entity reference term based on the sorting results as the dialogue context, inputting the multiple-choice questions into the large language model in turn for answering, and obtaining the entity linking results for each entity reference term. The method of the present invention can effectively capture the potential connections between entity reference terms and performs well in document-level entity linking tasks.

Description

Technical Field

[0001] The present invention relates to an entity linking method and system, and in particular to an entity linking method and system based on a large language model. Background Art

[0002] Entity linking refers to correctly linking the entity references given in a document to the entities in the knowledge base. It is a basic task in natural language processing and plays a key role in information extraction, knowledge graph construction, problem system construction and other fields.

[0003] Existing entity linking technologies fall into two categories. The first category is rule-based methods, which rely on manual rules, dictionary matching, or probabilistic models to match entity referents. While these methods perform well in specific domains, they are expensive to maintain and struggle to handle complex semantics. The second category is deep learning-based methods, which use neural networks to automatically learn representations of entities and context, enabling end-to-end entity linking. These methods can automatically learn representations and handle complex semantics, but require large amounts of domain-specific annotated data and suffer from poor generalization capabilities.

[0004] With the rapid development of large language models (LLMs), leveraging their powerful language understanding and zero-shot learning capabilities, the entity linking task is gradually shifting towards a new paradigm centered around large language models. This paradigm has demonstrated excellent generalization performance across a wide range of application scenarios and has garnered widespread attention. However, current mainstream methods often independently process each entity reference in a document, neglecting the underlying semantic connections between entity mentions within the document, resulting in poor performance in document-level entity linking tasks. To address this issue, some studies have attempted to model entity linking as a comprehensive question-answering task, integrating all entity references and their candidate entity information within the same document to achieve unified disambiguation. However, these approaches still have significant gaps in practical effectiveness and have yet to demonstrate sufficient performance advantages. Therefore, fully leveraging large language models for joint document-level disambiguation remains a key challenge in current entity linking research. Summary of the Invention

[0005] In order to solve the above problems, this paper proposes an entity linking method and system based on large language model multi-round dialogue technology.

[0006] In a first aspect, an embodiment of the present application provides an entity linking method based on a large language model, the method comprising:

[0007] Using a large language model to enhance the context of the entity reference term given in the entity document, generate a candidate entity list for the entity reference term, and generate description information for each candidate entity in the candidate entity list;

[0008] Using the enhanced context, candidate entity list, and candidate entity description information, a multiple-choice question answering task is constructed for each entity referent.

[0009] A reference graph is constructed using the entity referents in the document and the candidate entity list generated for the entity referents. The association between the entity referents is obtained through a random walk algorithm with restarts.

[0010] All entity referents are sorted. Based on the sorting results, the multiple-choice questions corresponding to the entity referents are input into the large language model for answering. When inputting, the question-answer pairs of entity referents whose correlation with the current entity referent exceeds the threshold are selected as the dialogue context to obtain the entity linking results of each entity referent.

[0011] In the embodiment of the present invention, the step of enhancing the context of a given entity reference in a document using a large language model includes:

[0012] The entity referent and the characters before and after the entity referent are taken as context, and the context enhancement is completed by asking and answering questions based on the question template of the large language model, wherein the question template includes: the text content and the enhanced context of the entity referent output by the large language model, wherein the text content includes the context of the entity referent.

[0013] In an embodiment of the present invention, the step of generating a candidate entity list for an entity reference term includes:

[0014] According to the enhanced context, the entity linking model is used to filter the entities in the knowledge base, and a candidate entity list containing multiple entities that are most likely to be linked is generated. The candidate entity list of the entity referent and the matching score of each candidate entity are obtained, and a preset number of candidate entities are taken as the candidate entity list.

[0015] In the embodiment of the present invention, the step of generating description information for each candidate entity in the candidate entity list includes:

[0016] The description information includes: a first part of the content is one of the contents of the corresponding entity description information in the knowledge base, one of which is a summary description of the current entity; and a second part of the content is the most context-relevant content recalled from the corresponding entity description information in the knowledge base, wherein the recall adopts the vector recall method;

[0017] After the entity document is divided into multiple blocks according to a fixed character length, an embedding is generated for each block using a text embedding model. The cosine similarity between the embedding of the enhanced context and the embedding of all blocks is calculated. The block with the highest score is used as the second part of the candidate entity description information; the first and second parts of the description are concatenated as the description information of the candidate entity.

[0018] In the embodiment of the present invention, the question-answering task step of constructing a multiple-choice question for each entity referent includes:

[0019] For entity referents, the obtained enhanced context is used as the context of the multiple-choice question, the obtained candidate entity list is used as the options, and the obtained candidate entity description information is used as the description information of each option;

[0020] The template for multiple-choice questions includes: context, question, options, and the last option is none of the above.

[0021] In the embodiment of the present invention, the step of constructing a reference graph and obtaining the degree of association between entity referents includes:

[0022] A weighted graph is constructed containing two types of nodes: entity referents and candidate entities. The weight between an entity referent and a candidate entity is the matching score between the candidate entity and the entity referent. The weight between candidate entities is the semantic relevance between the candidate entities. This semantic relevance is calculated by counting the number of in-link entities shared by two entities in the knowledge base.

[0023] The association between entity referents is calculated using a random walk algorithm with restarts. Starting from each entity referent to be disambiguated, its propagation path in the reference graph is explored to measure its correlation with other entity referents.

[0024] In the embodiment of the present invention, the step of sorting all entity reference items includes:

[0025] A heuristic method is used to estimate the link difficulty of entity referents and sort them. The highest matching score among the candidate entities of the entity referent is calculated as the matching difficulty score of the entity referent, and the entities are sorted according to the score.

[0026] In an embodiment of the present invention, the step of sequentially inputting the multiple-choice questions corresponding to the entity referents into the large language model for answering based on the sorting results, selecting question-answer pairs of entity referents whose relevance to the current entity referent exceeds a threshold as the conversation context, and obtaining an entity linking result for each entity referent includes:

[0027] When asking a multiple-choice question about one of the entity referents, select up to N entity referent links with the highest correlation with the current entity referent from previously processed entity referents, including the multiple-choice question and the answer from the large model as the conversation context; where N is the number of entity referents used as the conversation context;

[0028] The question template of the large language model includes: task definition, instruction description and examples, among which the instruction description includes: context, question, options and answers.

[0029] In a second aspect, an embodiment of the present application provides an entity linking system based on a large language model, which adopts the entity linking method based on the large language model described above, and the system includes:

[0030] Context Enhancement Module: Used to enhance the context of entity referents using a large language model;

[0031] Candidate entity generation module: used to generate a list of candidate entities for entity referents;

[0032] Description information recall module: used to generate description information for each candidate entity in the candidate entity list;

[0033] Multiple-choice construction module: used to construct a fixed-format multiple-choice question for each entity referent using the enhanced context, candidate entity list, and candidate entity description information;

[0034] Entity referent association calculation module: This module uses the entity referents in the document and the candidate entity list generated for the entity referents to construct a reference graph, and uses a random walk algorithm with restarts to obtain the association between the entity referents;

[0035] Sorting module: used to sort all entity referents;

[0036] Entity linking module: Based on the sorting results, the multiple-choice questions corresponding to the entity referents are input into the large language model for answering. During input, the question-answer pairs of entity referents whose correlation with the current entity referent exceeds the threshold are selected as the dialogue context to obtain the entity linking results for each entity referent.

[0037] In a third aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of an entity linking method based on a large language model when the program is executed by a processor.

[0038] In a fourth aspect, an embodiment of the present application provides an electronic device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the program, the steps of the entity linking method based on a large language model are implemented.

[0039] Compared with the related existing technologies, it has the following outstanding beneficial effects:

[0040] The method of the present invention converts the entity linking task into a multiple-choice question answering task, fully utilizing the context within the document and the rich world knowledge inherent in the large language model for reasoning. It does not require pre-defined rules, large amounts of labeled data, or training or fine-tuning of the model, and has low maintenance costs and good generalization.

[0041] The method of the present invention processes the linking task of all entity referents in a document in the same conversation context of a large model, can effectively capture the potential connections between entity referents, and performs well in the document-level entity linking task. BRIEF DESCRIPTION OF THE DRAWINGS

[0042] The drawings described herein are used to provide a further understanding of the present application and constitute a part of the present application. The illustrative embodiments of the present application and their descriptions are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0043] Figure 1 Schematic diagram of the entity linking method based on a large language model of the present invention;

[0044] Figure 2 Schematic diagram of the entity linking system based on the large language model of the present invention;

[0045] Figure 3 Schematic diagram of computer hardware of the present invention. DETAILED DESCRIPTION

[0046] In this disclosure, "at least one" means one or more, and "plurality" means two or more. "At least one of the following" or similar expressions refers to any combination of these items, including any combination of single or plural items. For example, "at least one of a, b, or c" can mean: a, b, c, ab, ac, bc, or abc, where a, b, and c can be single or plural.

[0047] It should also be understood that the term "and / or" in this document simply describes an association between related objects, indicating that three possible relationships exist. For example, "A and / or B" can represent: A alone, A and B together, or B alone. A and B can be singular or plural. Furthermore, the character " / " in this document generally indicates an "or" relationship between the related objects, but it may also indicate an "and / or" relationship. For specific understanding, please refer to the context.

[0048] It should also be understood that in various embodiments of the present invention, the size of the serial numbers of the above-mentioned processes does not mean the order of execution. The execution order of each process should be determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiments of the present invention.

[0049] In the several embodiments provided by the present invention, it should be understood that the disclosed devices, apparatuses, and methods can be implemented in other ways. For example, the device embodiments described above are merely illustrative. For example, the division of the units is merely a logical function division. In actual implementation, there may be other division methods, such as multiple units or components can be combined or integrated into another device, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be through some interface, indirect coupling or communication connection of the device or unit, which can be electrical, mechanical or other forms.

[0050] The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed across multiple network units. Some or all of these units may be selected to achieve the purpose of this embodiment according to actual needs.

[0051] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit.

[0052] If the functions are implemented as software functional units and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the portion that contributes to the prior art, or the portion of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for enabling a computer device (which can be a personal computer, server, or network device, etc.) to execute all or part of the steps of the methods described in various embodiments of the present invention. The aforementioned storage media include various media that can store program code, such as USB flash drives, mobile hard drives, read-only memories (ROM), random access memories (RAM), magnetic disks, or optical disks.

[0053] To illustrate the above-mentioned features and effects of the present invention more clearly and easily, the following embodiments are specifically described below with reference to the accompanying drawings. This specification discloses one or more embodiments incorporating the features of the present invention. The disclosed embodiments are for illustrative purposes only. The scope of protection of the present invention is not limited to the disclosed embodiments; the present invention is defined by the appended claims.

[0054] The following is a system embodiment corresponding to the above method embodiment. This embodiment can be implemented in conjunction with the above embodiment. The relevant technical details mentioned in the above embodiment are still valid in this embodiment and will not be repeated here to reduce repetition. Accordingly, the relevant technical details mentioned in this embodiment can also be applied to the above embodiment.

[0055] The method of the present invention aims to propose an entity linking method and system based on multi-round dialogue technology using a large language model. The method of the present invention converts the entity linking task into a multiple-choice question-answering task, fully utilizing the context within the document and the rich world knowledge inherent in the large language model for reasoning. It does not require pre-defined rules, large amounts of labeled data, or training or fine-tuning of the model, resulting in low maintenance costs and good generalization. The method of the present invention processes the linking task of all entity referents in the document within the same dialogue context of the large model, effectively capturing the potential connections between entity referents and performing well on document-level entity linking tasks.

[0056] The following describes the method of the embodiment of the present application in detail with reference to specific embodiments:

[0057] Example 1

[0058] like Figure 1 As shown, the embodiment of the present application proposes an entity linking method based on a large language model, the method comprising:

[0059] Step S1: Use the large language model to enhance the context of the entity reference term given in the entity document; in a specific embodiment of the present invention, step S1 can summarize and generalize the description of the entity reference term in the original document, reduce the context length and increase the information density; in addition, the world knowledge of the large language model itself can be used to supplement the description information of the entity reference term.

[0060] Step S2: Generate a candidate entity list for the entity referent. In a specific embodiment of the present invention, the number of entities to be matched in step S2 is in the millions, and the length of the conversation context of the large language model is limited. Step S2 can effectively filter out most irrelevant entities, retaining only a small number of entities that may be linked as candidate entities.

[0061] Step S3: Generate description information for each candidate entity in the candidate entity list; In a specific embodiment of the present invention, in step S3, since the names of the candidate entities are usually very similar, the description information can help the large language model effectively distinguish these entities.

[0062] Step S4: Using the enhanced context, candidate entity list, and candidate entity description information, a multiple-choice question-answering task is constructed for each entity reference. In a specific embodiment of the present invention, the entity linking task in step S4 is a typical task with complex, detailed, and strict requirements. Although large language models are good at understanding and generating natural language, they may have difficulty accurately following very specific and complex task instructions. By converting the entity linking task into a multiple-choice question-answering task, it is more in line with the characteristics of text generation by large language models and can effectively improve performance. In addition, by limiting the large language model to only select from given options, hallucinations can be avoided.

[0063] Step S5: constructing a reference graph using the entity referents and the candidate entity list to obtain the correlation between the entity referents; in this embodiment of the present invention, a random walk algorithm with restart is used for calculation;

[0064] Step S6: sorting all entity referents. The sorting in step S5 is to estimate the linking difficulty of the entity referents by a heuristic method. The entity referents that are easier to be linked are processed first in step S7.

[0065] Step S7: Based on the sorting results, the multiple-choice questions corresponding to the entity referents are sequentially input into the large language model for answering. During the input, the question-answer pairs of the entity referents whose relevance to the current entity referent exceeds a threshold are selected as the conversation context to obtain the entity linking results for each entity referent;

[0066] In a specific embodiment of the present invention, when processing subsequent entity referent links, the association information obtained in step S5 is used to select up to N entity referent links with the highest association with the current entity referent from previously processed entity referents. This includes multiple-choice questions and responses from the large model as conversation context, thereby indirectly considering the connections between them. The sorting performed in step S5 avoids the potential accumulation of errors caused by previous linking errors. Furthermore, because the most difficult entities to match are processed later, they have more contextual information to utilize, resulting in higher accuracy. By limiting the number and association of conversation contexts, the overhead of conversations with the LLM is effectively reduced.

[0067] In the embodiment of the present invention, the step S1 of enhancing the context of a given entity reference in a document using a large language model includes:

[0068] The entity referent and the characters before and after the entity referent are taken as context, and the context enhancement is completed by asking and answering questions based on the question template of the large language model, wherein the question template includes: the text content and the enhanced context of the entity referent output by the large language model, wherein the text content includes the context of the entity referent.

[0069] More specifically, in an embodiment of the present invention, a large language model is used to enhance the context of entity references. Context It refers to the text content near the entity reference in the document. Specifically, in this embodiment, the entity reference and the 150 characters before and after the entity reference are used as context. Context enhancement is achieved by answering questions using a large language model, which includes but is not limited to Wenxin Yiyan, Tongyi Qianwen, DeepSeek, etc. The specific question template is as follows:

[0070] Consider the following text:

[0071] Text content: {context of the entity referent}

[0072] Based on the text above, please provide more descriptive information about the {entity referent}. Your answer must include the {entity referent}. Answer as follows:

[0073] The output of the large language model is the entity reference Enhanced context .

[0074] In the embodiment of the present invention, the step S2 of generating a list of candidate entities for the entity reference term includes:

[0075] According to the enhanced context, the entity linking model is used to filter the entities in the knowledge base, and a candidate entity list containing multiple entities that are most likely to be linked is generated. The candidate entity list of the entity referent and the matching score of each candidate entity are obtained, and a preset number of candidate entities are taken as the candidate entity list.

[0076] More specifically, in an embodiment of the present invention, a candidate entity list is generated for an entity reference term. The number of entities to be matched is often in the millions, and the length of the conversation context of a large language model is limited, so it is necessary to first filter the entities in the knowledge base and generate a list containing the entities that are most likely to be linked. entities as candidate entity lists In this embodiment, the traditional entity linking model BLINK is used for filtering. Specifically, for entity reference items , the enhanced context obtained in step S1 Input to the BLINK model to obtain the candidate entity list of the entity referent and the matching score of each candidate entity, and take the top k candidate entities as the candidate entity list In this embodiment, k=10.

[0077] In the embodiment of the present invention, the step S3 of generating description information for each candidate entity in the candidate entity list includes:

[0078] The description information in step S3 includes two parts. The first part is the first sentence of the corresponding entity description information in the knowledge base, which is usually a summary description of the current entity; the second part is the sentence with the strongest relevance to the enhanced context obtained in step S1, which is recalled from the corresponding entity description information in the knowledge base using the vector recall method.

[0079] The description information includes: a first part of the content is one of the contents of the corresponding entity description information in the knowledge base, one of which is a summary description of the current entity; and a second part of the content is the most context-relevant content recalled from the corresponding entity description information in the knowledge base, wherein the recall adopts the vector recall method;

[0080] After the entity document is divided into multiple blocks according to a fixed character length, an embedding is generated for each block using a text embedding model. The cosine similarity between the embedding of the enhanced context and the embedding of all blocks is calculated. The block with the highest score is used as the second part of the candidate entity description information; the first and second parts of the description are concatenated as the description information of the candidate entity.

[0081] More specifically, in the embodiment of the present invention, description information is generated for each candidate entity in the candidate entity list. Since the names of the candidate entities are usually very similar, such as apple, Apple Company, apple tree, etc., adding description information to the candidate entities can help the large language model to distinguish and identify them. The description information consists of two parts. Assume that there is a description of the entity in the knowledge base. The description document is , then the first part of the description information is the first sentence of the description document, which is usually the document summary and the most comprehensive and concise introduction to the entity, that is, The second part is the enhanced context obtained in step S1, which is recalled by vector recall. Most similar sentences.

[0082] Specifically, the document will be described Divide into multiple blocks according to fixed character length, . Then use the text embedding model , generating embeddings for each block In order to obtain the description information that best matches the context, the enhanced context embedding obtained in step S1 is subjected to cosine similarity calculation with the embedding of all blocks of the entity document. The block with the highest score is used as the second part of the candidate entity description information, that is:

[0083]

[0084] Finally, in the embodiment of the present invention, the two parts of the description are spliced ​​as the candidate entity Description information In this embodiment, the character length is selected as 150, the overlapping character length between blocks is 20, and the text embedding model is selected as Qwen2.5-32B-Instruct.

[0085] In the embodiment of the present invention, the question-answering task step S4 of constructing a multiple-choice question for each entity referent includes:

[0086] For entity referents, the obtained enhanced context is used as the context of the multiple-choice question, the obtained candidate entity list is used as the options, and the obtained candidate entity description information is used as the description information of each option;

[0087] The template for multiple-choice questions includes: context, question, options, and the last option is none of the above.

[0088] More specifically, in the embodiment of the present invention, a multiple-choice question is constructed for each entity reference item. Since the entity linking task is a typical task with complex, detailed and strict requirements, and the large language model has difficulty in accurately following very specific and complex task instructions, it performs poorly. Therefore, converting the entity linking task into a multiple-choice question answering task is more in line with the characteristics of large language model text generation. Specifically, for the entity reference item , the enhanced context obtained in step S1 As the context of the question, the candidate entity list obtained in step S2 As options, the information obtained in step S3 is used as the description of each option. In particular, the last option is "None of the above". The specific single-choice question template is as follows:

[0089] Context: {enhanced context of the entity referent}

[0090] Question: Which of the following entities is the {entity referent} described above?

[0091] Options:

[0092] Candidate entity 1: description information of candidate entity 1

[0093] Candidate entity 2: description information of candidate entity 2

[0094] …

[0095] (k+1) None of the above.

[0096] In the embodiment of the present invention, the above construction of the reference graph including entity referents and candidate entities and the calculation of the association between the entity referents by the random walk algorithm with restarts, step S5 includes:

[0097] Construct a weighted graph containing two types of nodes: entity referents and candidate entities. The weight between an entity referent and a candidate entity is the matching score between the candidate entity and the entity referent obtained in step 2. The weight between candidate entities is the semantic relevance between the candidate entities. The semantic relevance is obtained by calculating the number of in-link entities shared by two entities in the knowledge base.

[0098] The association between entity referents is calculated using a random walk algorithm with restarts. Starting from each entity referent to be disambiguated, its propagation path in the reference is explored, and its correlation with other entity referents is measured.

[0099] A reference graph is constructed using all entity referents and all candidate entities recalled for the entity referents in step S2. The graph is a weighted graph, where the weight between the entity referent and the candidate entity is the matching score obtained in step S2, and the weight between the candidate entities is the semantic association between the two entities.

[0100] The semantic relevance is obtained by calculating the number of in-link entities shared by two entities in the knowledge base. The calculation method is:

[0101]

[0102] Where a and b represent two candidate entities, A and B are the sets of entities linked to a and b in the knowledge base, respectively. |A ∩B| represents the number of entities that are linked to both entities, and |W| is the total number of entities in the knowledge base. The higher the number of in-link entities two entities share (i.e., frequent contextual co-occurrence), the higher their semantic relevance SR(a, b).

[0103] The association index between entity referents is calculated by random walk algorithm with restart, starting from entity referent m i , explore its propagation path in the semantic graph, and then measure its relevance to other entity mentions. Specifically, first define the normalized transfer matrix A according to the graph structure, and the initial distribution p (0) Set to node m i The upper value is 1, and the rest are 0. Each step of the walk is updated as follows:

[0104]

[0105] Iterate until convergence and obtain the steady-state distribution p * , where each dimension p *(v) represents the node v relative to the starting point m i The access probability p * (v) All other entities are referred to as terms m. j The access probability p * (m j ) as m i and m j Finally, in the collective entity linking process, the disambiguation decision within the entire document can be jointly optimized based on the association information between these entity referents.

[0106] In an embodiment of the present invention, the above-mentioned step S6 of sorting all entity reference terms includes: sorting all entity reference terms means using a heuristic method to evaluate the link difficulty of the entity reference terms, and the entity reference terms with lower evaluation difficulty have higher scores and are processed earlier in step S7.

[0107] A heuristic method is used to estimate the link difficulty of entity referents and sort them. The matching score of one of the candidate entities of the entity referent is calculated as the matching difficulty score of the entity referent, and the entities are sorted according to the score.

[0108] More specifically, in an embodiment of the present invention, all entity reference items are sorted. Since the linking difficulty of different entity reference items varies, even in the same document, the information density and quality of their contexts are also different, so a heuristic method is first used to estimate the linking difficulty of the entity reference items and sort them. In this embodiment, the matching score of the first candidate entity of the entity reference item obtained in step S2 is used as the matching difficulty score of the entity reference item. The higher the score, the easier it is to process, and the items are sorted from high to low according to the score.

[0109] In the embodiment of the present invention, according to the sorting results, the multiple-choice questions corresponding to the entity referents are sequentially input into the large language model for answering. During the input, question-answer pairs of entity referents whose relevance to the current entity referent exceeds a threshold are selected as the conversation context to obtain the entity linking result of each entity referent. Step S7 includes:

[0110] When asking a multiple-choice question about one of the entity referents, select up to N entity referent links with the highest correlation with the current entity referent from previously processed entity referents, including the multiple-choice question and the answer from the large model as the conversation context; where N is the number of referents used as the conversation context;

[0111] In a specific embodiment of the present invention, a maximum of N entity referents are selected from entity referents that meet two conditions, and their question-answer pairs are used as conversation contexts. The two conditions include:

[0112] 1. The entity referent has completed the link;

[0113] 2. The relevance between the entity reference term and the entity reference term currently being processed exceeds a threshold.

[0114] More specifically, in the embodiment of the present invention, based on the sorting results, the multiple-choice questions corresponding to the entity referents are sequentially input into the large language model for answering. During the input, the question-answer pairs of up to N entity referents with the highest correlation with the current entity referent are used as the dialogue context to obtain the entity linking result of each entity referent. Specifically, when asking the question about the i-th entity referent m i Multiple choice question Q i When the question-answer pair of the associated entity referent is passed to the large language model as the dialogue context; the associated entity referent refers to the entity referent with the highest correlation degree with the current entity referent, that is, the entity referent with the highest correlation degree with the current entity referent obtained in step S5 and exceeding the threshold δ, and the entity linking is completed before processing the current entity referent, and the entity referent is selected from the highest to the lowest according to the correlation degree, which is denoted as R i The question-answering of the associated entity reference items is the answer to the multiple-choice questions and the large model when processing these reference items. , when processing the current entity reference, the input and answer of the large language model are:

[0115]

[0116] Specifically, the sorting result here means that the lower the ranking in step S7, that is, the more difficult it is to be processed, the later it is processed in this step, so that the richer the conversation context information that can be used is, and the higher the accuracy rate can be.

[0117] For example: Assume that the entity referents in the document are sorted in order of matching difficulty from easy to difficult as m1, m2, m3, m4, and m5, and each entity referent is processed in turn.

[0118] 1. Process m1 and input the multiple-choice questions of m1 into LLM to get the answers.

[0119] 2. Process m2 and determine whether the correlation between m1 and m2 exceeds the threshold. If so, when the multiple-choice question for m2 is input to the LLM, the question-answer pair of m1 is also input as the conversation context. ...

[0121] 5. Process m5. Select up to N entity referents with the highest correlation with m5 and exceeding the threshold from the processed m1, m2, m3, and m4 as related entity referents. Input the question-answer pairs of these related referents into the LLM as the conversation context along with the multiple-choice questions for processing m5.

[0122] Specifically, the prompt here refers to a question template, which consists of a task definition, instruction description, and examples, as follows:

[0123] Task definition:

[0124] You are an assistant who is very good at natural language processing. You will complete entity linking by selecting a single option. Your task is to select the most matching entity from the list of candidate entities given a context. If none of them match, select "None of the Above".

[0125] Instruction Description:

[0126] Context: Please analyze the given text carefully.

[0127] Question: Determine the meaning of the entity referent in a sentence based on the context.

[0128] Options: Select the entity that best fits the meaning of the entity referent. If there is none, select "None of the Above".

[0129] Answer: First give the option number and the name of the entity you choose, such as (1) apple. Then give your explanation as to why you chose this option instead of the other options.

[0130] Example:

[0131] Context: Apple is a company that produces mobile phones, tablets, and computers.

[0132] Question: Which of the following entities is the apple described above?

[0133] Options:

[0134] (1) Apple: a round, edible fruit.

[0135] (2) Apple (technology company): Apple is an American multinational technology company.

[0136] (3) None of the above.

[0137] Answer: (2) Apple (a technology company). Because the Apple mentioned in the passage is a company that produces high-tech products, not a fruit.

[0138] In this embodiment of the present invention, by allowing the large model to select from given options rather than generating text in an open-ended manner, the hallucination problem of large language models can be effectively avoided. Entity linking for the entire document within the same conversational context can fully utilize the relationships between entity referents in the document, improving the accuracy of document-level entity linking.

[0139] As described above, the method of the present invention can be better implemented.

[0140] Example 2

[0141] like Figure 2 As shown, the embodiment of the present application provides an entity linking system 200 based on a large language model, which adopts the entity linking method based on the large language model as described above. The system 200 includes:

[0142] Context enhancement module 210: used to enhance the context of entity referents using a large language model;

[0143] Candidate entity generation module 220: used to generate a candidate entity list for the entity referent;

[0144] Description information recall module 230: used to generate description information for each candidate entity in the candidate entity list;

[0145] Single-choice construction module 240: used to construct a fixed-format single-choice question for each entity reference using the enhanced context, candidate entity list, and description information of the candidate entity;

[0146] Entity referent relevance calculation module 250: constructs a reference graph using entity referents in the document and a candidate entity list generated for the entity referents, and obtains the relevance between the entity referents using a random walk algorithm with restarts;

[0147] Sorting module 260: used to sort all entity reference items,

[0148] Entity linking module 270: Based on the sorting results, the multiple-choice questions corresponding to the entity reference items are input into the large language model in turn for answering. When inputting, the question-answer pairs of entity reference items whose correlation with the current entity reference item exceeds the threshold are selected as the dialogue context to obtain the entity linking results of each entity reference item.

[0149] Example 3

[0150] An embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of the entity linking method based on a large language model.

[0151] Example 4

[0152] An embodiment of the present application provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the program, the steps of the entity linking method based on a large language model are implemented.

[0153] In addition, combined Figure 1 The entity linking method based on a large language model described in the embodiment of the present application can be implemented by an electronic device, such as a computer device. Figure 3 Schematic diagram of the hardware structure of a computer device according to an embodiment of the present application.

[0154] In some embodiments, the computer device may further include a communication interface 83 and a bus 80. Figure 3 As shown, the processor 81, the memory 82, and the communication interface 83 are connected via a bus 80 and communicate with each other.

[0155] Specifically, the processor 81 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0156] The memory 82 may be used to store or cache various data files required for processing and / or communication, as well as possible computer program instructions executed by the processor 81 .

[0157] The processor 81 implements any one of the entity linking methods based on a large language model in the above embodiments by reading and executing computer program instructions stored in the memory 82 .

[0158] The technical features of the above-mentioned embodiments can be combined arbitrarily. In order to make the description concise, not all possible combinations of the technical features in the above-mentioned embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0159] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present invention. It should be noted that a person skilled in the art could make various modifications and improvements without departing from the spirit of the present application, all of which fall within the scope of protection of the present application. Therefore, the scope of protection of the present patent application shall be determined by the appended claims.

Claims

1. An entity linking method based on a large language model, characterized in that: The method comprises: Using a large language model to enhance the context of an entity reference term given in an entity document, generating a candidate entity list for the entity reference term, and generating description information for each candidate entity in the candidate entity list; Using the enhanced context, the candidate entity list, and the description information of the candidate entities, construct a multiple-choice question answering task for each entity referent; A reference graph is constructed using the entity referents and the candidate entity list to obtain the correlation between the entity referents; All entity referents are sorted, and based on the sorting results, the multiple-choice questions corresponding to the entity referents are sequentially input into the large language model for answering. During the input, question-answer pairs of entity referents whose relevance to the current entity referent exceeds a threshold are selected as the conversation context to obtain entity linking results for each entity referent; The question-answering task step of constructing a multiple-choice question for each entity reference term includes: For the entity referent, the enhanced context is used as the context of the multiple-choice question, the candidate entity list is used as the options, and the candidate entity description information is used as the description information of each option; The template of the multiple-choice question includes: context, question, options, and the last option is none of the above.

2. The entity linking method based on a large language model according to claim 1, characterized in that: The step of enhancing the context of a given entity reference term in the entity document using the large language model includes: The entity reference term and the characters before and after the entity reference term are taken as context, and the context enhancement is completed by asking and answering the large language model according to the question template, wherein the question template includes: text content and the enhanced context of the entity reference term output by the large language model, wherein the text content includes the context of the entity reference term.

3. The entity linking method based on a large language model according to claim 1, characterized in that: The step of generating a candidate entity list for the entity referent includes: According to the enhanced context, the entity linking model is used to filter the entities in the knowledge base, and a candidate entity list containing multiple entities that are most likely to be linked is generated. The candidate entity list of the entity referent and the matching score of each candidate entity are obtained, and a preset number of the candidate entities are taken as the candidate entity list.

4. The entity linking method based on a large language model according to claim 1, characterized in that The step of generating description information for each candidate entity in the candidate entity list includes: The description information includes: a first part of content that is one of the contents of the corresponding entity description information in the knowledge base, one of which is a summary description of the current entity; and a second part of content that is the content most closely associated with the context that is recalled from the corresponding entity description information in the knowledge base, wherein the recall adopts a vector recall method; After the entity document is divided into multiple blocks according to a fixed character length, an embedding is generated for each block using a text embedding model. The cosine similarity between the enhanced embedding of the context and the embedding of all the blocks is calculated. The block with the highest score is used as the second part of the candidate entity description information; the first and second parts of the description are spliced ​​as the description information of the candidate entity.

5. The entity linking method based on a large language model according to claim 1, characterized in that The step of sorting all entity referents includes: A heuristic method is used to estimate the link difficulty of entity reference terms and to sort them, a matching score of one of the candidate entities of the entity reference term is calculated as the matching difficulty score of the entity reference term, and the entities are sorted according to the score.

6. The entity linking method based on a large language model according to claim 1, characterized in that: The step of sequentially inputting the multiple-choice questions corresponding to the entity referents into the large language model for answering based on the sorting results, selecting the question-answer pairs of the entity referents whose relevance to the current entity referent exceeds a threshold as the conversation context, and obtaining the entity linking result of each entity referent, includes: When asking a multiple-choice question about one of the entity referents, select, from previously processed entity referents, up to N entity referent links with the highest degree of relevance to the current entity referent, including the multiple-choice question and the answer from the large model as the conversation context; where N is the number of entity referents used as the conversation context; The question template of the large language model includes: task definition, instruction description and examples, wherein the instruction description includes: context, question, options and answers.

7. An entity linking system based on a large language model, using the entity linking method based on a large language model as described in any one of claims 1 to 6, characterized in that: The system comprises: Context Enhancement Module: Used to enhance the context of entity referents using a large language model; Candidate entity generation module: used to generate a list of candidate entities for entity referents; Description information recall module: used to generate description information for each candidate entity in the candidate entity list; A single-choice construction module is used to construct a fixed-format single-choice question for each entity reference item by using the enhanced context, the candidate entity list, and the description information of the candidate entity; Entity referent relevance calculation module: uses the entity referent and the candidate entity list to construct a reference graph to obtain the relevance between entity referents; Sorting module: used to sort all entity referents; Entity linking module: Based on the sorting results, the multiple-choice questions corresponding to the entity referents are sequentially input into the large language model for answering. During the input, the question-answer pairs of entity referents whose relevance to the current entity referent exceeds the threshold are selected as the dialogue context to obtain the entity linking results for each entity referent; The single-choice building block includes: For the entity referent, the enhanced context is used as the context of the multiple-choice question, the candidate entity list is used as the options, and the candidate entity description information is used as the description information of each option; The template of the multiple-choice question includes: context, question, options, and the last option is none of the above.

8. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the program is executed by a processor, the steps of the entity linking method based on a large language model described in any one of claims 1 to 6 are implemented.

9. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein: When the processor executes the program, the steps of the entity linking method based on a large language model are implemented as described in any one of claims 1 to 6.

Images