Entity linking method and device, computer equipment, storage medium and computer program product

Abstract

The invention relates to an entity linking method and device, computer equipment, a storage medium and a computer program product. The method comprises the steps of obtaining a to-be-processed word, and determining an entity set; the entity set comprises a plurality of standardized entities; retrieving in the entity set based on the to-be-processed word to obtain a plurality of candidate entities matched with the to-be-processed word; on the basis of the to-be-processed word and the candidate entities, constructing a plurality of selection prompt texts; and processing according to the multi-item selection prompt text to output a target identifier, and taking a candidate entity indicated by the target identifier as a link entity of the to-be-processed word. By adopting the method, the entity linking accuracy can be improved.

Description

Technical Field

[0001] The present application relates to the field of computer technology, and in particular to an entity linking method, apparatus, computer device, storage medium, and computer program product. Background Art

[0002] With the development of computer technology, entity linking technology has emerged. Entity linking technology can be widely applied in fields such as search engines, intelligent customer service, and biomedicine. In the biomedical field, BioEL (Biomedical Entity Linking) aims to associate mentions (terms to be mapped), such as diseases and drugs, with standard entities in biomedical ontologies, and then apply the association results to medical tasks such as medical question answering.

[0003] However, in entity linking technology, there is diversity in the expression of entities. Entities with similar surface forms can have different meanings, making it easy to link a mention to the wrong standard entity.

[0004] Traditionally, the cosine similarity between a mention and its candidate entities is calculated, and the candidate with the highest cosine similarity is selected as the mention's linked entity. However, when a mention is very similar to multiple candidate entities, the linking result is easily affected, leading to misjudgment of the incorrect linked entity. Summary of the Invention

[0005] Based on this, it is necessary to provide an entity linking method, apparatus, computer device, computer-readable storage medium and computer program product that can improve the accuracy of entity linking in order to address the above technical problems.

[0006] In a first aspect, the present application provides an entity linking method. The method comprises:

[0007] Obtaining a word to be processed and determining an entity set; the entity set includes multiple standardized entities;

[0008] Searching the entity set based on the word to be processed to obtain multiple candidate entities matching the word to be processed;

[0009] Constructing a multiple-choice prompt text based on the word to be processed and the multiple candidate entities;

[0010] The target identifier is outputted according to the multiple-choice prompt text, and the candidate entity indicated by the target identifier is used as the link entity of the word to be processed.

[0011] In a second aspect, the present application further provides a physical linking device. The device comprises:

[0012] A data acquisition module is used to acquire words to be processed and determine an entity set; the entity set includes multiple standardized entities;

[0013] A retrieval module, configured to search the entity set based on the word to be processed to obtain a plurality of candidate entities matching the word to be processed;

[0014] A text construction module, configured to construct a multiple-choice prompt text based on the word to be processed and the multiple candidate entities;

[0015] The linking module is used to process the multiple-choice prompt text to output a target identifier, and use the candidate entity indicated by the target identifier as a linking entity of the word to be processed.

[0016] In a third aspect, the present application further provides a computer device comprising a memory and a processor, wherein the memory stores a computer program, and the processor implements the steps of the above-mentioned entity linking method when executing the computer program.

[0017] In a fourth aspect, the present application further provides a computer-readable storage medium having a computer program stored thereon, which implements the steps of the above-mentioned entity linking method when executed by a processor.

[0018] In a fifth aspect, the present application further provides a computer program product, which includes a computer program that implements the steps of the above-mentioned entity linking method when executed by a processor.

[0019] The above-mentioned entity linking method, apparatus, computer device, storage medium and computer program product obtain a word to be processed and determine an entity set; the entity set includes multiple standardized entities; based on the word to be processed, a search is performed in the entity set to obtain multiple candidate entities that match the word to be processed; and then a multiple-choice prompt text can be constructed based on the word to be processed and the multiple candidate entities. Through the multiple-choice prompt text, the interactive relationship between each candidate entity and between the word to be processed and the candidate entity can be explicitly established, so that the interaction between different candidate entities and the word to be processed can be more comprehensively characterized. Furthermore, the target identifier is output based on the multiple-choice prompt text, and the candidate entity indicated by the target identifier is used as the link entity of the word to be processed. On the one hand, processing through the multiple-choice prompt text can directly compare and contrast the differences between different candidate entities, thereby improving the accuracy of entity linking. On the other hand, by outputting the target identifier instead of the specific entity name, the generation of invalid entities can be avoided. BRIEF DESCRIPTION OF THE DRAWINGS

[0020] Figure 1 An application environment diagram of an entity linking method in one embodiment;

[0021] Figure 2 Schematic diagram of a flow chart of an entity linking method in one embodiment;

[0022] Figure 3 is a flowchart of an entity linking method in one embodiment;

[0023] Figure 4 A flowchart illustrating steps for determining a similar example in one embodiment;

[0024] Figure 5 A schematic diagram of a link page in a biomedical scenario in one embodiment;

[0025] Figure 6 is a flowchart of an entity linking method in another embodiment;

[0026] Figure 7 Schematic diagram for evaluating the accuracy of entity linking methods;

[0027] Figure 8 Schematic diagram of prediction results of an entity linking method in one embodiment;

[0028] Figure 9 is a structural block diagram of a physical linking device in one embodiment;

[0029] Figure 10 FIG. 1 is a diagram showing the internal structure of a computer device in one embodiment. DETAILED DESCRIPTION

[0030] In order to make the purpose, technical solutions and advantages of this application more clear, the following further describes this application in detail with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain this application and are not intended to limit this application.

[0031] The entity linking method provided in the embodiment of the present application can be applied to Figure 1 In the application environment shown, terminal 102 communicates with server 104 via a network. The data storage system can store data that server 104 needs to process. The data storage system can be set up separately, integrated on server 104, or placed on the cloud or other network servers. Terminal 102 and server 104 can be used independently to execute the entity linking method of this application, or terminal 102 and server 104 can be used in conjunction to execute the entity linking method of this application.

[0032] Taking the collaborative execution of this application by terminal 102 and server 104 as an example, during the specific entity linking process, users can consult knowledge in various fields such as biomedicine, common sense, law, games, music, etc. based on their own needs. For example, in biomedicine, users can consult information related to diseases, genes, drugs, etc. Users can describe their questions through terminal 102 through text or voice. Terminal 102 processes the questions described by the user, obtains the words to be processed, and sends the words to be processed to server 104. Of course, terminal 102 can also directly send the questions described by the user to server 104, and the server processes the questions described by the user to obtain the words to be processed. Then, server 104 can determine an entity set; the entity set includes multiple standardized entities; server 104 searches the entity set based on the words to be processed to obtain multiple candidate entities that match the words to be processed; server 104 constructs a multiple-choice prompt text based on the words to be processed and the multiple candidate entities; server 104 processes the multiple-choice prompt text and outputs a target identifier, and uses the candidate entity indicated by the target identifier as the link entity of the words to be processed. Furthermore, the server 104 can use the structure and relationship of linked entities to perform precise queries and reasoning to provide more accurate information retrieval and reasoning capabilities through entity linking, and feed back the query results to the terminal 102, which helps researchers, doctors and decision makers better understand and apply knowledge in the biomedical field.

[0033] Terminal 102 runs an application, which refers to any computer program that provides an interactive platform between terminal 102 and server 104. Examples include biomedical applications, social networking applications, educational applications, shopping applications, and video applications, among other applications with business consulting services. Terminal 102 may be, but is not limited to, various desktop computers, laptops, smartphones, tablets, IoT devices, portable wearable devices, intelligent voice interaction devices, smart home appliances, in-vehicle terminals, aircraft, and the like. IoT devices may include smart speakers, smart TVs, smart air conditioners, and smart in-vehicle devices. Portable wearable devices may include smart watches, smart bracelets, and head-mounted devices. Server 104 may be implemented as a standalone server or a server cluster consisting of multiple servers. The server may also be a cloud server that provides basic cloud computing services such as cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, middleware services, domain name services, security services, CDN, and big data and artificial intelligence platforms. The terminal and server may be connected directly or indirectly via wired or wireless communication.

[0034] It should be noted that the embodiments of the present invention can be applied to various scenarios, including but not limited to cloud technology, artificial intelligence, smart transportation, assisted driving, etc.

[0035] Artificial Intelligence (AI) involves theories, methods, techniques, and application systems that use digital computers or machines controlled by digital computers to simulate, extend, and expand human intelligence, to perceive the environment, acquire knowledge, and use that knowledge to achieve optimal results. In other words, AI is a comprehensive technology within computer science that seeks to understand the essence of intelligence and produce new intelligent machines that can respond in a manner similar to human intelligence. AI also encompasses the study of the design principles and implementation methods of various intelligent machines, enabling them to possess the capabilities of perception, reasoning, and decision-making.

[0036] Artificial intelligence technology is a comprehensive discipline covering a wide range of fields, including both hardware-level and software-level technologies. Basic artificial intelligence technologies generally include sensors, dedicated artificial intelligence chips, cloud computing, distributed storage, big data processing technology, operating / interactive systems, mechatronics, and other technologies. Artificial intelligence software technology mainly includes computer vision technology, speech processing technology, natural language processing technology, and machine learning / deep learning. The following is a detailed introduction to the entity linking method of this application:

[0037] Before explaining this application in detail, some of the terms involved are explained first:

[0038] BioEL (Biomedical Entity Linking) is a natural language processing technique designed to link entities in medical text (such as diseases, drugs, and symptoms) to standardized entities in a medical knowledge base. Its goal is to associate terms in medical text with standardized entities in a medical knowledge base, thereby providing more accurate and consistent entity identification.

[0039] UMLS (Unified Medical Language System) is a medical knowledge base developed and maintained by the National Library of Medicine (NLM). It aims to integrate and standardize various terminologies, classifications, relationships, and semantic information in the medical field to promote interoperability and semantic consistency of medical information.

[0040] BERT (Bidirectional Encoder Representations from Transformers): A pre-trained language model based on the Transformer architecture, BERT learns universal language representations by pre-training on large amounts of unlabeled text. Unlike traditional unidirectional language models, BERT uses a bidirectional encoder that simultaneously considers contextual information, better capturing the semantics and context of words and sentences.

[0041] Multiple Choice Question Answering (MCQA) is a form of question-answering task. In MCQA, given a question and a set of candidate answers, the respondent needs to select the most appropriate answer as the correct answer to the question.

[0042] Bi-encoder: The bi-encoder model is a bidirectional encoder model that encodes two input texts into fixed-length vector representations. Each text is passed through a separate encoder, typically using a pre-trained language model (such as BERT) to generate a representation for the text. A similarity score is then calculated between the two text representations to determine their relevance or similarity. The bi-encoder model is suitable for tasks requiring efficient computation, such as information retrieval and sentence matching.

[0043] Cross-encoder: A cross-encoder model is a cross-encoder model that simultaneously encodes two input texts into a fixed-length vector representation. This model structure typically uses an encoder such as the Transformer to process two texts simultaneously and generate a joint representation of them. The joint representation is then input into a classifier or regressor to predict the relevance or similarity score of the text pair. The cross-encoder model is suitable for tasks that require a more global semantic understanding, such as textual entailment and question-answering systems.

[0044] A pre-training model, also known as a cornerstone model or a large model, refers to a deep neural network (DNN) with large parameters. It is trained on massive amounts of unlabeled data. Leveraging the function approximation capabilities of large-parameter DNNs, the pretrained machine learning (PTM) extracts common features from the data. Through techniques such as fine tuning, efficient parameter fine tuning (PEFT), and prompt-tuning, it is then adapted for downstream tasks. Therefore, pre-trained models can achieve ideal results in few-shot or zero-shot scenarios. PTMs can be categorized by the data modality they process, including language models (ELMO, BERT, GPT), vision models (swin-transformer, ViT, V-MOE), speech models (VALL-E), and multimodal models (ViBERT, CLIP, Flamingo, Gato). Multimodal models represent features from two or more data modalities. Pre-trained models are important tools for outputting AI-generated content (AIGC) and can also serve as a universal interface for connecting multiple task-specific models.

[0045] In one embodiment, Figure 2 As shown, an entity linking method is provided, which is applied to a computer device (the computer device can be specifically Figure 1 The terminal 102 or server 104 in FIG. 1 is used as an example to illustrate the method, which includes the following steps:

[0046] Step 202: Obtain a word to be processed and determine an entity set; the entity set includes multiple standardized entities.

[0047] Among them, pending words are words with specific meanings in the text that require entity linking. Specifically, pending words can refer to mentions. Mentions vary in description and naming, resulting in fragmented and confusing information. These are expressions in the text that require disambiguation. By entity linking pending words, pending words from different sources can be linked to an entity set that includes standardized entities. This establishes an association between the pending words and the standardized entities in the entity set, improving data consistency and comparability.

[0048] An entity is a word or phrase that has a descriptive meaning or a specific meaning. Entities can typically be names of people, places, organizations, products, or other items with a specific meaning within a specific field. Examples include, but are not limited to, names of diseases, drugs, and organisms in the biomedical field, or specialized terms in law.

[0049] Standardized entities are those that, after undergoing standardization, possess uniform regulations and universality. In the biomedical field, these standardized entities can be the various terms integrated into systems such as the Unified Medical Language System (UMLS), the Clinical Medicine Knowledge Base (CMKB), and the Chinese Unified Medical Language System (CUMLS).

[0050] An entity set can be a knowledge base that stores multiple standardized entities. A knowledge base is a semantic network that describes conceptual entities in the objective world and the relationships between them. An entity is a uniquely identified object stored in a knowledge base. Alternatively, an entity set can be a knowledge graph, a structured semantic knowledge base that symbolically describes concepts, entities, and their relationships in the physical world. At the data level, a knowledge graph consists of entities with types and attributes as nodes and relationships between entities as edges, forming a heterogeneous graph structure.

[0051] Specifically, the computer device can obtain natural language text sent by the user through the terminal, such as news statements or legal statements, and perform keyword extraction on the natural language text to obtain the words to be processed. It is understood that the terminal can also directly send the words to be processed to the computer device. For example, the terminal can directly send the words "ulcerated," "haemoglobin," "type 1 diabetes," "type 2 diabetes," etc. to the computer device.

[0052] Furthermore, the computer device retrieves entity sets that match the type of the word being processed. For example, for a word in the biomedical field, the computer device may retrieve entity sets that match the biomedical field; for a word in the legal field, the computer device may retrieve entity sets that match the legal field. This reduces the amount of data required for subsequent searches and improves entity linking efficiency.

[0053] In some embodiments, when a computer device extracts keywords from a natural language text, it may use a word segmentation tool or Deep Text (a text understanding engine based on deep learning), content classification, or other methods to perform word segmentation processing on the natural language text, extracting words from the natural language text, such as nouns, entity words, non-nouns, stop words, etc., or extracting words with a certain meaning in a certain field, such as the names of diseases, drugs, and organisms in the medical field, or proprietary vocabulary involved in law. Furthermore, the computer device can remove stop words and non-noun words after the word segmentation processing to obtain words to be processed. Among them, stop words refer to words that have no actual meaning, such as pronouns, auxiliary words, adjectives, adverbs, etc.

[0054] Step 204 : searching the entity set based on the word to be processed to obtain a plurality of candidate entities matching the word to be processed.

[0055] Among them, the candidate entity is an entity that is retrieved and has a certain association with the word to be processed. The candidate entity that matches the word to be processed can specifically be an entity that is identical or approximately identical to the word to be processed in entity name, expression form, and textual semantics. For example, if the word to be processed is "Zhang San", the entity name of the matching candidate entity can be "Zhang San", "Zhang San San", etc.; for another example, if the word to be processed is "ulcer", the entity name of the matching candidate entity can be "ulcerative lesion", "skin ulcer", "trauma", etc., and the embodiment of the present application does not limit this.

[0056] Specifically, the computer device can match the word to be processed with each standardized entity in the entity set to determine multiple candidate entities. When matching, the computer device can calculate the similarity, edit distance or similarity coefficient between the word to be processed and each standardized entity, and filter out candidate entities based on the calculation results. For example, the computer device can calculate the similarity coefficient between the word to be processed and each standardized entity, and filter out candidate entities based on the size of the similarity coefficient. Generally, the closer the similarity coefficient is to 1, the more similar the word to be processed and the standardized entity are; the closer the similarity coefficient is to 0, the less similar the word to be processed and the standardized entity are. The computer device can set a similarity coefficient threshold and determine entities that reach the similarity coefficient threshold as candidate entities. The similarity coefficient threshold can be set to 0.8. As long as the similarity coefficient reaches 0.8, the corresponding entity can be determined as a candidate entity.

[0057] For example, when performing matching, the computer device may calculate the similarity between the word to be processed and each standardized entity by calculating cosine similarity. When calculating the cosine similarity, the computer device may obtain a feature vector of the word to be processed and a feature vector of each standardized entity, and calculate the cosine similarity between the feature vector of the word to be processed and the feature vector of each standardized entity, and then determine the candidate entity based on the obtained cosine similarity.

[0058] Step 206: construct a multiple-choice prompt text based on the word to be processed and the multiple candidate entities.

[0059] A multiple-choice prompt is a text generated by combining a pending word with multiple candidate entities. For example, a multiple-choice prompt can be constructed by combining, fusing, or concatenating the pending word with multiple candidate entities.

[0060] The multiple-choice prompt text is a combination of the word to be processed and multiple candidate entities. It can more comprehensively represent the relationships between candidate entities and the word to be processed, and between candidate entities. By constructing the multiple-choice prompt text, the computer device effectively translates the entity linking task into a multiple-choice question-answering task. Specifically, in the multiple-choice question-answering task, the computer device selects the correct answer (i.e., the linked entity) from a set of answer options (i.e., candidate entities) based on the word to be processed.

[0061] Specifically, the computer device can perform text mapping processing based on the to-be-processed word and multiple candidate entities to obtain a text sequence, and determine multiple-choice prompt text based on the text sequence. When performing text mapping processing, the computer device can use a set mapping template to expand the mapping. The mapping template can have different implementation methods. For example, the template can be defined using a regular expression, and the text mapping can be expanded using the regular expression; the mapping template can also be defined using a conversion function, and the text mapping can be expanded using the conversion function.

[0062] In some embodiments, the computer device may combine the word to be processed and its matching candidate entities in a certain order to generate a multiple-choice prompt text. For example, the computer device may place the word to be processed first and then arrange the candidate entities in order, adding a predetermined marker between each word as a separator, such as a space, punctuation mark, serial number, or character.

[0063] Step 208: Process the multiple-choice prompt text to output a target identifier, and use the candidate entity indicated by the target identifier as a link entity of the word to be processed.

[0064] The target identifier is a symbol predicted based on the multiple-choice prompt text. The candidate entity indicated by the target identifier is the final mapping target, that is, the linked entity. The linked entity is the correct entity that matches the word to be processed, determined from the candidate entities.

[0065] The target identifier can be presented as text, or various identifiers such as letters, numbers or feature codes, which are not limited here. For each candidate entity, there is an entity identifier that matches it, and the entity identifier output by the computer device is the target identifier. For example, letters can be used as the entity identifier of the candidate entity. For any proxy word, there are three candidate entities corresponding to it. The entity identifiers of the three candidate entities can be "A", "B", and "C". When the identifier output by the computer device is "A", the target identifier is "A", and the candidate entity pointed to by "A" is determined as the linked entity. When the identifier output by the computer device is "B", the target identifier is "B", and the candidate entity pointed to by "B" is determined as the linked entity.

[0066] Specifically, when predicting the identifier based on the multiple-choice prompt text, the computer device may utilize a pre-trained neural network model to output the target identifier by inputting the multiple-choice prompt text into the neural network model. Alternatively, the computer device may utilize another identifier prediction algorithm other than a neural network model to output the target identifier. For example, the computer device may utilize rule-based methods, support vector machines (SVMs), decision tree algorithms, and other algorithms.

[0067] In some embodiments, the entity identifier can specifically be a sequential identifier of the candidate entity. For any candidate entity of the word to be processed, when the computer device determines the entity identifier of the candidate entity, it can be set according to the position of the candidate entity in the candidate entity ranking result. The candidate entity ranking result is obtained after sorting each candidate entity. When the computer device sorts each candidate entity, it can be expanded according to the degree of similarity between each candidate entity and the word to be processed, or it can be sorted randomly. For example, when candidate entity 1 is in the first place in the candidate entity ranking result, the entity identifier of candidate entity 1 can be A, and when candidate entity 1 is in the second place in the candidate entity ranking result, the entity identifier of candidate entity 1 can be updated to B.

[0068] In some embodiments, after determining a linked entity, the computer device can conduct further queries and searches based on the linked entity to obtain descriptive information related to the linked entity. For example, in the biomedical field, the descriptive information obtained from entity linking can include the name of a disease, symptoms, treatment methods, medications, surgeries, etc. For another example, if the word to be processed is a place name, the descriptive information obtained from entity linking can include the culture, scenic spots, transportation conditions, etc. associated with the place.

[0069] In one embodiment, reference Figure 3 As shown in the figure, it is a flowchart of the entity linking method in a specific application:

[0070] Figure 3 The process involves terminals and servers. Users can send text to the server through the terminal. The server obtains the text sent by the terminal, processes the text, obtains the words to be processed, and performs entity linking processing on the words to be processed to obtain the linked entities of the words to be processed.

[0071] The server may obtain a corresponding entity set based on the field to which the word to be processed belongs, for example, obtaining an entity set corresponding to the medical field, obtaining an entity set corresponding to the literary field, etc., but is not limited thereto.

[0072] Furthermore, the computer device can search within the entity set and retrieve the top-N most similar candidate entities for the processing word from the entity set, where N can be 2, 3, 4, and so on. The goal of entity linking is to map the processing word to the correct entity e (i.e., the linked entity) in an ontology ξ (i.e., the entity set), where e∈ξ. The computer device can construct a multiple-choice prompt based on the processing word and multiple candidate entities. This prompt is then input into a pre-trained neural network model, which outputs a target identifier, which is then used to determine the final linked entity. Each candidate entity is associated with an entity identifier, such as "A," "B," or "C." By inputting the processing word and all candidate entities into the neural network model, the differences between different candidate entities can be directly compared and contrasted, and the interactions between the processing word and candidate entities and between candidate entities can be effectively modeled. In addition, the structure of the multiple-choice prompt text causes the neural network model to output the entity identifier associated with the candidate entity it selects. By directly generating the entity identifier instead of the entity name, it can avoid generating entities that do not exist in the entity set and reduce the dependence on some normalization strategies.

[0073] In the above-mentioned entity linking method, a word to be processed is obtained and an entity set is determined; the entity set includes multiple standardized entities; a search is performed in the entity set based on the word to be processed to obtain multiple candidate entities that match the word to be processed; and then a multiple-choice prompt text can be constructed based on the word to be processed and the multiple candidate entities. Through the multiple-choice prompt text, the interactive relationship between each candidate entity and between the word to be processed and the candidate entity can be explicitly established, so that the interaction between different candidate entities and the word to be processed can be more comprehensively characterized. Furthermore, the target identifier is output based on the multiple-choice prompt text, and the candidate entity indicated by the target identifier is used as the link entity of the word to be processed. On the one hand, processing through the multiple-choice prompt text can directly compare and contrast the differences between different candidate entities, thereby improving the accuracy of entity linking. On the other hand, by outputting the target identifier instead of the specific entity name, the generation of invalid entities can be avoided.

[0074] In one embodiment, a search is performed in an entity set based on the word to be processed to obtain multiple candidate entities that match the word to be processed, including: encoding based on the word to be processed to obtain a first vector representation of the word to be processed; encoding based on each standardized entity to obtain a second vector representation corresponding to each standardized entity; calculating a first similarity between the first vector representation and each second vector representation; and determining the standardized entities corresponding to the first similarity that meets a first preset similarity condition as multiple candidate entities that match the word to be processed.

[0075] The vector representation is the representation of semantic information in the form of vectors. The first vector representation is the vector representation of the semantic information of the word to be processed; the second vector representation is the vector representation of each standardized entity.

[0076] The preset similarity condition is a condition set for performing similarity judgment. The first preset similarity condition is a condition used to determine whether the standardized entity is a candidate entity that matches the word to be processed. The first preset similarity condition can be set based on a first similarity threshold, a first sorting result, etc., where the first sorting result refers to the result obtained by sorting the first similarities by size. When setting the first preset similarity condition, adaptive adjustments can be made based on the actual entity linking accuracy, entity linking scenario, field, etc.

[0077] In some embodiments, the computer device may use various text encoding networks to encode the processed words and standardized entities to obtain corresponding vector representations. For example, the text encoding network may be BERT (Bidirectional Encoder Representations from Transformers), ALBERT (A Lite BERT, a deep learning model for natural language processing), BioBERT (Biology BERT, a biomedical language representation model), RoBERTa (Robustly Optimized BERT Pretraining Approach, a robustly optimized BERT pretraining model), etc., which are not limited in the embodiments of the present application.

[0078] In some embodiments, the first preset similarity condition may be determined based on a first similarity threshold, i.e., as long as the first similarity reaches the similarity threshold, the first preset similarity condition is determined to be satisfied. For example, the first similarity threshold may be set to 0.97. If a first similarity of 0.98 exists, the first preset similarity condition is satisfied, and the computer device may determine the standardized entity corresponding to the first similarity as a candidate entity.

[0079] In other embodiments, the first preset similarity condition may be determined based on the first ranking result, that is, as long as the ranking of any first similarity falls within a set order range, such as in the top 4, top 3, etc., the first preset similarity condition is determined to be satisfied. For example, the computer device may sort the first similarities according to the magnitude of the similarity, and if the first similarities ranked in the top 4 satisfy the first preset similarity condition, the computer device may determine the standardized entity corresponding to the first similarities ranked in the top 4 as a candidate entity.

[0080] In the above embodiment, the computer device determines the first vector representation of the word to be processed and the second vector representation corresponding to each standardized entity, and calculates the first similarity between the first vector representation and each second vector representation, so as to accurately determine multiple candidate entities matching the word to be processed, thereby improving the accuracy of entity linking.

[0081] In one embodiment, a multiple-choice prompt text is constructed based on a word to be processed and multiple candidate entities, including: sorting multiple candidate entities according to the similarity between the candidate entities and the word to be processed; determining the sequence identifiers corresponding to the candidate entities according to the sorting order of the candidate entities; combining the word to be processed, multiple candidate entities, and the sequence identifiers corresponding to each candidate entity according to a preset format to obtain a first text sequence; and determining the multiple-choice prompt text based on the first text sequence.

[0082] The sorting order is used to represent the order of each candidate entity in the sorting result. The sorting order can be used to clearly identify the position of each candidate entity in the sorting result.

[0083] A sequence identifier is a symbol that identifies an object or the sequential relationship between objects. For a candidate entity, the sequence identifier can be the entity identifier of the candidate entity. The sequence identifier not only distinguishes different candidate entities but also identifies the position of each candidate entity in the ranking result. The sequence identifier can be any identifier such as a number, a letter, or a feature code, and is not limited in this embodiment of the present application.

[0084] The preset format can be used to define the format for combining the pending word, candidate entities, and the sequence identifiers corresponding to each candidate entity. For example, the preset format can define the overall order of the pending word, candidate entities, and sequence identifiers, the markings between the pending word and each candidate entity (such as spaces, punctuation marks, or sequence numbers), the order of each candidate entity, and the representation format between a candidate entity and its sequence identifier. When setting the preset format, you can make adaptive adjustments based on the actual entity type and linking scenario.

[0085] Specifically, the computer device can obtain the similarity between the candidate entity and the word to be processed by calculating the Euclidean distance, cosine similarity, etc. between the candidate entity and the word to be processed. Furthermore, the computer device can sort the candidate entities according to the similarity to obtain a sorting result. The computer device can set a corresponding sequence identifier for each candidate entity according to the sorting order of the candidate entities. It is understood that when the order of any candidate entity in the sorting result changes, its sequence identifier will also change accordingly.

[0086] In some embodiments, the computer device may directly determine the first text sequence as the multiple-choice prompt text. The computer device may also process the first text sequence, such as by concatenating it with other text sequences, and then determine the concatenated result as the multiple-choice prompt text. For another example, the computer device may adjust the order of the candidate entities in the first text sequence and determine the adjusted first text sequence as the multiple-choice prompt text.

[0087] In a specific application, given an entity word, that is, mention, and its corresponding candidate entity: , represents the candidate entity set, For each candidate entity in the candidate entity set. Among them, each candidate entity is sorted from large to small according to its similarity with the mention. The computer device can use a simple template Will Mapped to a text sequence to obtain the first text sequence, the representation of the first text sequence can be: Among them, A, B, C and D are the sequential identifiers of each candidate entity.

[0088] In the above embodiment, the computer device determines the sequential identifiers corresponding to the candidate entities, and combines the word to be processed, multiple candidate entities, and the sequential identifiers corresponding to each candidate entity in a preset format to obtain a first text sequence, and determines a multiple-choice prompt text based on the first text sequence. The multiple-choice prompt text not only constructs the interaction between the word to be processed and the candidate entity, but also focuses on the fine-grained interaction between the candidate entities, which can provide a more comprehensive and detailed expression for the interaction between the word to be processed and the candidate entities, thereby improving the accuracy of entity linking.

[0089] In one embodiment, the entity linking method further includes: searching from a preset corpus based on the word to be processed to obtain multiple similar instances matching the word to be processed; wherein one instance contains a preset word, a candidate entity of the preset word, and a sequential identifier of the linked entity of the preset word; and mapping each similar instance to a corresponding second text sequence.

[0090] The preset corpus is a database used to store examples. The computer device can obtain examples corresponding to different fields, such as medicine and law, and construct a preset corpus based on the obtained examples. The computer device can also directly determine the preset corpus based on training examples in training data from historical training processes.

[0091] Each instance in the predefined corpus includes the predefined word, its candidate entities, and the sequential identifiers of its linked entities. Similar instances are instances where the predefined word in an instance has similar features, attributes, or semantics to the word being processed.

[0092] Specifically, the computer device may determine the degree of similarity between the word to be processed and each preset word, and then determine a plurality of similar instances matching the word to be processed based on the degree of similarity between the word to be processed and each preset word.

[0093] In some embodiments, the examples in the preset corpus are stored in the form of key-value pairs. The computer device can define the key-value pairs based on the training set: ,in It is calculated

[0094] (i.e., the feature vector of the i-th preset word), for The corresponding N candidate options, for The symbol of the corresponding correct answer (i.e., the linked entity), D is the training set. The computer device can build a database based on the key-value pair ,database Contains a collection of all key-value pairs constructed by all instances, the database The preset corpus, database The representation is as follows:

[0095]

[0096] In some embodiments, the computer device may perform text mapping based on multiple similar instances matching the word to be processed to obtain a second text sequence. When performing text mapping, the computer device may use regular expressions to expand the mapping of multiple similar instances.

[0097] Furthermore, determining the multiple-choice prompt text based on the first text sequence includes: constructing the multiple-choice prompt text based on the first text sequence and each second text sequence.

[0098] Specifically, when the computer device constructs the multiple-choice prompt text using the first text sequence and each second text sequence, it can combine the first text sequence and each second text sequence to obtain a combined text sequence, and determine the combined text sequence as the multiple-choice prompt text. The computer device can also select one of the first text sequence and each second text sequence, such as selecting the first text sequence or the second text sequence as the multiple-choice prompt text, which is not limited in this embodiment of the present application.

[0099] In the above embodiment, the computer device determines multiple similar instances that match the entity to be processed, determines a second text sequence through the multiple similar instances, and then determines a multiple-choice prompt text based on the first text sequence and the second text sequence. Therefore, on the basis of a more comprehensive understanding of the interaction between different candidate entities and the words to be processed, it is possible to refer to similar similar instances from the entire preset corpus as prediction clues, thereby enhancing the generalization ability of long-tail entities.

[0100] In one embodiment, reference Figure 4 As shown, searching from a preset corpus based on the word to be processed to obtain multiple similar instances matching the word to be processed includes the following steps:

[0101] Step 402: Encode the word to be processed to obtain a first vector representation of the word to be processed;

[0102] Step 404: Calculate a second similarity between the first vector representation and the vector representation of each preset word in the preset corpus;

[0103] Step 406 : Filter preset words corresponding to the second similarity that meet the second preset similarity condition, and use the instances associated with the filtered preset words as similar instances that match the word to be processed.

[0104] The second preset similarity condition is a condition set to determine whether the instance associated with the preset word is similar to the instance to be processed. The second preset similarity condition can be set based on a set second similarity threshold, a second sorting result, etc., wherein the second sorting result refers to the result obtained by sorting the second similarities by size. When setting the second preset similarity condition, adaptive adjustments can be made based on the actual entity linking accuracy, entity linking scenario, field, etc.

[0105] In some embodiments, the second preset similarity condition may be determined based on a second similarity threshold, i.e., as long as the second similarity reaches the second similarity threshold, the second preset similarity condition is determined to be satisfied. For example, the second similarity threshold may be set to 0.85. If a second similarity of 0.86 exists, the second preset similarity condition is satisfied, and the computer device may determine that the instance associated with the preset word corresponding to the second similarity is a similar instance.

[0106] In other embodiments, the second preset similarity condition may be determined based on a ranking result, i.e., as long as the ranking of any second similarity falls within a set order range, such as being in the top 5 or top 6, the second preset similarity condition is determined to be satisfied. For example, the computer device may sort the second similarities by similarity level, and if the second similarities ranked in the top 5 satisfy the second preset similarity condition, the computer device may determine the instance associated with the preset word corresponding to the second similarity as a similar instance.

[0107] In some embodiments, the instances that may be stored in the preset corpus may include the word to be processed, candidate entities corresponding to the word to be processed, and sequential identifiers of linked entities of the word to be processed. When the computer device searches the preset corpus, the instance with the highest similarity among the first N similar instances retrieved may be removed to avoid data leakage, i.e., the instance to which the word to be processed belongs may be removed.

[0108] In the above embodiment, the computer device determines the second similarity, and based on the second similarity and the set second preset similarity condition, quickly and accurately determines similar instances matching the word to be processed from the preset corpus.

[0109] In one embodiment, a target identifier is output based on a multiple-choice prompt text, and a candidate entity indicated by the target identifier is used as a link entity of the word to be processed, including: inputting the multiple-choice prompt text into an entity linking model; using the entity linking model to predict an identifier for the multiple-choice prompt text, obtain a target identifier, and use the candidate entity associated with the target identifier as a link entity of the word to be processed.

[0110] The entity linking model is a network model built using artificial intelligence algorithms and can be used to predict target identities. Typically, entity linking models can be constructed using a variety of algorithms, including supervised and unsupervised learning. In practice, entity linking models can be constructed using various generative models, including diffusion generative models, variational autoencoder models, and adversarial generative models.

[0111] Specifically, a computer device can input multiple-choice prompt text into an entity linking model to generate a target identifier, where the target identifier is a symbol associated with the linked entity. The input and output structure of the entity linking model only requires the prediction of a single symbol (such as A) rather than a specific entity name. This avoids generating invalid entities. Due to the uncertainty of the entity linking model, entities that are not in the entity set may be generated. Generating only a symbol associated with the answer can greatly reduce the prediction difficulty.

[0112] In some embodiments, the entity linking model can be specifically in the form of an EL-BERT model, which is a variant of the BERT (Bidirectional Encoder Representation from Transformers) model. BERT is a bidirectional encoding model that pre-trains deep bidirectional representations from unlabeled text by jointly adjusting the left and right contexts of all layers. The EL-BERT model improves the BERT model pre-training task and cleverly integrates the entity knowledge base information into the pre-training task, making it more suitable for the entity linking task scenario and improving the accuracy of entity linking.

[0113] In the above embodiment, the computer device inputs the multiple-choice prompt text into the entity linking model, which then predicts the target identifier. Outputting the target identifier significantly reduces the difficulty of prediction and improves entity linking efficiency. It also avoids generating entities that do not exist in the entity set, thereby improving entity linking accuracy.

[0114] In one embodiment, the training steps of the entity linking model include: obtaining sample data; the sample data includes sample words and sample candidate entities of the sample words; constructing an input-output training pair based on the sample data; the input-output training pair includes input data and output criteria, the input data includes sample words and sample candidate entities of the sample words, and the output criteria includes sequential identifiers corresponding to the linked entities of the sample words; inputting the input data into the entity linking model to be trained to obtain a predicted identifier predicted by the entity linking model to be trained; based on the difference between the predicted identifier and the output criteria, adjusting the parameters of the entity linking model to be trained and continuing the training until the training is completed to obtain a trained entity linking model.

[0115] The sample data is the data used for model training. The sample data can be determined based on natural language text from different fields that implement entity linking.

[0116] Sample data includes sample words and candidate entities associated with these words. These sample words and candidate entities vary across different fields. For example, in the biomedical field, a sample word might be "suffering from ulcers," and candidate entities might include "ulcerative mass," "ulcerous lesion," "healed ulcer," or "mucosal ulcer." Another example might be "haemoglobin," and candidate entities might include "haemoglobin C" and "hemoglobin." In other fields, a sample word might be "Li Si," and candidate entities might include "Li Si," "Li Si Si," or "Li Xiao Si."

[0117] The input-output training pair includes input data and output criteria. The input data is the features or variables used to train the entity linking model, and the output criteria is the target output of the entity linking model set during the training process.

[0118] In some embodiments, the computer device can construct input-output training pairs based on the sample words and sample candidate entities of the sample words in the sample data by combining and sorting them. For example, for the sample candidate entities corresponding to any sample word, the computer device can sort the sample candidate entities according to their similarity with the sample words, and combine the sample words with the sample candidate entities sorted by similarity to construct the input data; the computer device can determine the linked entities from the sample candidate entities, and determine the sequence identifiers corresponding to the linked entities as the output standard. It is understandable that for each sample data, the computer device can also increase the number and diversity of input-output training pairs by adding noise, randomly inserting or deleting the sample candidate entities of a sample word, etc., so as to improve the accuracy of the entity linking model.

[0119] In some embodiments, the computer device trains an entity link model based on the constructed input and output training pairs. The computer device can input input data into the entity link model to be trained, obtain a prediction identifier predicted by the entity link model to be trained, and update the model parameters of the entity link model to be trained based on the difference between the prediction identifier and the output standard. For example, the input layer in the entity link model to be trained can be updated, and training can be continued after the update until the training is completed. The computer device can also end the training when it determines that the training convergence conditions are met, thereby obtaining a trained entity link model.

[0120] In the above embodiment, the computer device can construct input-output training pairs based on sample data, calculate the difference between the predicted identification and the output standard, adjust the parameters of the entity link model to be trained, and continue to train the entity link model after adjusting the parameters, thereby efficiently realizing the training of the entity link model.

[0121] In one embodiment, constructing an input-output training pair based on sample data includes: for any sample word, combining the sample word and each sample candidate entity of the sample word to obtain a first sample text sequence; determining input data based on the first sample text sequence; determining an output standard according to the order corresponding to the linked entities of the sample word in the first sample text sequence in the input data; and constructing an input-output training pair based on the input data and the output standard.

[0122] Among them, the linked entity of the sample word is one of the entities in each sample candidate entity of the sample word. The order corresponding to the linked entity of the sample word can be used to characterize the position of the linked entity in the sample candidate entity ranking result. The output standard corresponding to the input data is determined according to the order of the linked entities in the first sample text sequence in the input data. The order corresponding to the linked entities of the sample word can be represented by a sequence identifier. For the linked entities of the same sample word, their positions in the sample candidate entity ranking result are different, and accordingly, their sequence identifiers will also be different, and the output standard will also change accordingly. For example, for a certain sample word, its linked entity is in the first place in the sample candidate entity ranking result, and the corresponding sequence identifier can be A; if its linked entity is in the third place in the sample candidate entity ranking result, the corresponding sequence identifier can be C.

[0123] In some embodiments, for any sample word, the computer device can combine the characteristics of the entity linking model, such as whether it is sensitive to order, etc., and process the sample candidate entities of the sample word according to the set sequential processing strategy, and combine the sample candidate entities of the sample word and the processed sample candidate entities to obtain a first sample text sequence. Among them, when the computer device processes the sample candidate entities of the sample word according to the sequential processing strategy, it can rank the linked entities among the sample candidate entities of the sample word in the first place, and the remaining sample candidate entities can be randomly sorted or sorted according to the size of similarity. The computer device can also directly sort the sample candidate entities according to the size of similarity to obtain the sorted sample candidate entities.

[0124] In other embodiments, the computer device may also, in accordance with a data augmentation strategy, disrupt the order of each sample candidate entity, or add noise, randomly insert, or delete sample candidate entities based on the sample candidate entities obtained by sorting according to the sequential processing strategy, to improve the generalization ability of the model. It is understandable that the computer device may choose to directly obtain the first sample text sequence according to the sequential processing strategy, or the computer device may choose to obtain the first sample text sequence using the data augmentation strategy. The specific method for obtaining the first sample text sequence may be determined in combination with the actual entity linking scenario, entity linking efficiency requirements, entity linking accuracy requirements, etc.

[0125] In some embodiments, the computer device can determine the input data based on the obtained first sample text sequence. For example, the computer device can directly determine the first sample text sequence as the input data, and then determine the output standard based on the order corresponding to the linked entities of the sample words in the first sample text sequence in the input data, that is, the order identifiers corresponding to the linked entities. The computer device can also choose to determine the first sample text sequence processed by the data enhancement strategy as the input data. If the position of the linked entity of the sample word changes, the corresponding output standard can also be updated accordingly. Of course, the computer device can also splice the first sample text sequence with other samples and determine the spliced text sequence as the input data. The embodiments of the present application are not limited here.

[0126] In the above embodiment, the computer device combines the sample words and the sample candidate entities of the sample words to obtain different first sample text sequences, and determines the input data based on the first sample text sequences, and determines the output standard in combination with the input data, thereby improving the robustness of the entity linking model prediction.

[0127] In one embodiment, the first sample text sequence includes the first sample text sequence before enhancement and the first sample text sequence after enhancement; combining the targeted sample words and the sample candidate entities of the targeted sample words to obtain the first sample text sequence, including: sorting the sample candidate entities of the targeted sample words, combining the targeted sample words and the sorted sample candidate entities, to obtain the first sample text sequence before enhancement; shuffling and rearranging the sorting of the sample candidate entities of the targeted sample words, combining the targeted sample words and the re-sorted sample candidate entities, to obtain the enhanced first sample text sequence.

[0128] Sorting the sample candidate entities for the sample word may refer to sorting the sample candidate entities according to their similarity to the sample word, or may refer to ranking the linked entities among the sample candidate entities first and sorting the other sample candidate entities according to their similarity to the sample word. Shuffling and reordering the sample candidate entities for the sample word refers to randomly sorting the sample candidate entities or changing the order of the sample candidate entities sorted by similarity.

[0129] Specifically, the first sample text sequence includes the first sample text sequence before enhancement and the first sample text sequence after enhancement. The computer device can construct an input-output training pair based only on the first sample text sequence before enhancement, or can construct an input-output training pair based only on the first sample text sequence after enhancement, or can construct an input-output training pair based on both the first sample text sequence before enhancement and the first sample text sequence after enhancement.

[0130] In some embodiments, when a computer device constructs an input-output training pair based on the first sample text sequence before enhancement and the first sample text sequence after enhancement, the output standard may also include a first output standard and a second output standard, the first output standard may correspond to the first sample text sequence, and the second output standard may correspond to the second output standard.

[0131] In the above embodiment, the computer device can obtain the first sample text sequence before enhancement and the first sample text sequence after enhancement by combining, sorting, shuffling and rearranging each sample candidate entity of the sample word, so that the model can learn the sequential identification and the correct answer, that is, the association between the linked entities during the training process, instead of just remembering the position of the correct answer, thereby enhancing the robustness of the model prediction.

[0132] In one embodiment, the entity linking method further includes: searching from a preset corpus based on the targeted sample word to obtain multiple sample similar instances matching the targeted sample word; wherein one instance contains a preset word, a candidate entity of the preset word, and a sequential identifier of the linked entity of the preset word; mapping each sample similar instance to a corresponding second sample text sequence; determining input data based on the first sample text sequence, including: constructing the input data of the targeted sample word based on the first sample text sequence and each second sample text sequence.

[0133] In some embodiments, the computer device may perform text mapping based on multiple sample similar instances that match the sample word to obtain each second sample text sequence. When performing text mapping, the computer device may use regular expressions to expand the mapping of multiple sample similar instances.

[0134] In some embodiments, when constructing input data, the computer device may construct the input data of the sample word based on the first sample text sequence before enhancement and each second sample text sequence; the computer device may also construct the input data of the sample word based on the first sample text sequence after enhancement and each second sample text sequence. The computer device may also construct the input data of the sample word based on the first sample text sequence before enhancement, the first sample text sequence after enhancement, and each second sample text sequence. It is understandable that the output standard corresponding to the input data is determined based on the order of the linked entities in the first sample text sequence in the input data. Therefore, when the first sample text sequence determining the input data changes, the output standard will also change accordingly.

[0135] The above is an embodiment. When the computer device constructs the input data for the sample word, it can determine based on the first sample text sequence and each second sample text sequence, and more comprehensively understand the interaction between different sample candidate entities and sample words, and then refer to similar sample similar instances from the entire preset corpus as prediction clues, thereby enhancing the generalization ability of long-tail entities.

[0136] In one embodiment, the candidate entities are determined by a retrieval model; the training steps of the retrieval model include: obtaining sample training pairs; the sample training pairs include positive sample pairs and negative sample pairs; the positive sample pairs include sample words and positive sample candidate entities of the sample words; the negative sample pairs include sample words and negative sample candidate entities of the sample words; the positive sample pairs and the negative sample pairs are input into the retrieval model to be trained, and the retrieval model to be trained calculates the positive sample similarity of the positive sample pairs and the negative sample similarity of the negative sample pairs; based on the positive sample similarity and the negative sample similarity, the training loss is determined, and the retrieval model to be trained is trained based on the training loss until the training is completed, and a trained retrieval model is obtained.

[0137] The sample training pairs are used for model training to achieve candidate entity retrieval. Positive candidate entities are standardized sample entities that match the sample terms. Negative candidate entities may include all sample entities excluding the standardized sample entities. Positive similarity is the calculated similarity between the sample terms and the positive candidate entities, while negative similarity is the calculated similarity between the sample terms and the negative candidate entities.

[0138] Specifically, the computer device can calculate a training loss based on the positive sample similarity and the negative sample similarity, and adjust the parameters of the retrieval model based on the calculated training loss. For example, the retrieval model hyperparameters and learning rate can be adjusted, and training can be continued based on the retrieval model after the parameters are adjusted until the training is completed, thereby obtaining a trained retrieval model.

[0139] In a specific application, a computer device can use contrastive learning to train a retrieval model. The optimization goal is to shorten the distance between each pair of positive samples and increase the distance between each pair of negative samples. The specific formula used is as follows:

[0140]

[0141] in, The calculation formula is as follows: Represents the similarity of positive samples, represents the negative sample similarity, is a temperature hyperparameter, The positive sample candidate entities are excluded The set of negative sample candidate entities other than .

[0142] In the above embodiment, when training the retrieval model, the computer device establishes positive sample pairs and negative sample pairs, thereby training the retrieval model through comparative learning, so that the retrieval model approaches the optimization goal of shortening the distance of the positive sample pairs and increasing the distance of the negative sample pairs during the training process, which can improve the accuracy of the subsequent retrieval model in determining candidate entities.

[0143] In one embodiment, the method for determining the negative sample candidate entity of the sample word includes: determining the positive sample candidate entity of other sample words except the sample word as the negative sample candidate entity of the sample word; or determining the sample candidate entity whose similarity with the positive sample candidate entity of the sample word meets the preset similarity condition but belongs to the wrong entity as the negative sample candidate entity of the sample word.

[0144] Specifically, for any sample word, when determining a negative candidate entity for the sample word, the computer device may determine a positive candidate entity that matches other sample words in the training sample as a negative candidate entity for the sample word. The computer device may also determine an entity that has a high degree of similarity to the sample word but is not a linked entity of the sample word as a negative candidate entity.

[0145] In some embodiments, if there is a sample candidate entity whose similarity with the sample word reaches a set similarity threshold, such as 90%, but the sample candidate entity is not a positive sample candidate entity that matches the sample word, it is determined to be a negative sample candidate entity of the sample word.

[0146] In the above embodiment, when training the retrieval model, the computer device establishes positive sample pairs and negative sample pairs, thereby training the retrieval model through comparative learning, so that the retrieval model approaches the optimization goal of shortening the distance of the positive sample pairs and increasing the distance of the negative sample pairs during the training process, thereby improving the accuracy of the subsequent model in determining candidate entities.

[0147] The present application also provides an application scenario, which applies the above-mentioned message processing method. Specifically, the entity linking method of the present application is described in detail below using an actual application scenario as an example, wherein the actual application scenario can be a biomedical scenario.

[0148] In the biomedical field, reference Figure 5 Figure 2 shows a schematic diagram of a linking page in a biomedical scenario. Knowledge graphs contain a vast amount of information describing entities such as diseases, genes, and drugs. Entity linking allows entities from different sources to be linked to a standardized knowledge graph, improving data consistency and comparability. For example, for hypertension, a user can enter information related to hypertension in a query box. The computer device responds to the query by linking the hypertension entity to the knowledge graph, leveraging its structure and relationships for precise query and reasoning, obtaining relevant descriptive information. This information can include medications, precautions, lifestyle habits, and test methods related to hypertension. The descriptive information obtained through entity linking helps researchers, physicians, and decision makers better understand and apply knowledge in the biomedical field. In summary, biomedical entity linking not only improves data consistency and comparability, enriches the content and semantics of knowledge graphs, but also enhances information retrieval and reasoning capabilities, promotes cross-disciplinary research and collaboration, advances research and application in the biomedical field, and improves the accuracy and efficiency of medical decision-making.

[0149] The overall architecture of the entity linking method proposed in this application is as follows Figure 6As shown. First, the retriever recalls the top-N most similar candidate entities for m from the entity set. For example, when the word to be processed is "ulcerated", the top-N candidate entities matching "ulcerated" can be: "ulcerated mass (ulcerous mass)", "ulcer lesion (ulcerous lesion)", "healed ulcer (healed ulcer)" and "mucosal ulcer (mucosal ulcer)". Furthermore, the generator can recall the top-K most similar similar instances for m from the preset corpus. When the word to be processed is "ulcerated", the preset words for similar instances may include "ulcers (ulcer)", "ulcer lesion", "erosion (erosion)", "superficial ulceration (superficial ulcer)", etc.

[0150] The computer device can combine the word to be processed, as well as multiple candidate entities and similar instances retrieved, into a retrieval-enhanced multiple-choice prompt. This is then processed by a generator to produce the final answer. For example, if the word to be processed is "ulcerated," its similar instances could be: "mention: ulcers; options: A.ulcer lesion; B.skin ulcer; C.lesion; D.ulcerated mass," and the answer: A.

[0151] The computer device can use SapBERT[7] as an encoder to generate corresponding vector representations for the mention and each candidate entity. The vector representation f(m) of the mention can be expressed as:

[0152]

[0153] Where [CLS] represents the hidden state of the special token in the last layer of SapBERT, which aims to obtain a fixed-size feature vector for each input. The vector representation f(e) of entity e is calculated in a similar way to word m. Therefore, a pair The similarity is expressed as follows:

[0154]

[0155] Among them, g represents cosine similarity.

[0156] After the retriever training is completed, in the process of actually determining the candidate entities, the computer device can obtain the feature vectors of the mention and all entities based on the vector representation formula, and then calculate the similarity between the mention and each entity based on the similarity calculation formula, and use the top N entities with the highest similarity as candidate entities for the word to be processed.

[0157] During the training process of the generator, the training goal of the generator is to maximize the likelihood of the following formula:

[0158]

[0159] Among them, m refers to the sample word, O refers to the sample candidate entity of the sample word, and a refers to the order identifier corresponding to the linked entity of the sample word. Because the generator is sensitive to the order of answer options, a slight change in the order will change the answer predicted by the model. In order to improve the robustness of the generator prediction, a simple but effective data augmentation strategy is adopted during the training process. By randomly swapping the order of the sample candidate entities in O, different Input-output training pairs, where swapping the order of options in O may change the symbol a corresponding to the correct answer, but the answer remains the same. Theoretically, each (m, O) can generate N! different examples. Experiments have shown that generating an additional training pair for each input (m, O) by performing an additional random swap is sufficient. This strategy allows the model to learn the association between symbols and answers, rather than simply memorizing the answer position.

[0160] In actual entity linking tasks, by constructing multiple-choice prompt text, the mention and its enumerated candidate entities are passed to the generator together. This allows for explicit modeling of mention-entity and entity-entity interactions. The mention and all entity information are fed into the generator as input, rather than just the mention and a single entity. Furthermore, the generator's input-output structure only requires the generator to predict a single symbol (such as A) rather than a specific entity name, thus avoiding the generation of invalid entities. (Due to the uncertainty of the generator, entities that do not exist in the ontology may be generated. Generating only a symbol associated with the answer greatly reduces the difficulty of prediction.)

[0161] In addition, for the generator, a kNN (K-Nearest Neighbors, K nearest neighbor algorithm) module can be introduced to enhance the generalization ability of the generator for long-tail entities, so that it can refer to similar instances from the entire preset corpus as prediction clues. in , which contains N candidate options and a symbol associated with the correct answer . Computer devices can define key-value pairs ,in It is calculated The feature vector of . Given an input mention, it is defined as , by outputting The eigenvector of ,according to and in the database Cosine similarity retrieval and The most similar top-K ( is a hyperparameter) nearest neighbors :

[0162]

[0163] During training, Already included in the database Therefore, the top-1 retrieved must be itself, so in order to avoid data leakage, the nearest neighbor can be removed middle itself.

[0164] Computer equipment can be used separately These examples Convert them to text sequences and then compare them to the input sequence Stitched together, Corresponding to This forms a retrieval-enhanced multiple-choice prompt text that is fed into the generator to generate the answer. The training objective of the generator now becomes the following:

[0165]

[0166] Among them, ⊕ is the concatenation operation and a is the answer symbol of (x, O). This can dynamically select some similar instances for each input x as reference clues for the generator to make predictions.

[0167] refer to Figure 7As shown, an evaluation diagram is provided to verify the accuracy of the entity linking method provided by this application. Specifically, experiments can be conducted on three public medical entity linking datasets. The results are shown in the figure below, where the evaluation indicator used is accuracy. It can be seen that the entity linking method provided by this application outperforms previous methods on all datasets, proving the effectiveness of our proposed method. Among them, models 1-12 can refer to BioSyn, ResCNN (residual convolutional neural network), SapBERT (Transformer-based pre-trained language model), Clustering-based, Prompt-BioEL, GenBioEL, BioBART-base, BioBART-large, GPT3.5, PaLM-2, Claude-2, LLaMA-2-13b, respectively. Among them, for Prompt-BioEL, an additional synonym corpus is required for further pre-training, while the entity linking method provided by this application can only use existing training data without the need for an additional corpus. In addition, Claude-2 outperforms other LLM-based methods (i.e., GPT3.5, PaLM-2, LLaMA-2-13b, etc.) in all datasets, but still lags significantly behind other supervised methods. This suggests that fine-tuning small-scale models is still the best choice for medical entity linking tasks because they can acquire domain-specific knowledge through parameter adjustment.

[0168] refer to Figure 8 As shown, the words to be processed include "feel uncomfortable (feel uncomfortable)", "blurred vision (blurred vision)", and similar instances corresponding to feel uncomfortable may include "feltuncomfortable (feel uncomfortable)", "uneasy (feel uneasy or uncomfortable)", and "uncom-fortable (uncomfortable)"; similar instances corresponding to "blurred vision" include "blurry vision (blurred instance)", "mistyvision (misty vision)", "blurry (blurry)" and "blurring of visual image (blurring of visual image)". Figure 8 In Figure 1, two long-tail entities, "uneasy" and "misty vision," are shown to illustrate the usefulness of the KNN module. Without the KNN module, the model tends to map mentions to entities that are similar but incorrect in meaning. However, incorporating the KNN module allows the model to leverage relevant instances in the training set, making more informed decisions by referencing these instances.

[0169] It is understood that the above-mentioned scenario embodiments are used to illustrate the entity linking method provided by this application and are not intended to limit the application scenarios of this application. The entity linking method of this application can be applied to many fields that need to associate entities in text with external knowledge bases or data sources, thereby helping to improve the intelligent level of information retrieval, analysis, and processing. For example, in various fields such as law, intelligent customer service, and news media.

[0170] It should be understood that, although the various steps in the flowcharts involved in the various embodiments described above are displayed in sequence according to the instructions of the arrows, these steps are not necessarily executed in sequence in the order indicated by the arrows. Unless otherwise specified herein, there is no strict order restriction on the execution of these steps, and these steps can be executed in other orders. Moreover, at least a portion of the steps in the flowcharts involved in the various embodiments described above can include multiple steps or multiple stages, and these steps or stages are not necessarily executed and completed at the same time, but can be executed at different times, and the execution order of these steps or stages is not necessarily to be carried out in sequence, but can be executed in turn or alternately with other steps or at least a portion of steps or stages in other steps.

[0171] Based on the same inventive concept, the present application also provides an entity linking device for implementing the entity linking method described above. The solution provided by this device is similar to the solution described in the method described above. Therefore, the specific limitations of one or more entity linking device embodiments provided below can be found in the above-mentioned limitations of the entity linking method and will not be repeated here.

[0172] In one embodiment, Figure 9 As shown, an entity linking device is provided, including: a data acquisition module 902, a retrieval module 904, a text construction module 906 and a linking module 908, wherein:

[0173] The data acquisition module 902 is used to acquire the words to be processed and determine an entity set; the entity set includes multiple standardized entities.

[0174] The retrieval module 904 is configured to search the entity set based on the word to be processed to obtain a plurality of candidate entities that match the word to be processed.

[0175] The text construction module 906 is used to construct a multiple-choice prompt text based on the word to be processed and multiple candidate entities.

[0176] The linking module 908 is configured to process the multiple-choice prompt text to output a target identifier, and use the candidate entity indicated by the target identifier as a linking entity of the word to be processed.

[0177] In some embodiments, the retrieval module 904 is also used to encode based on the word to be processed to obtain a first vector representation of the word to be processed; encode based on each standardized entity to obtain a second vector representation corresponding to each standardized entity; calculate the first similarity between the first vector representation and each second vector representation; and determine the standardized entities corresponding to the first similarity that meets the first preset similarity condition as multiple candidate entities matching the word to be processed.

[0178] In some embodiments, the text construction module 906 is also used to sort multiple candidate entities according to the similarity between the candidate entities and the word to be processed; determine the sequence identifier corresponding to the candidate entity according to the sorting order of the candidate entities; combine the word to be processed, multiple candidate entities, and the sequence identifier corresponding to each candidate entity according to a preset format to obtain a first text sequence; and determine a multiple-choice prompt text based on the first text sequence.

[0179] In some embodiments, the text construction module 906 is also used to search from a preset corpus based on the word to be processed to obtain multiple similar instances matching the word to be processed; wherein an instance contains a preset word, a candidate entity of the preset word, and a sequential identifier of a linked entity of the preset word; each similar instance is mapped to a corresponding second text sequence; and is also used to construct a multiple-choice prompt text based on the first text sequence and each second text sequence.

[0180] In some embodiments, the retrieval module 904 is further configured to encode the word to be processed to obtain a first vector representation of the word to be processed; calculate a second similarity between the first vector representation and the vector representations of each preset word in a preset corpus; select preset words corresponding to the second similarity that meet the second preset similarity condition, and use the instances associated with the selected preset words as similar instances that match the word to be processed. In some embodiments, the linking module 908 is further configured to input the multiple-choice prompt text into an entity linking model; perform identifier prediction on the multiple-choice prompt text using the entity linking model to obtain a target identifier, and use the candidate entity associated with the target identifier as a link entity for the word to be processed.

[0181] In some embodiments, the entity linking method also includes a training module; the training module is used to obtain sample data; the sample data includes sample words and sample candidate entities of the sample words; an input-output training pair is constructed based on the sample data; the input-output training pair includes input data and output standards, the input data includes sample words and sample candidate entities of the sample words, and the output standard includes the sequential identifier corresponding to the linked entity of the sample word; the input data is input into the entity linking model to be trained to obtain the predicted identifier predicted by the entity linking model to be trained; based on the difference between the predicted identifier and the output standard, the parameters of the entity linking model to be trained are adjusted and the training is continued until the training is completed to obtain a trained entity linking model.

[0182] In some embodiments, the training module is further used to combine the sample word and each sample candidate entity of the sample word for any sample word to obtain a first sample text sequence; determine input data based on the first sample text sequence; determine output standards according to the order corresponding to the linked entities of the sample word in the first sample text sequence in the input data; and construct input and output training pairs based on the input data and the output standards.

[0183] In some embodiments, the first sample text sequence includes the first sample text sequence before enhancement and the first sample text sequence after enhancement; the training module is further used to sort the sample candidate entities of the sample words, combine the sample words and the sorted sample candidate entities, and obtain the first sample text sequence before enhancement; shuffle and rearrange the sorting of the sample candidate entities of the sample words, combine the sample words and the re-sorted sample candidate entities, and obtain the first sample text sequence after enhancement.

[0184] In some embodiments, the training module is further used to retrieve from a preset corpus based on the targeted sample word to obtain multiple sample similar instances that match the targeted sample word; wherein an instance contains a preset word, a candidate entity of the preset word, and a sequential identifier of a linked entity of the preset word; each sample similar instance is mapped to a corresponding second sample text sequence; and is also used to construct input data for the targeted sample word based on the first sample text sequence and each second sample text sequence.

[0185] In some embodiments, the training module is also used to obtain sample training pairs; the sample training pairs include positive sample pairs and negative sample pairs; the positive sample pairs include sample words and positive sample candidate entities of the sample words; the negative sample pairs include sample words and negative sample candidate entities of the sample words; the positive sample pairs and the negative sample pairs are input into the retrieval model to be trained, and the retrieval model to be trained calculates the positive sample similarity of the positive sample pairs and the negative sample similarity of the negative sample pairs; based on the positive sample similarity and the negative sample similarity, the training loss is determined, and the retrieval model to be trained is trained based on the training loss until the training is completed, and a trained retrieval model is obtained.

[0186] In some embodiments, the training module is also used to determine the positive sample candidate entities of sample words other than the sample word as negative sample candidate entities of the sample word; or, to determine the sample candidate entities whose similarity with the sample word meets the preset similarity condition but belongs to the wrong entity as the negative sample candidate entity of the sample word.

[0187] Each module in the above-mentioned physical link device can be implemented in whole or in part through software, hardware, or a combination thereof. Each module can be embedded in or independent of the processor in the computer device in hardware form, or can be stored in the memory of the computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0188] In one embodiment, a computer device is provided. The computer device may be a server or a terminal. The internal structure diagram thereof may be as follows: Figure 10 As shown. The computer device includes a processor, a memory, an input / output interface (Input / Output, abbreviated as I / O) and a communication interface. The processor, memory and input / output interface are connected through a system bus, and the communication interface is connected to the system bus through the input / output interface. The processor of the computer device is used to provide computing and control capabilities. The memory of the computer device includes a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system, a computer program and a database. The internal memory provides an environment for the operation of the operating system and computer program in the non-volatile storage medium. The database of the computer device is used to store entity sets. The input / output interface of the computer device is used to exchange information between the processor and an external device. The communication interface of the computer device is used to communicate with an external terminal through a network connection. When the computer program is executed by the processor, an entity linking method is implemented.

[0189] Those skilled in the art will understand that Figure 10The structure shown in the figure is only a block diagram of a part of the structure related to the solution of the present application, and does not constitute a limitation on the computer device to which the solution of the present application is applied. The specific computer device may include more or fewer components than shown in the figure, or combine certain components, or have a different component arrangement.

[0190] In one embodiment, a computer device is provided, including a memory and a processor, wherein a computer program is stored in the memory, and the processor implements the steps of the above-mentioned entity linking method when executing the computer program.

[0191] In one embodiment, a computer-readable storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the above-mentioned entity linking method are implemented.

[0192] In one embodiment, a computer program product is provided, comprising a computer program, which implements the steps of the above-mentioned entity linking method when executed by a processor.

[0193] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the relevant laws, regulations and standards of relevant countries and regions.

[0194] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM). The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may be, but are not limited to, general-purpose processors, central processing units (CPUs), graphics processing units (GPUs), digital signal processors (DSPs), programmable logic devices (PLDs), data processing logic devices based on quantum computing, and the like.

[0195] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0196] The above-described embodiments merely represent several implementation methods of the present application. While the descriptions are relatively specific and detailed, they should not be construed as limiting the scope of the present application. It should be noted that a person of ordinary skill in the art may make various modifications and improvements without departing from the spirit of the present application, and these modifications and improvements fall within the scope of protection of the present application. Therefore, the scope of protection of the present application shall be determined by the appended claims.

Claims

1. An entity linking method, characterized in that: The method comprises: Obtaining a word to be processed and determining an entity set; the entity set includes multiple standardized entities; Searching the entity set based on the word to be processed to obtain multiple candidate entities matching the word to be processed; Constructing a multiple-choice prompt text based on the word to be processed and the multiple candidate entities; The target identifier is outputted according to the multiple-choice prompt text, and the candidate entity indicated by the target identifier is used as the link entity of the word to be processed.

2. The method according to claim 1, characterized in that The searching in the entity set based on the word to be processed to obtain multiple candidate entities matching the word to be processed includes: Encoding the word to be processed to obtain a first vector representation of the word to be processed; Encoding is performed based on each of the standardized entities to obtain a second vector representation corresponding to each of the standardized entities; Calculating first similarities between the first vector representation and each of the second vector representations; The standardized entities corresponding to the first similarity that meets the first preset similarity condition are determined as multiple candidate entities that match the word to be processed.

3. The method according to claim 1, characterized in that The step of constructing a multiple-choice prompt text based on the word to be processed and the multiple candidate entities includes: Sorting the plurality of candidate entities according to the similarity between the candidate entities and the word to be processed; Determine the sequence identifier corresponding to the candidate entity according to the sorting order of the candidate entity; Combining the word to be processed, the multiple candidate entities, and the sequence identifier corresponding to each candidate entity according to a preset format to obtain a first text sequence; A multiple-selection prompt text is determined based on the first text sequence.

4. The method according to claim 3, characterized in that: The method further comprises: Based on the word to be processed, a search is performed from a preset corpus to obtain multiple similar instances matching the word to be processed; wherein one instance contains a preset word, a candidate entity of the preset word, and a sequential identifier of a linked entity of the preset word; Mapping each similar instance to a corresponding second text sequence; The determining of the multiple-selection prompt text based on the first text sequence includes: Based on the first text sequence and each second text sequence, a multiple-choice prompt text is constructed.

5. The method according to claim 4, characterized in that The searching from a preset corpus based on the word to be processed to obtain multiple similar instances matching the word to be processed includes: Encoding the word to be processed to obtain a first vector representation of the word to be processed; Calculating a second similarity between the first vector representation and a vector representation of each preset word in a preset corpus; Preset words corresponding to the second similarity that meet the second preset similarity condition are screened, and instances associated with the screened preset words are used as similar instances that match the word to be processed.

6. The method according to any one of claims 1 to 5, characterized in that The step of processing the multiple-choice prompt text to output a target identifier, and using the candidate entity indicated by the target identifier as a link entity of the word to be processed includes: Inputting the multiple-choice prompt text into an entity linking model; Through the entity linking model, identifier prediction is performed on the multiple-choice prompt text to obtain a target identifier, and the candidate entity associated with the target identifier is used as the link entity of the word to be processed.

7. The method according to claim 6, characterized in that The training steps of the entity linking model include: Acquire sample data; the sample data includes sample words and sample candidate entities of the sample words; constructing an input-output training pair based on the sample data; the input-output training pair includes input data and output criteria, the input data includes sample words and sample candidate entities of the sample words, and the output criteria includes sequence identifiers corresponding to the linked entities of the sample words; Inputting the input data into the entity linking model to be trained to obtain a prediction identifier predicted by the entity linking model to be trained; Based on the difference between the predicted identifier and the output standard, the parameters of the entity linking model to be trained are adjusted and the training is continued until the training is completed to obtain a trained entity linking model.

8. The method according to claim 7, characterized in that The constructing of input-output training pairs based on the sample data includes: For any sample word, combining the sample word and each sample candidate entity of the sample word to obtain a first sample text sequence; determining input data based on the first sample text sequence; Determining an output standard according to the order of the linked entities corresponding to the sample words in the first sample text sequence in the input data; An input-output training pair is constructed based on the input data and the output standard.

9. The method according to claim 8, characterized in that The first sample text sequence includes a first sample text sequence before enhancement and a first sample text sequence after enhancement; The step of combining the sample word and each sample candidate entity of the sample word to obtain a first sample text sequence includes: Sorting the sample candidate entities of the targeted sample words, combining the targeted sample words and the sorted sample candidate entities to obtain a first sample text sequence before enhancement; The order of each sample candidate entity of the targeted sample word is disrupted and rearranged, and the targeted sample word and the rearranged sample candidate entity are combined to obtain an enhanced first sample text sequence.

10. The method according to claim 8, characterized in that The method further comprises: Based on the targeted sample word, a search is performed from a preset corpus to obtain multiple sample similar instances matching the targeted sample word; wherein one instance contains a preset word, a candidate entity of the preset word, and a sequential identifier of a linked entity of the preset word; Mapping each sample similar instance to a corresponding second sample text sequence; The determining of input data based on the first sample text sequence includes: Based on the first sample text sequence and each second sample text sequence, input data for the sample word is constructed.

11. The method according to any one of claims 1 to 5, wherein the candidate entity is determined by a retrieval model; the training step of the retrieval model comprises: Obtaining sample training pairs; the sample training pairs include positive sample pairs and negative sample pairs; the positive sample pairs include sample words and positive sample candidate entities of the sample words; the negative sample pairs include sample words and negative sample candidate entities of the sample words; Inputting the positive sample pairs and the negative sample pairs into the retrieval model to be trained, and calculating the positive sample similarity of the positive sample pairs and the negative sample similarity of the negative sample pairs by the retrieval model to be trained; Based on the positive sample similarity and the negative sample similarity, a training loss is determined, and the retrieval model to be trained is trained based on the training loss until the training is completed, thereby obtaining a trained retrieval model.

12. The method according to claim 11, characterized in that The method for determining the negative sample candidate entity of the sample word includes: Determine the positive sample candidate entities of other sample words except the sample word as the negative sample candidate entities of the sample word; or, A sample candidate entity whose similarity to the sample word meets a preset similarity condition but is an erroneous entity is determined as a negative sample candidate entity of the sample word.

13. A physical linking device, characterized in that: The device comprises: A data acquisition module is used to acquire words to be processed and determine an entity set; the entity set includes multiple standardized entities; A retrieval module, configured to search the entity set based on the word to be processed to obtain a plurality of candidate entities matching the word to be processed; A text construction module, configured to construct a multiple-choice prompt text based on the word to be processed and the multiple candidate entities; The linking module is used to process the multiple-choice prompt text to output a target identifier, and use the candidate entity indicated by the target identifier as a linking entity of the word to be processed.

14. A computer device comprising a memory and a processor, wherein the memory stores a computer program, wherein: When the processor executes the computer program, the steps of the method according to any one of claims 1 to 12 are implemented.

15. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

16. A computer program product comprising a computer program, characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 12 are implemented.

Images