An entity linking method based on prompt scoring
By using a prompt-based scoring method, combined with sparse and dense vectors and contextual information, the shortcomings of existing entity linking models in specific domains are addressed, achieving more efficient and accurate entity linking and reducing reliance on external knowledge.
Patent Information
- Application Number
- CN202311233705.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-09-22
- Publication Date
- 2025-12-19
- Estimated Expiration
- 2043-09-22
AI Technical Summary
Existing entity linking models lack comprehensive entity description information in specific domains, underutilize contextual information, and are computationally intensive, relying on external knowledge supplementation.
We adopt a prompt-based scoring method, which involves data cleaning, candidate entity discovery by combining sparse and dense vectors, constructing prompt sentences by combining contextual information, and using the BERT model for attention mechanism processing. Finally, we obtain the final entity link score through a classifier.
It improves the accuracy of entity linking and the generalization ability of the model, reduces the dependence on external knowledge, reduces computational complexity, and improves the performance of the model in different domains.
Smart Images
Figure CN117272107B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to an entity linking method based on prompt scoring, and belongs to the technical field of language processing. BACKGROUND
[0002] The entity linking method mainly adopts a two-stage processing strategy. Mihalcea et al. first proposed this strategy, which effectively balances the efficiency and quality of entity linking through two key stages of candidate entity discovery and sorting. The candidate entity discovery stage needs to construct a candidate entity list, which includes methods based on text similarity, such as the work of Gangemi A et al., Zhang Y et al., and S. Zwicklbauer et al. They match various mention words in the text to establish a candidate entity list. On this basis, there are methods that use alias expansion to overcome errors caused by abbreviations and the like. For example, M. Pershina et al. use coreference resolution technology to expand the given mention word into the longest mention word in the text, and then match it with the entities in the knowledge base to form candidate entities. In addition, there are methods based on the prior probability of the corresponding relationship between the mention word and the entity, such as the research of V.I. Spitkovsky et al., which greatly reduces the computational complexity of the entity linking task by capturing the entity linking frequency of the mention word in the web data, and ensures that the candidate list contains correct linked entities by widely discovering candidate entities. The BioSyn model proposed by Sung M et al. combines statistical features and pre-trained language model features to represent entities, thereby increasing the entity linking performance.
[0003] To further improve the accuracy of entity linking, the candidate entity ranking stage is particularly important. Early methods mainly use rule-based, feature-based and traditional machine learning methods. For example, Mihalcea et al. use feature engineering methods to sort and disambiguate candidate entities by semantic similarity and contextual information. Hoffart et al. consider the information of candidate entities and entities in external knowledge base, and use rules and features for entity linking and disambiguation. In recent years, neural network-based entity linking methods have achieved good results in improving the ranking results of candidate entities. Zhang Y et al. proposed an entity linking method based on mention-entity joint learning, which learns the embedding representation between entities and context through multi-layer neural network, and performs entity linking through ranking algorithm. In addition, pre-trained language models such as BERT and RoBERTa also play an important role in entity linking tasks. Devlin et al. learned the representation between entities and context by pre-training the BERT model, which greatly improved the linking performance. Sung M et al. proposed to encode entities and mentions into the same dense space, and realize linking through embedding similarity. Lai T et al. integrated synonym knowledge into the similarity method through contrastive learning, further improving the performance of entity linking.
[0004] To better represent entities, some researchers have introduced new perspectives. One is to use knowledge enhancement strategies, including using neural networks to integrate more features to enrich the semantic expression of entities [9] , and introducing double semantic representation. Another is the graph-based method, which improves the semantic expression ability of entity vectors through graph learning. For example, Vretinaris A et al. use graph convolutional neural network to realize biomedical entity linking, and Fang Z et al. propose a method based on graph attention network. Li B Z et al. integrate entity recognition and linking, and propose a joint method. However, the existing entity linking model still has some shortcomings: 1 For specific domain entity linking tasks, the existing knowledge base cannot provide complete entity description information; 2 The context information has a certain influence on the results of entity linking, but the existing model is insufficient in utilizing the context. 3 The existing method usually needs external knowledge supplement, and the overall computational complexity of the model is large. SUMMARY
[0005] In view of the above problems, the present application proposes an entity linking method based on prompt scoring, which can improve the linking performance of the entity linking model.
[0006] The technical scheme of the present application is: an entity linking method based on prompt scoring, comprising: data cleaning of mentions to eliminate word abbreviations and spelling errors; vectorizing entities E in the knowledge base and mentions M in the text, and performing inner product calculation and obtaining scores The top 20 entities in the knowledge base are selected as candidate entities; the candidate entities are constructed into prompt sentences with the mention words, and the context information is spliced as model input into the Bert model after the attention mechanism, to obtain the final vector representation h of the token in the input sequence i ∈R h , which contains the interaction information between the token and other position tokens; the position label <cls>Vector h at point i ∈R h The data is fed into a classifier model to obtain the positive or negative probability of the prompt, and the probability p of predicting it as positive is obtained as the probability score. Finally and The final ranking score S is obtained by summing the results after adjusting the hyperparameter α. m,e , fraction A m,e The highest-ranking entity is the final linking result.
[0007] Furthermore, the specific steps of the entity linking method based on prompt scoring are as follows:
[0008] Step 1: Data Preprocessing: Spell checking was performed to correct spelling errors in mentions; Ab3P abbreviation parsing was also used; for compound mentions, they were broken down into individual mentions; for example, "breast cancer and ovarian cancer" was broken down into "breast cancer" and "ovarian cancer".
[0009] Step 2, Mention Word and Entity Vectorization: In the candidate entity discovery stage, the similarity between mention words and entities is calculated by combining TF-IDF sparse representations and dense representations based on pre-trained language models; let the sparse representations of mention word M and a certain entity E be respectively... and Dense characterization is respectively and
[0010] Step 3: Obtain candidate entities and their text similarity scores: Calculate sparse vector representations. and inner product and dense characterization and inner product Then, the two inner product values are summed to obtain the first-stage score. The top 20 entities and their scores, ranked from highest to lowest, are then selected as candidate entities.
[0011]
[0012]
[0013]
[0014] Where λ is the weight used to adjust the ratio of sparse representation similarity to dense representation similarity;
[0015] Step4, constructing a classification model input: using the candidate entities obtained from step Step3 to construct a prompt sentence and splice context information as a Bert model input; input the sequence into the bert model to achieve the purpose of vectorization, and through the self-attention mechanism, the information of the input sequence is fused, and after multiple rounds of attention mechanism processing, the final vector representation h of the token in the input sequence is obtained i ∈R h ;
[0016] Step5, obtaining classification scores: extracting the vector h i ∈R h of the position label <cls>the vector h c ∈R h , h c is sent into the classifier to get the class probability p, and the classifier model is trained using the cross-entropy loss function, and after the prompt classifier training is completed, the class probability p is used as the similarity score of the second stage
[0017] Step 6, the first stage score in step Step 3 and the second stage score in step Step 5 are weighted and summed to determine the final ranking, and the entities linked to the mention M are determined according to the ranking of the score.
[0018] Further, the step Step 4 includes the following contents:
[0019] The mention M in the text and the candidate entity E in the entity library are combined to construct a question sentence form using the prompt, and the specific form is:
[0020] prompt: is the mention M similar to the entity E?
[0021] Then the mention M, the context context_left and context_right of the mention, and the constructed prompt are combined into a complete input sequence; special characters <cls>to indicate the beginning of the input sequence, a special character <sep>to separate the sequence so that the three parts of information are spliced together; the input sequence is in the form:
[0022] input: <cls>context left <sep> M <sep>context right <sep>prompt.
[0023] Further, the step Step4 further includes the following content:
[0024] Each token of the input sequence input is converted into its corresponding word vector representation e i ∈R h by embedding layer, so that the entire input sequence is represented as a word vector sequence:
[0025] {e1,e2,e3,……,e n}=Embedding(input)
[0026] The relative position information of different tokens in the input sequence is indicative of the sequence, so each token of the input sequence input is converted into its corresponding position embedding p i ∈R h by position embedding, so that the entire input sequence is represented as a position embedding vector sequence:
[0027] {p1,p,p3,……,p n}=Position_Embedding(input)
[0028] For each token, a comprehensive vector f i ∈R i is obtained by performing addition operation on the word vector e i and the position embedding vector p h :
[0029] f i =e i +p i
[0030] In order to capture more context information, the pre-training model uses a multi-layer attention mechanism to perform multiple iterations on the entire feature vector sequence; first, the feature vector f i is multiplied by a randomly initialized matrix to calculate three new vectors Q-query, K-key, and V-value, and then attention operation is performed:
[0031]
[0032] Q=f i ·w Q
[0033] K=f i ·w K
[0034] V=f i • w V
[0035] where d k denotes the dimension of Q and K vectors, and Softmax is a normalized exponential function.
[0036] After multiple rounds of attention mechanism processing, the final vector representation h i ∈R h of the token in the input sequence is obtained, which contains the interaction information of the token with other position tokens.
[0037] Further, the step Step5 includes the following contents:
[0038] Step5.1, constructing training samples for training the prompt classifier; all entities with the same CUI in the entity library corresponding to a mention word are taken as positive samples of the training set, and an equal number of other entities are sampled as negative samples; for the positive samples, the classification label of the prompt input sequence constructed by a mention word and its corresponding standard entity is assigned as 1; for the prompt input sequence constructed by the negative sample, the classification label is assigned as 0
[0039] Step5.2, the classification label of the input sequence is determined according to the following rules: <cls>The feature vector h corresponding to the position c ∈R h is mapped to a class probability p by feeding into a fully connected layer and a sigmoid activation function:
[0040] p=Sigmoid(W1*h c +b)
[0041] where W1∈R 1*h and b are parameters to be trained.
[0042] The cross-entropy loss is used to train the classification model, and its formula is as follows:
[0043]
[0044] where N is the number of training samples, y i is the true label of the i-th sample, and p i is the probability predicted by the model for the i-th sample.
[0045] After the prompt classifier training is completed, the mention word M and the candidate entity E are fed into the model to obtain the class probability p, which is the similarity score of the second stage:
[0046] Further, the step Step6 includes the following contents:
[0047] The two-stage scores calculated in the above steps are summed up as the basis for subsequent ranking. Specifically, the scores of the 20 candidate entities in the candidate entity discovery stage and the scores based on the prompt classifier are weighted and summed up to determine the final ranking.
[0048]
[0049] Where alpha is an adjustable hyperparameter, and the size of alpha is set to control the proportion of the second-stage score.
[0050] Finally, the entity linked to the mention word M is determined according to the score of S m,e .
[0051] The beneficial effects of the present application are:
[0052] 1、Utilize sparse vector and dense vector respective similarity score screening candidate entity, and its similarity score, through constructing prompt sentence form, convert entity linking task into binary classification task, and through fusing context information, enhance semantic information, make that classification task accuracy is improved, and, fusing candidate entity discovery stage text similarity score and classification score carry out score sorting, avoid to a certain extent the dependence on external knowledge, and improve the generalization ability of classification model in different fields, model contribution summary as follows: (1) propose a prompt scoring based entity linking method; (2) model only relies on internal information of data set, reduces the complexity of model and generalization ability.The present application carries out experiment on medical field data set NCBI, BC5CDR, electric power field data set respectively, and the experimental results show the effectiveness of the proposed classification model.
[0053] 2、The present application utilizes the combination of sparse vectors and dense vectors in the candidate discovery phase to obtain text similarity scores and candidate entities, ensuring the accuracy of candidate discovery. In obtaining classification scores, the mention and entity are linked using a prompt sentence, and context information is fused, to a certain extent, solving the problem of information isolation coding in the previous double-tower structure model and the problem of mismatch between the two feature information. The method of the present application can improve the linking performance of the entity linking model to a certain extent. BRIEF DESCRIPTION OF DRAWINGS
[0054] Figure 1 A flowchart of the prompt scoring based entity linking method of the present application;
[0055] Figure 2 A result graph of the prompt scoring based entity linking method of the present application. DETAILED DESCRIPTION
[0056] Example 1: This example is carried out on medical disease entity linking (NCBI) data set, (BC5CDR) data set; as Figure 1 The flowchart of the prompt scoring based entity linking method of the present application is shown in the figure, a prompt scoring based entity linking method, the specific steps of the method are as follows:
[0057] Step 1, spelling check is performed to solve the spelling error problem in the mention word; at the same time, Ab3P abbreviation analysis is used; for composite mention words, the composite mention words are split into individual mention words;
[0058] The medical disease entity linking (NCBI) dataset is used in this embodiment, containing 792 documents. The dataset provides manually annotated disease mentions in each document, where each CUI is mapped to the MEDIC dictionary. The dataset contains 6881 mentions, of which we extract 100 documents as the validation set, and 100 as the test set. We also use the BC5CDR dataset, which contains 1500 documents and 12750 mentions. The documents are evenly distributed as the training set, validation set, and test set.
[0059] Table 1 is the statistical information of the entity linking dataset
[0060]
[0061] Step 2, in the candidate entity discovery stage, the TF-IDF sparse representation and the dense representation based on the pre-trained language model are combined to calculate the similarity of the mention words and the entity. Let the sparse representation of the mention word M and a certain entity E be and The dense representation is and
[0062] Step 3, calculate the inner product of the sparse vector representation and The inner product of the dense representation and and the inner product of the dense representation and The two inner product values are added to obtain the first stage score and and the scores are arranged from high to low to obtain the top 20 entities and their scores as candidate entities;
[0063]
[0064]
[0065]
[0066] Where λ is the weight used to adjust the proportion of sparse representation similarity and dense representation similarity;
[0067] Step 4, use the candidate entities obtained from Step 3 to construct a prompt sentence and splice the context information as the input of the Bert model. The input sequence is sent into the bert model to achieve the purpose of vectorization, and through the self-attention mechanism, the information of the input sequence is fused. After multiple rounds of attention mechanism processing, the final vector representation h of the token in the input sequence is obtained i ∈R h ;
[0068] Further, the step Step4 includes the following contents:
[0069] The prompt is used to combine the mention M in the text and the candidate entity E in the entity library to construct a question sentence, and the specific form of the prompt sentence is:
[0070] Prompt: is the mention M similar to the entity E?
[0071] Then, the mention M, the context (context_left, context_right) of the mention, and the constructed prompt are combined into a complete input sequence; special characters <cls>to indicate the beginning of the input sequence, a special character <sep>to indicate the separation of the sequences so that the three parts of information are spliced together; the input sequence is in the form as follows: input: <cls>context left <sep> M <sep>context right <sep>prompt.
[0072] Further, the step Step4 further includes the following content:
[0073] Each token of the input sequence input is converted into its corresponding word vector representation e i ∈R h by embedding layer, so that the entire input sequence is represented as a word vector sequence:
[0074] {e1,e2,e3,……,e n}=Embedding(input)
[0075] The relative position information of different tokens in the input sequence is indicative of the sequence, so each token of the input sequence input is converted into its corresponding position embedding p i ∈R h by position embedding, so that the entire input sequence is represented as a position embedding vector sequence:
[0076] {p1,p,p3,……,p n}=Position_Embedding(input)
[0077] For each token, a comprehensive vector f i ∈R i is obtained by performing addition operation on the word vector e i and the position embedding vector p h :
[0078] f i =e i +p i
[0079] In order to capture more context information, the pre-training model uses a multi-layer attention mechanism to perform multiple iterations on the entire feature vector sequence; first, the feature vector f i is multiplied by a randomly initialized matrix to calculate three new vectors Q-query, K-key, and V-value, and then attention operation is performed:
[0080]
[0081] Q=f i ·w Q
[0082] K=f i ·w K
[0083] V=f i • w V
[0084] where d k denotes the dimension of Q and K vectors, and Softmax is a normalized exponential function.
[0085] After multiple rounds of attention mechanism processing, the final vector representation h i ∈R h of the token in the input sequence is obtained, which contains the interaction information between the token and other position tokens.
[0086] Step 5, extract the vector h i ∈R h of the position label <cls>Vector h c ∈R h , h c is sent into the classifier to obtain the class probability p, and the classifier model is trained using the cross-entropy loss function. After the prompt classifier training is completed, the class probability p is used as the similarity score in the second stage
[0087] Further, the step Step5 includes the following contents:
[0088] Step5.1, constructing a training sample for training the prompt classifier; all entities with the same CUI in the entity library corresponding to a mention word are taken as positive samples of the training set, and an equal number of other entities are sampled as negative samples; for the positive samples, the prompt input sequence constructed by a mention word and its corresponding standard entity, the classification label is assigned as 1; for the prompt input sequence constructed by the negative sample, the classification label is assigned as 0
[0089] Step5.2, the input sequence in <cls>The feature vector h corresponding to the position c ∈R h is mapped to a class probability p by feeding into a fully connected layer and a sigmoid activation function:
[0090] p=Sigmoid(W1*h c +b)
[0091] where W1∈R 1*h and b are parameters to be trained.
[0092] The cross-entropy loss is used to train the classification model, and its formula is as follows:
[0093]
[0094] where N is the number of training samples, y i is the true label of the i-th sample, and p i is the probability predicted by the model for the i-th sample.
[0095] After the prompt classifier training is completed, the mention word M and the candidate entity E are fed into the model to obtain the class probability p, which is the similarity score of the second stage:
[0096] Step 6, the first stage score in step Step 3 and the second stage score in step Step 5 are weighted and summed to determine the final ranking, and the entity linked to the mention word M is determined according to the ranking of the score.
[0097] Specifically, the score of the 20 candidate entities in the candidate entity discovery stage and the score based on the prompt classifier are weighted and summed to determine the final ranking.
[0098]
[0099] where alpha is an adjustable hyperparameter, and the proportion of the second stage score is controlled by setting the size of alpha.
[0100] Finally, the entity linked to the mention word M is determined according to the value of S m,e .
[0101] To verify the effectiveness of the model proposed in the present application, the present application selects the following representative systems related to the present application as baseline models from the aspect of deep learning.
[0102] The application adopts Acc@k value as an evaluation index to measure the model performance: the application uses Acc@k to evaluate the model. If the correct CUI is included in the top k predictions of the model, it is determined to be correct prediction. If the prediction of each mention in the composite mention is correct, we consider the prediction of the composite entity to be correct.
[0103]
[0104] In the formula represents the number of correct link entities in the top k candidate entities, and K represents the total number of top k predicted entities.
[0105] Figure 2 The result of adjusting the weight α of the prompt scoring-based entity linking method proposed in the application. The adjustment of the proportion of two-stage scores shows that adjusting the weight within a certain range will promote the model.
[0106] DNorm(Leaman, Islamaj and Lu 2013) is a conditional random field (CRF) model that combines statistical features and knowledge resources.
[0107] Sieve-based Model(D'Souza and Ng,2015) is a sieve-based step-by-step processing model for entity recognition and relationship extraction tasks. It gradually improves the accuracy of information extraction by applying a series of sieves step by step, and has scalability and flexibility.
[0108] TaggerOne(Leaman and Lu 2016) is a conditional random field-based model for entity tagging in biomedical text. It combines multiple features to capture the context information of entities, and uses a decoding algorithm in the tagging stage to predict the sequence of entity labels.
[0109] CNN-based Ranking(Li et al.,2017b) is a convolutional neural network (CNN) based ranking model.
[0110] Normco(Wright,2019) is a deep learning-based disease normalization model that can automatically map disease names in biomedical literature to a unified standard identifier by combining attention mechanisms and recurrent neural networks.
[0111] BNE (Phan et al., 2019) is a method that can generate robust biomedical name representations through word embeddings, character embeddings, and CNN-based encoders, as well as contrastive learning.
[0112] BERT-based Ranking (Ji et al., 2019) is a ranking model that learns the contextual information and semantic associations of text based on the BERT model, thereby improving the accuracy and effectiveness of text ranking and ranking.
[0113] LNM (Lihu Chen et al., 2020) model realizes efficient biomedical entity linking through input representation, context encoding, entity encoding, similarity calculation, and entity linking steps.
[0114] BioSyn (Sung et al., 2020) model adopts an advanced entity representation method, which obtains entity ranking results by calculating the sum of the similarity of the statistical vectors of the mention words and entities, respectively.
[0115] Table 2 is the acc@1 (%) experimental results of two entity linking datasets
[0116]
[0117]
[0118] In order to analyze the effect of each component in the model corresponding to the method of the application, the performance of the model in the NCBI dataset when different components are deleted is experimented. Among them, "-context_left" represents removing the context information in the prompt sentence; "-context" represents removing the overall context information in the prompt sentence; " -score2" represents removing the prompt score in the second stage, and only relying on the similarity score during candidate generation for entity linking; " -score1" represents removing the similarity score during candidate generation, and only relying on the prompt scoring score for entity linking. The experimental results are shown in Table 3.
[0119] Table 3 is the ablation experiment results
[0120]
[0121] " -" represents removing the corresponding model component.
[0122] As can be seen from the table, the effect of the model is not ideal when only the score of a certain stage is used, especially when only the second stage score is used, the model performance decreases the most. In the second stage scoring process, if the context information of the mention word is removed for prompt scoring, the model performance will decrease by 0.63% and 0.62% respectively in Acc@1 and Acc@5. This is because the prompt sentence pattern lacks the corresponding semantic information at this time, so that the score obtained is not accurate. When the second stage only uses part of the context of the mention word, the prompt scoring is negatively affected by the incomplete context semantic information, so that the model performance also decreases.
[0123] The specific embodiments of the present application are described in detail above with reference to the accompanying drawings, but the present application is not limited to the above-described embodiments, and various changes can be made within the knowledge of those skilled in the art without departing from the spirit of the present application.< / cls> < / cls> < / sep> < / sep> < / sep> < / cls> < / sep> < / cls> < / cls> < / sep> < / sep> < / sep> < / cls> < / sep> < / cls> < / cls> < / cls>
Claims
1. A method for entity linking based on prompt scoring, characterized in that, include: The mentions were cleaned to remove abbreviations and spelling errors; Vectorize entity E in the knowledge base and mention word M in the text, calculate the inner product, and obtain the score. The top 20 entities in the knowledge base are selected as candidate entities; Next, the candidate entities and mention words are combined to construct a prompt sentence, and contextual information is concatenated as input to the BERT model. After passing through the attention mechanism, the final vector representation h of the tokens in the input sequence is obtained. i ∈R h This includes information about the interaction between this token and other location tokens; Location label <cls>Vector h at point i ∈R h The data is fed into a classifier model to obtain the positive or negative probability of the prompt, and the probability p of predicting it as positive is obtained as the probability score. Finally and The final ranking score S is obtained by summing the results after adjusting the hyperparameter α. m,e , the fraction S m,e The highest-ranking entity is used as the final linking result;< / cls> The specific steps of the entity linking method based on prompt scoring are as follows: Step 1: Spell checking was performed to correct spelling errors in the references; Ab3P abbreviation parsing was also used; for compound references, they were broken down into individual references. Step 2: In the candidate entity discovery stage, TF-IDF sparse representation and dense representation based on pre-trained language model are combined to calculate the similarity between mention words and entities. Let the sparse representations of the mention M and an entity E be respectively and Dense characterization is respectively and Step 3: Calculate the sparse vector representation and inner product and dense characterization and inner product Then, the two inner product values are summed to obtain the first-stage score. The top 20 entities and their scores, ranked from highest to lowest, are then selected as candidate entities. Step 4: Construct a prompt sentence using the candidate entities obtained in Step 3 and concatenate context information as input to the BERT model; feed the input sequence into the BERT model to achieve vectorization, and fuse the information of the input sequence through a self-attention mechanism. After multiple rounds of attention processing, obtain the final vector representation h of the token in the input sequence. i ∈R h ; Step 5: Extract vector h i ∈R h Middle position label <cls>Vector h at point c ∈R h , will h c The data is fed into a classifier to obtain the class probability p, and the classifier model is trained using the cross-entropy loss function. After the prompt classifier training is completed, the class probability p is used as the similarity score in the second stage. < / cls> Step 6: Transfer the first stage score from Step 3. and the second stage score in Step 5 Perform a weighted summation to determine the final ranking, and determine the entities linked to the mention word M based on the ranking of scores; Step 4 includes the following: Using the concept of prompts, we combine the mention word M in the text with the candidate entity E in the entity database to construct a question sentence, specifically in the following form: prompt:is the mention M similar to the entity E? Then, the mention word M, its context_left and context_right, and the constructed prompt are combined into a complete input sequence; special characters are used. <cls>To indicate the start of the input sequence, special characters. <sep>The input sequence is used to represent the separators in order to concatenate the three parts of information together; the input sequence is in the following form: input: <cls>context left <sep> M <sep>context right <sep> prompt.< / sep> < / sep> < / sep> < / cls> < / sep> < / cls> 2. The entity linking method based on prompt scoring according to claim 1, characterized in that: Step 4 also includes the following: Each token in the input sequence is converted into its corresponding word vector representation through an embedding layer. i ∈R h ; In this way, the entire input sequence can be represented as a sequence of word vectors: {e1,e2,e3,……,e n }=Embedding(input) The relative positional information of different tokens in the input sequence has a representational effect on the sequence. Therefore, positional embedding is used to convert each token in the input sequence into its corresponding positional embedding p. i ∈R h This allows the entire input sequence to be represented as a sequence of position embedding vectors: {p1,p,p3,……,p n }=Position_Embedding(input) For each token, by analyzing the word vector e i and position embedding vector p i Performing the summation operation yields a combined vector f. i ∈R b : f i =e i +p i To capture more contextual information, the pre-trained model uses a multi-layer attention mechanism to iterate the entire feature vector sequence multiple times; first, the feature vector f... i With a randomly initialized matrix Multiply to calculate three new vectors: Q-query, K-key, and V-value, then perform attention operations. Q=f i ·w Q K=f i ·w K V=f i ·w V Where, d k Represents the dimensions of vectors Q and K, and Softmax is the normalized exponential function; After multiple rounds of attention processing, the final vector representation h of the tokens in the input sequence is obtained. i ∈R h It contains information about the interaction between this token and other location tokens.
3. The entity linking method based on prompt scoring according to claim 1, characterized in that: Step 5 includes the following: Step 5.1: Construct training samples for training the prompt classifier; take all entities with the same CUI in the entity library corresponding to a mention word as positive samples in the training set, and sample an equal number of other entities as negative samples; for positive samples, that is, the prompt input sequence constructed by a mention word and its corresponding standard entities, its classification label is assigned to 1; for the prompt input sequence constructed by negative samples, its classification label is assigned to 0. Step 5.2: Transfer the input sequence to... <cls>The feature vector h corresponding to the position c ∈R h The input is fed into a fully connected layer and a sigmoid activation function, which maps it to class probability p:< / cls> p=Sigmoid(W1*h c +b) Where W1∈R 1*h b are the parameters to be trained; The cross-entropy loss method is used to train the classification model, and its formula is shown below: Where N is the number of training samples, y i is the true label of the i-th sample, while p i It is the probability that the model predicts for the i-th sample; After the prompt classifier training is completed, the mention word M and the candidate entity E are fed into the model to obtain their class probabilities p, which is the similarity score in the second stage:
4. The entity linking method based on prompt scoring according to claim 1, characterized in that: Step 6 includes the following: Scores of 20 candidate entities during the candidate entity discovery phase Compared with the score based on the Prompt classifier Perform a weighted summation to determine the final ranking; α is an adjustable hyperparameter, and the proportion of the score in the second stage is controlled by setting the value of α. Ultimately, according to S m,e The score determines the entity linked to the mention word M.