A medical entity linking method and system based on a pre-trained model and a graph convolution network

Abstract

The application discloses a medical entity linking method based on a pre-training model and a graph convolution network, which comprises the following steps: establishing a medical entity linking data set and generating a self-supervised data set; fine-tuning the pre-training model by using the self-supervised data set; embedding the entity mention and the entity by using the fine-tuned pre-training model, calculating the similarity of each entity mention, and sorting the entity similarity to obtain the candidate entity most similar to each entity mention in the knowledge base; combining one entity mention and the corresponding candidate entity to obtain a data pair, initializing the graph node representation by embedding the data pair by using the pre-training model, constructing a graph convolution network by using the entity mention and the corresponding candidate entity, learning the graph formed by the data pair by using the graph convolution network, and learning the parameters of the pre-training model used for initializing the graph node initialization embedding by using a small learning rate; ranking the nodes after scoring, and selecting the entity in the node with the maximum score as the result of entity linking.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of computers, and relates to self-supervised learning, a pre-training model, a graph convolution network, in particular to a medical entity linking method and system based on a pre-training model and a graph convolution network. BACKGROUND

[0002] 1) Self-supervised learning

[0003] In recent years, self-supervised learning has attracted widespread attention. Self-supervised learning models can be divided into three categories: generative, contrastive and adversarial. Generative models are represented by autoencoders; contrastive models learn representations by contrasting positive and negative samples; adversarial models retain the generator structure composed of encoders and decoders, which can provide powerful learning representation capabilities for adversarial models. In the field of natural language processing, before the concept of self-supervised learning was proposed, language models embodied the idea of self-supervised learning. For example, the Word2Vec model implements center word prediction (CBOW model) and adjacent word prediction (Skip-Gram model); the pre-training tasks of Skip-ThoughtVectors, BERT and many pre-training language models proposed thereafter contain self-supervised learning tasks. The self-supervised learning framework contains an auxiliary task of automatic labeling that is different from the core task of downstream, through the interaction of different parts of data, the learning representation of data is realized, and the learned intermediate feature layer representation or model weight is used in the downstream supervised learning prediction task, thereby reducing the demand for a large amount of labeled data and making full use of the multiple modes that each piece of data may be associated. In the field of medical entity linking, there is less labeled data, but there are relatively more knowledge bases. Through these knowledge bases, self-supervised learning tasks can be constructed, which will be of great help to the downstream entity linking task.

[0004] 2) Pre-training model

[0005] In the classification model, the pre-training model of the self-encoding is mainly used. BERT uses the encoder in the Transformer in the language model, and the upper and lower layers in the model can obtain the attention information of all positions, better obtaining the bidirectional relationship in the sentence. Masked Language Model (MLM) and Next Sentence Prediction (NSP) are introduced in BERT, which is conducive to learning the general knowledge in the field. For the medical field, unsupervised pre-training on medical corpus is conducive to obtaining general knowledge in the medical field and improving the effect of the model.

[0006] 3) Graph convolutional network

[0007] The graph neural network draws ideas from convolutional neural networks, recurrent networks and deep autoencoders, aiming to extend existing neural networks and define and design neural network structures for processing graph-structured data. Although traditional methods have achieved great success in extracting features of Euclidean space data, graph neural networks can not only reflect the complex relationship between objects using graphs, but also perform feature extraction and representation on data generated in non-Euclidean space. It has shown more powerful functions in learning graph-structured data. Because of the superior performance of GNN (Graph Neural Network) in graph learning, it is widely used in recommendation systems. In addition, it is also applied to many fields, such as text classification, natural language processing, disease prediction, feature relationship extraction, etc.

[0008] The principle of the graph neural network is mainly to first construct a graph structure model to reflect the relationship between entities and entities; then use a specific method to describe the node and obtain a node containing the final neighbor node information and topological structure feature state, which has been continuously updated and iterated; finally, output the representation form of the node in a certain specific way, and obtain the required information.

[0009] Due to the problems of labeling difficulty and the like, the medical entity linking dataset is small in size, thereby causing the model to have a poor learning effect, the medical knowledge base is mined, a plurality of synonym pairs and difficult negative example pairs are generated, the discrimination capability of the model for medical entity pairs is improved, and the recognition effect of the model on the target dataset is enhanced. In addition, the current entity linking method only focuses on the one-to-one relationship between the entity mention and the entity, and the similarity between the entity mention and the entity is calculated after the entity mention and the entity are directly represented by the related features. However, due to the fact that the entity linking dataset in the medical field usually lacks related information such as context information between the entity mention and the entity, the linking by directly using the names of the entity mention and the entity has the limitation of lacking information. The one-to-many relationship between the entity mention and the entity is used, and it is expected that the model can learn the following information by constructing a graph network: if the entity e1 is similar to the entity e2, and the entity mention is not similar to the entity e1, and is relatively similar to the entity e2, then the relationship between the entity mention and e2 can be learned by the entity e1, and the representation of the entity mention and the entity is further optimized. SUMMARY

[0010] In order to solve the problems in the prior art, the purpose of the present application is to provide a medical entity linking method based on a pre-training model and a graph convolutional network. Specifically, a self-supervised dataset is constructed using the medical knowledge base SNOMED CT, the original pre-training model is fine-tuned, and the representation of the entity is enhanced based on the graph convolution method to improve the effect of the classifier.

[0011] Entity linking refers to the process of unambiguously and correctly pointing identified entity objects in free text to target entities in a knowledge base. In this invention, entity linking is used to link non-canonical medical entities in user-generated data to medical concepts in SNOMED-CT and AMT (the Australian Medicines Terminology), facilitating the retrieval of relevant information for these medical concepts in subsequent tasks, such as mining symptom-related diseases through knowledge graphs.

[0012] The specific technical solution for achieving the purpose of this invention is as follows:

[0013] This invention proposes a method for linking medical entities based on a pre-trained model and a graph convolutional network, comprising the following steps:

[0014] Step 1: Establish a medical entity linking dataset. The data comes from 17,324 adverse drug reaction (ADR) annotations collected from blog posts on askpatient.com. A self-supervised training dataset U is generated by processing the medical knowledge base SNOMED CT from the Unified Medical Language System (UMLS) developed by the National Library of Medicine. SNOMED CT contains a large number of concepts and related synonyms. Positive training examples are constructed using synonyms. In mini-batches, other data are selected to form negative examples, thus forming the training dataset. The intersection of the synonym pairs generated in this process with the entity pairs in the medical entity linking dataset is 0.

[0015] Step 2: Fine-tune the pre-trained model using the self-supervised dataset U. Since pre-trained models are typically trained using two tasks—Masked Language Modeling (MLM) and Next Sentence Prediction (NSP)—they differ significantly from the entity linking task. Fine-tuning the pre-trained model's parameters using the self-supervised training dataset from Step 1 will benefit downstream tasks.

[0016] Step 3: Use the fine-tuned pre-trained model E to embed entity mentions and entities, obtaining embedding representations for set M and set N, where M represents the set of entity mentions and N represents the set of entities. For each entity mention i, use... Calculate their similarity. Indicates entity mention M i Embedded representation, Represents entity N j The embedded representation, specifically the calculation method of cosine_similarity is as follows: For Nj The similarity is ranked to obtain 50 candidate entities in the knowledge base most similar to each entity mention;

[0017] Step four: combine an entity mention with the corresponding 50 candidate entities to obtain 50 data pairs (e i ,m j ), initialize the graph node representation using the pre-training model E, construct a graph convolution network using the entity mention and the corresponding candidate entity, learn the graph formed by the above 50 data pairs using the graph convolution network, and learn the parameters of the pre-training model used to initialize the graph node initialization embedding using a small learning rate;

[0018] Step five: train the classifier, rank the nodes after scoring, and select the entity in the node with the highest score as the result of entity linking.

[0019] The self-supervised training data set U fine-tunes the pre-training model, specifically by the following way:

[0020] For a piece of data in the medical knowledge base SNOMED CT, as shown below: C0482287||kp^b antibody: presence or threshold: time point: serum / plasma: ordinal type, by constructing a dictionary with CUI as the key and the related entity corresponding to the CUI as the value, and enumerating the binary tuple in the value, the following can be obtained Where (x, y) represents a (name, CUI) pair, and further represents a (drug name, CUI) pair, such as (remdesivir, C4726677). CUI is the abbreviation of Concept Unique Identifier, which represents a unique identifier for a unique real medical concept. For data in a small batch mini-batch, a represents an anchor, i.e., a medical concept corresponding to the current CUI, p represents a positive example, i.e., a synonym, and n represents a negative example, which is mined by the distance from the anchor, where each triple is in the form of (x a ,x p ,x n ). Where x a is called an anchor, x p is a positive match of x a (that is, y a =y p ), and x n is a negative match of x a (that is, y a ! = y n ), and y represents the CUI label corresponding to x. The formula is calculated as follows: ||f(x a )-f(x p )||2<||f(x a) f(x n ||2+λ, when the distance between the positive example and the medical concept and the distance between the negative example and the medical concept is less than λ, it is considered that the negative example is difficult to distinguish at this time, and a sample pair with the label 0 is generated (anchor, negative example) for model training, thereby increasing the distance between the positive and negative samples.

[0021] The loss function when training the model is:

[0022]

[0023] wherein, represents the loss function. represents a mini-batch, represents the positive and negative samples constructed in the small batch mini-batch, and α, β are hyperparameters for controlling the temperature, S in , S ip represents the similarity of anchor i and the positive and negative samples, since the loss function should be as small as possible, the loss of the negative sample should be as large as possible, and thus S ip the negative sign is taken, and ∈ represents the offset of the similarity matrix.

[0024] The graph convolution network is specifically constructed in the following manner, and the graph node is the embedding initialization representation of the entity and the entity mention wherein E represents a pre-trained model, represents = [CLS] mention i [SEP] entity j , wherein [CLS], [SEP] are special symbols in the pre-trained model, and are used to represent the start of the input and the separation of the two meaningful fields. The graph is a full connection undirected graph, and there is a connection between any two edges, and the weight of the connected edge is the similarity between nodes wherein represents a full connection layer, represents a connection, and an adjacency matrix is obtained

[0025] The updating method of the graph convolution learning l-1 layer and the l layer node representation is as follows:

[0026]

[0027]

[0028]

[0029] wherein H (0) represents the node representation generated by the 0th layer using the above Initialize for each node, H (l) where represents the node representation generated in the lth layer, represents the sigmoid function, represents the tanh function, represents the Hadamard point-by-point multiplication operator, and z (l) acts as a gate to regulate the information from adjacent pairs of the lth layer, where represents the parameter to be learned in the lth layer, and Z, S Z, NO, S O, N represent different matrices, and the highway-gate is used to alleviate the problem that all node representations tend to be the same when the number of layers of the graph convolution network is large, that is, over-smoothing;

[0030] After the combination of the entity and the entity mention, the following is obtained where [CLS], [SEP] are special symbols in the pre-training model, and are used to represent the beginning of the input and the separation of the two meaningful fields, respectively.

[0031] In step five, the classifier is trained by using a loop iteration algorithm to obtain the node representation of the nth layer, and then a fully connected layer is used to score the graph nodes, and the graph node with the highest score is selected.

[0032] The present application provides a system for implementing the above method, which comprises a self-supervised data acquisition module, a self-supervised training module, a graph convolution model training module, and a graph node classification and sorting module.

[0033] The self-supervised training module is used to generate difficult classification negative pairs in addition to the positive pairs generated in the previous step in small batch training, and a loss function is designed to train the pre-training model.

[0034] The graph convolution model training module is used to learn the mutual relationship between nodes, and then optimize the representation between similar nodes and dissimilar nodes.

[0035] The graph node classification and sorting module is used to construct a classification model by using MLP, convert the problem into a machine learning multi-classification prediction problem, and then sort to obtain the node with the highest score.

[0036] The beneficial effects of the present application include: the present application designs a self-supervised learning fine-tuning pre-training model, which makes the pre-training model more suitable for entity linking problems in the medical field. Since the pre-training model is mainly pre-trained through the Masked Language Model (MLM) and Next Sentence Prediction (NSP) tasks. For the scene of using pre-training model for downstream classification task, the [CLS] special symbol added at the input head when the pre-training model is tokenized is usually used for classification. However, due to the difference between the pre-training task of sentence prediction and this kind of classification problem, directly using the output of the [CLS] position of the pre-training model for classification will result in poor effect. Through self-supervised training, the model can learn the domain knowledge in the medical field and optimize the representation of the [CLS] position. The present application makes full use of the advantage of rich medical domain knowledge base to improve the representation effect of the pre-training model. On the basis of embedding representation of entities and entity mentions using traditional pre-training model, graph convolution network is added to further strengthen the representation between entity mentions and entities. The traditional pre-training entity linking method mainly uses the pre-training model to embed the [CLS] mention i [SEP]entity j After embedding representation, the vector at the corresponding position of [CLS] is directly used for classification, while ignoring the help of other entities to the classification of entity mentions. The present application learns the relationship between entity mentions and multiple entities through graph convolution network, which is beneficial to the model to make judgment and further optimize the effect of the model.

[0037] The traditional method such as WordGRU embeds entity mentions and entities based on character and word related features, and the pre-training related method such as BIOSYN directly classifies the vector at the corresponding position of [CLS] and selects the entity with the highest score. From the experimental results, the effect of the present application is better than the above-mentioned methods, and the hit rate is increased by 0.6 and 1.7 respectively. BRIEF DESCRIPTION OF DRAWINGS

[0038] Figure 1 The flowchart of the medical entity linking method of the present application.

[0039] Figure 2 The graph shows the effect of different parameters of graph convolution in the present application.

[0040] Figure 3 The graph shows the effect of the present application. DETAILED DESCRIPTION

[0041] The present application will be further described in combination with the following specific embodiments and drawings. The process, conditions, experimental methods, etc. for implementing the present application are the general knowledge and common sense in the art, and the present application does not have special restrictions.

[0042] The application discloses a medical entity linking method based on a pre-training model and a graph convolution network. First, a pre-training model is used to obtain a candidate entity mention corresponding to an entity and a representation of an entity-entity mention pair, and then a graph convolution network is used to obtain the mutual relationship between them, so that the distance between correct entity-entity mention pairs is closer and the distance between incorrect entity-entity mention pairs is farther, so as to improve the discrimination ability of the model. The innovation of the application is that the pre-training fine-tuning and knowledge base method are introduced into the process of generating the candidate entity mention corresponding to the entity, which improves the recall rate and enhances the representation ability of the model by introducing the graph convolution network. Finally, experiments are conducted on a scene data set to verify the effectiveness of the application.

[0043] The application comprises the following steps:

[0044] Step one: establish a medical entity linking data set U, process a medical knowledge base SNOMED CT (The Systematized Nomenclature of Human and Veterinary Medicine Clinical Terms) to obtain a pre-training self-supervised data set U;

[0045] The medical entity linking data set T is mainly collected from 17,324 drug adverse reaction (ADR) annotations in askapatient.com blog articles. These mentions are mapped to 1,036 medical concepts based on SNOMED-CT and AMT (the Australian Medicines Terminology).

[0046] For the self-supervised dataset U, the UMLS 2020AA version data was downloaded, which is the largest medical ontology dataset and contains many medical synonyms of different types. All English entries were extracted from the SNOMED-CT original file, and all entity names were converted to lowercase, and then duplicates were removed. After constructing a dictionary with CUI as the key and the relevant entity corresponding to the CUI as the value, and then enumerating the binary tuples in the value, a list containing 9,712,959 (name, CUI) entries was obtained. However, randomly generated mini-batch data on this list can result in very few positive pairs in the mini-batch. In order to ensure that there are enough positive samples in each mini-batch, the enumeration of positive samples was realized by enumerating all possible synonym pairs and CUI combinations, and offline positive sample pairs in the format (name1, name2, CUI) were generated, where name1 and name2 have the same CUI label. Data with more than 50 positive examples were reduced to 50. Finally, a training list containing 11,792,953 paired entries, i.e., the self-supervised training set U, was obtained.

[0047] Step two: self-supervised learning of medical pre-trained model using medical knowledge base SNOMED CT;

[0048] For training in each mini-batch, the present application uses dynamic triplet mining to find the training examples with the largest amount of information in the mini-batch (i.e., hard positive / negative pairs) for effective training. From all the names in the mini-batch, all possible triplets were constructed, where each triplet is in the form (x a ,x p ,x n ). Where x a is called the anchor, x p is the positive match of x a (i.e., y a =y p ), and x n is the negative match of x a (i.e., y a ! = y n ). Among the constructed triplets, all triplets that violate the following conditions were selected:

[0049] ||f(x a )-f(x p )||2<||f(x a )-f(x n )||2+λ,

[0050] where λ is a pre-set margin, i.e. only consider the triplets whose distance between negative sample and positive sample is less than λ. Collect all such positive-negative pairs and represent them as P, N.

[0051] The loss function when training the model is:

[0052]

[0053] where, represents the loss function. represents the mini-batch, represents the positive and negative samples constructed in the mini-batch, and α, β are hyperparameters controlling the temperature, S in , S ip represents the similarity between anchor i and positive and negative samples, since the loss function should be as small as possible for positive samples and as large as possible for negative samples, S ip takes a negative sign in front.

[0054] Step three: use the fine-tuned pre-trained model to get the candidate entities of the entity mention and get the initial representation. For each entity mention M i find the corresponding candidate entity set s, and use the pre-trained model to get the initial representation of M i and s.

[0055] Specifically, use E to represent the pre-trained model, and use the pre-trained model to embed all entities in the knowledge base D to get the representation set N. For entity mention M i use f to get the embedding representation and use calculate its similarity, represents the embedding representation of entity mention M i , represents the embedding representation of entity N j , and cosine_similarity is specifically calculated as Select the top 50 entities in terms of similarity as the candidate entity set s. Experiments show that the recall rate on the training set is 0.9450720685625243, the recall rate on the validation set is 0.9796696315120712, and the recall rate on the test set is 0.9760416666666667.

[0056] Step four: use m and s to build a graph convolutional network, learn the graph convolutional model, and fine-tune the pre-trained model;

[0057] For the construction of the graph convolutional network, first is the construction of the node. For entity mention M iFind the corresponding candidate entity set s, the application constructs a new set S = {(M i ,s i )|s i ∈s}, for the initialization of node representation, the application uses a pre-trained model for embedding representation, first combines the entity and entity mention to get The embedding representation of which is The initialization of the node is completed. Then the edges on the graph are established, the application constructs a fully connected graph, and the edge weight on the graph is calculated as follows: Where r i , r j represent nodes i and j, f represents a pre-trained model, A fully connected layer is used. The learning of the graph convolutional network is as follows,

[0058]

[0059]

[0060]

[0061] H (l) represents the node representation generated by the lth layer, σ1(·) represents the sigmoid function, σ2(·) represents the tanh function, and is a Hadamard point-by-point multiplication operator. W is the weight matrix to be learned. The gating mechanism of GCN imitates the gating mechanism in the Highway gate, which is used to adjust the proportion of updated information. In the above formula, z (l) acts as a gate to adjust the information of adjacent pairs from the lth layer. In this way, each node can retain its own representation and learn valuable information from its neighbors. After L' layers, the final node representation H (L′) is the output of the gated GCN.

[0062] Step five: the classifier is trained by a loop iteration algorithm, and the entity linking result is obtained by probability sorting.

[0063] Let After passing through a fully connected layer, the probability of matching between the entity mention and the entity is obtained, and the most similar entity is obtained after sorting the probabilities of 50 entities. The entity corresponding to the highest probability is taken as the final result.

[0064] Embodiment

[0065] ​Experiments are conducted on the following scenario dataset to evaluate the effect of the proposed multi-view active learning classifier. The 1-hit rate and 5-hit rate scores are used to evaluate the effect of the classifier, the 1-hit rate is the proportion of the expected entity score ranking highest in the 50 candidate entities, and the 5-hit rate is the proportion of the expected entity score ranking in the top five, both of which are higher the better.

[0066] Scenario dataset: The medical entity linking dataset T mainly comes from 17,324 drug adverse reaction (ADR) annotations collected in askapatient.com blog articles. These mentions are mapped to 1,036 medical concepts based on SNOMED-CT and AMT (the Australian Medicines Terminology).

[0067]

[0068] Experiments show that the method proposed in the present application exceeds many previous methods, and the 1-hit rate and 5-hit rate are improved, proving the effectiveness of the method. Since the data set used in the present application is a public data set, in the above method, some of the papers only give the 1-hit rate, so some of the 5-hit rates are missing. From the experimental effect, the present application is improved compared with the previous methods, for the 1-hit rate, the method of the present application is improved by 2.9% compared with WORDCNN, 0.6% compared with WordGRU, 0.2% compared with CharCNN+Attend+MT, 1.7% compared with BIOSYN, and 2.6% compared with BERTRANK, and for the 5-hit rate, compared with BIOSYN, the present application is improved by 8.4%, which is significantly improved.

[0069] The protection scope of the present application is not limited to the above embodiments. Changes and advantages that can be thought of by those skilled in the art without departing from the spirit and scope of the present application are included in the present application, and are protected by the appended claims.

Claims

1. A medical entity linking method based on a pre-trained model and a graph convolution network, characterized in that, Comprising the following steps: Step one: establish medical entity linking dataset and generate self-supervised dataset; Step two: fine-tune the pre-training model using the self-supervised dataset; Step three: using the fine-tuned pre-trained model to embed the entity mentions and the entities to obtain the embedding representation of the set M and the embedding representation of the set N, where M represents the set of entity mentions, N represents the set of entities, for each entity mention M i , calculate the similarity, sort the similarity of the entities N j to obtain the candidate entities in the knowledge base that are most similar to the each entity mention. Step four: combine an entity mention with the corresponding candidate entity to obtain a data pair, initialize the graph node representation using the pre-training model embedding, construct a graph convolution network using the entity mention and the corresponding candidate entity, learn the graph formed by the data pair using the graph convolution network, and learn the parameters of the pre-training model used to initialize the graph node initialization embedding using a small learning rate; comprising the following steps: Step 4.1: Construct the graph convolution network by the following way, the graph node is the embedding initialization representation of entity and entity mention Wherein E represents a pre-training model, represents = [CLS] mention i [SEP] entity j , wherein [CLS], [SEP] are special symbols in the pre-training model, respectively used to represent the beginning of the input and the separation of the two meaningful fields; the graph is a full connection undirected graph, and there is a connection between any two edges, and the weight of the connected edge is the similarity between the nodes Wherein represents a full connection layer, and represents connection, to obtain an adjacency matrix Step 4.2: The update method of the l-1 layer and the l layer node representation of the graph convolution learning is as follows: where H (0) represents the node representation generated in the 0th layer, and is initialized for each node, H (l) represents the node representation generated in the lth layer, σ1(·) represents a sigmoid function, σ2(·) represents a tanh function, ⊙ represents a Hadamard pointwise multiplication operator, z (l) serves as a gate to regulate information from adjacent pairs in the lth layer, represents the parameter to be learned in the lth layer, and the highway-gate is used to alleviate the problem that all node representations tend to be the same when the number of layers of the graph convolution network is large, i.e., over-smoothing, and subscripts Z, S Z, N O, S O, N represent different matrices. The entity and the entity mention are combined to obtain wherein [CLS], [SEP] are special symbols in the pre-training model, respectively used to represent the beginning of the input and the separation of the two fields; Step five: train the classifier, sort the nodes after scoring, and select the entity in the node with the highest score as the result of entity linking. 2.The pre-trained model and graph convolution network based medical entity linking method of claim 1, wherein, The self-supervised training dataset U fine-tunes the pre-training model, specifically by the following way: For a piece of data in the medical knowledge base SNOMED CT, a dictionary is constructed with the key as CUI and the value as the related entity corresponding to the CUI, and then a binary tuple in the value is enumerated to obtain where (x, y) represents a (drug name, CUI) pair; for data in a small batch mini-batch, a represents an anchor, that is, a medical concept corresponding to the current CUI, p represents a positive example, that is, a synonym, and n represents a negative example, and negative sample mining is performed through the distance from the anchor, where each triple is in the form of (x a ,x p ,x n ); where x a is called an anchor, x p is a positive match of x a , that is, y a =y p ; x n is a negative match of x a , that is, y a ! = y n ; y represents the CUI label corresponding to x; the formula is calculated as follows: ||f(x a )-f(x p )||2<||f(x a )-f(x n )||2+λ, when the distance between the positive example and the medical concept and the distance between the negative example and the medical concept are less than λ, it is considered that the negative example is a difficult-to-distinguish negative example at this time, a sample pair with a (anchor, negative example) label of 0 is generated for model training, and then the distance between the positive and negative samples is increased. 3.The pre-trained model and graph convolution network based medical entity linking method of claim 1, wherein, The loss function when training the model is: wherein, represents a loss function, represents a mini-batch, represents positive and negative samples, and a, b are hyperparameters that control temperature, S in , S ip represents the similarity of anchor point i with positive and negative samples, since the loss of positive samples should be as small as possible and the loss of negative samples should be as large as possible in the loss function, S ip the negative sign is taken in front, and e represents the offset of the similarity matrix. 4.The pre-trained model and graph convolution network based medical entity linking method of claim 1, wherein, The candidate entity generation comprises the following steps: firstly, embedding all entities in the knowledge base through the pre-training model after self-supervision to obtain E d Then, the similarity between the entity mention and all entities in E d is calculated, and the top 50 most similar entities are taken as candidate entities.

5. The pre-trained model and graph convolution network based medical entity linking method of claim 4, wherein, Using computing similarity, an embedding representation of entity mention M i , an embedding representation of entity N j , ranking the similarities for N j , obtaining 50 candidate entities most similar to each entity mention in the knowledge base. 6.The pre-trained model and graph convolution network based medical entity linking method of claim 1, wherein, The classifier uses a recurrent iterative algorithm for training to obtain the graph node representation of the nth layer, and then scores the graph nodes through a fully connected layer, and selects the graph node with the highest score.

7. A medical entity linking system based on a pre-trained model and a graph convolution network, characterized in that, The system adopts the method of any one of claims 1-6, and the system comprises: a self-supervised data acquisition module, a self-supervised training module, a graph convolution model training module, and a graph node classification and sorting module; wherein, The self-supervised data acquisition module is used to process the data in UMLS to form a self-supervised dataset; The self-supervised training module is used to generate difficult classification negative pairs in addition to the generated positive pairs in small batch training, design a loss function to train the pre-training model; The graph convolution model training module is used to learn the mutual relationship between nodes, and further optimize the representation between similar nodes and dissimilar nodes; The graph node classification and sorting module is used to construct a classification model using MLP, convert the problem into a machine learning multi-classification prediction problem, and then sort to obtain the node with the highest score.

Images