Medical Entity Disambiguation Method and Device Based on Relationship Fusion and Information Fusion

Abstract

The present invention discloses a medical entity disambiguation method and device based on relationship fusion and information fusion, belonging to the fields of artificial intelligence and natural language processing. The technical problem to be solved by the present invention is how to use natural language processing technology to enhance the context representation of medical entity mentions and the representation of candidate medical entities, so as to accurately judge the candidate medical entity corresponding to the medical entity mention. The adopted technical solutions are as follows: ① A medical entity disambiguation method based on relationship fusion and information fusion, the method comprising the following steps: S1, constructing a medical entity disambiguation knowledge base; S2, constructing a medical entity disambiguation model data set; S3, constructing a medical entity disambiguation model; S4, training the medical entity disambiguation model. ② A medical entity disambiguation device based on relationship fusion and information fusion, the device comprising: a medical entity disambiguation knowledge base construction unit, a medical entity disambiguation model data set construction unit, a medical entity disambiguation model construction unit, and a medical entity disambiguation model training unit.

Description

Technical Field

[0001] The present invention relates to the technical fields of artificial intelligence and natural language processing, and particularly relates to a medical entity disambiguation method and device based on relation fusion and information fusion. Background Art

[0002] With the rapid development of healthcare, the number of biomedical texts has increased significantly, including electronic health records, biomedical literature, and clinical trial reports. To better utilize the rich knowledge contained in medical texts, medical entity linking technology is crucial, which can identify medical entity mentions and automatically link the entity mentions to the corresponding candidate entities in the knowledge base. The medical entity disambiguation task is a subtask of medical entity linking, aiming to accurately match and disambiguate the entity mentions (such as diseases, drugs, medical terms, etc.) in medical texts with the corresponding candidate entities in the medical knowledge base. The medical entity disambiguation task is of great significance for accurately understanding medical texts, supporting clinical decision-making, improving information retrieval and knowledge graph construction, and promoting medical research. With the rapid development of deep learning, many methods based on neural network models have been applied to the entity disambiguation task in the general field. However, due to the huge differences in language features and knowledge bases, these methods cannot be directly applied to the entity disambiguation task in the medical field. In recent years, methods based on graph neural networks have modeled medical entity disambiguation as a graph matching problem and improved the ability to eliminate medical entity ambiguities by using the domain knowledge of medical knowledge bases and effective negative sampling schemes. However, this method does not fully utilize the domain knowledge of medical knowledge bases and has insufficient representation ability for modeling medical entity mentions. To address this problem, some work attempts to utilize the fine-grained type information of medical entities and integrate it into the neural network medical entity disambiguation model, which alleviates the problem of insufficient representation ability for medical entity mentions and candidate medical entities to a certain extent. However, these works focus on using the type information of medical entities and ignore other knowledge contained in medical knowledge bases and the interaction between medical entity mentions and candidate medical entities. In summary, the existing medical entity disambiguation methods fail to fully mine the knowledge contained in medical knowledge bases, cannot fully model the representations of medical entity mentions and candidate medical entities, and ignore the interaction between medical entity mentions and candidate medical entities, leaving great room for improvement. Summary of the Invention

[0003] In view of the deficiencies of existing medical entity disambiguation methods, the present invention proposes a medical entity disambiguation method based on relationship fusion and information fusion to enhance the representation of medical entity mentions and candidate medical entities, while enhancing the interaction between the two. This method and device fuse the relationship information between medical entity mentions through a relationship fusion module to enhance the context representation of medical entity mentions; fuse the synonym information and type information of candidate medical entities through an information fusion module to enhance the representation of candidate medical entities; enhance the interaction between the context representation of medical entity mentions and the representation of candidate medical entities through the cross-attention mechanism in the interaction module; obtain a matching score through the matching module and select the candidate medical entity with the highest score, that is, the most matching candidate medical entity.

[0004] The technical task of the present invention is realized in the following way. A medical entity disambiguation method based on relationship fusion and information fusion, the method includes the following steps:

[0005] S1. Construct a medical entity disambiguation knowledge base: Download the medical data set published on the official website as the original data for medical entity disambiguation;

[0006] S2. Construct a medical entity disambiguation model data set: Preprocess the original data set obtained by downloading in S1 and convert the data into data in json format;

[0007] S3. Construct a medical entity disambiguation model: Construct an encoding module, a knowledge fusion module, an interaction module, and a matching module;

[0008] S4. Train the medical entity disambiguation model: Train the medical entity disambiguation model constructed in step S3 on the training data set of the medical entity disambiguation model obtained in step S2.

[0009] A medical entity disambiguation device based on relationship fusion and information fusion, the device includes,

[0010] A medical entity disambiguation knowledge base construction unit, which is used to download the medical data set published on the official website as the original data for medical entity disambiguation;

[0011] A medical entity disambiguation model data set construction unit, which preprocesses the downloaded original data set and converts the data into data in json format to obtain the training data set of the model;

[0012] A medical entity disambiguation model construction unit, which is used to construct an encoding module, a knowledge fusion module, an interaction module, and a matching module;

[0013] A medical entity disambiguation model training unit, which is used to construct the loss function and optimization function required during model training and complete the training of the model.

[0014] A storage medium stores multiple instructions, which are loaded by a processor to execute the steps of the above-mentioned medical entity disambiguation method based on relationship fusion and information fusion.

[0015] An electronic device, which includes:

[0016] The above-mentioned storage medium; and

[0017] A processor for executing the instructions in the storage medium.

[0018] The medical entity disambiguation method based on relationship fusion and information fusion of the present invention has the following advantages:

[0019] (1) Through the preprocessing of the medical data set, the present invention generates formatted data in a specific form, providing data support for the subsequent establishment of a medical entity disambiguation model;

[0020] (2) The present invention proposes an encoding module, which uses a language model BioBERT pre-trained on a large-scale biomedical corpus to better encode medical entity mentions and candidate medical entities;

[0021] (3) The present invention proposes a relationship fusion module, which fuses the relationship information between medical entity mentions by using a relational graph convolutional neural network, thereby enhancing the encoding representation of medical entity mentions;

[0022] (4) The present invention proposes an information fusion module, which fuses the fine-grained information of entities, including synonym information and type information, by using an attention mechanism, thereby enhancing the encoding representation of candidate medical entities;

[0023] (5) The present invention proposes an interaction module, which enhances the interaction between the context representation of medical entity mentions and the representation of candidate medical entities by using a cross-attention mechanism;

[0024] (6) By comprehensively using the encoding module, the relationship fusion module, the information fusion module, the interaction module, and the matching module, the present invention can effectively improve the effect of medical entity disambiguation. Description of the Drawings

[0025] The present invention will be further described below with reference to the drawings.

[0026] Att Figure 1 Is a schematic diagram of the model framework for the medical entity disambiguation method based on relationship fusion and information fusion;

[0027] Att Figure 2 Is a flowchart of the medical entity disambiguation method based on relationship fusion and information fusion;

[0028] Att Figure 3Schematic diagram of the structure of a medical entity disambiguation device based on relationship fusion and information fusion;

[0029] Appendix Figure 4 Flowchart for constructing a relationship fusion module;

[0030] Appendix Figure 5 Flowchart for constructing an information fusion module;

[0031] Appendix Figure 6 Flowchart for training a medical entity disambiguation model based on relationship fusion and information fusion.

[0032] Term explanation:

[0033] The mention of a medical entity in the present invention refers to the form of a medical entity that has not been disambiguated in the text. For example: "Insulin is an important factor in the treatment of DM", in the above sentence, before the medical entity mention "DM" is disambiguated, it may represent the medical entity "diabetes mellitus" in the medical knowledge base or the medical entity "diabetic microangiopathy" in the medical knowledge base.

[0034] Hyperparameters refer to the parameters whose values need to be manually set before the start of the training process; these parameters cannot be automatically optimized through training; according to the differences in actual data sets, these parameters need to be manually set by the user. Detailed implementation manners

[0035] The medical entity disambiguation method and device of the present invention based on relationship fusion and information fusion are described in detail below with reference to the accompanying drawings of the specification and specific embodiments.

[0036] Embodiment 1:

[0037] The overall model framework structure of the present invention is as Figure 1 shown. It consists of Figure 1It can be seen that the main framework structure of the present invention includes an encoding module, a knowledge fusion module, an interaction module, and a matching module. Among them, the encoding module encodes the context of the medical entity mention and the candidate medical entity respectively to obtain the context representation of the medical entity mention and the representation of the candidate medical entity; the knowledge fusion module includes a relationship fusion module and an information fusion module. The relationship fusion module fuses the relationship information between medical entity mentions to enhance the context representation of medical entity mentions. The information fusion module fuses the synonym information and type information of candidate medical entities to enhance the representation of candidate medical entities; the interaction module enhances the interaction between the context representation of medical entity mentions and the representation of candidate medical entities through a cross-attention mechanism; the matching module obtains a matching score through dot product operation and uses the softmax function, and selects the candidate medical entity with the highest score, that is, the most matching candidate medical entity. Specifically as follows:

[0038] (1) The encoding module uses the language model BioBERT pre-trained on a large-scale biomedical corpus. BioBERT first uses a tokenizer to convert the context of the medical entity mention and the candidate medical entity into token vectors, and then encodes the token vectors through BioBERT respectively to obtain the context representation of the medical entity mention and the representation of the candidate medical entity;

[0039] (2) The knowledge fusion module includes a relationship fusion module and an information fusion module. As shown in Figure 4 , the relationship fusion module first retrieves the context of the medical entity mention in the knowledge graph to obtain the relationship between medical entity mentions, then encodes the relationship information representation obtained by the relationship graph convolutional neural network, and finally splices the context representation of the medical entity mention obtained by the encoding module and the relationship information representation between medical entity mentions to obtain an enhanced context representation of medical entity mentions. As shown in Figure 5 , the information fusion module first retrieves and encodes the candidate medical entity in the knowledge base to obtain the synonym information representation of the candidate medical entity, then fuses the representation of the candidate medical entity obtained by the encoding module and the synonym information representation of the candidate medical entity through an attention mechanism to obtain the representation of the candidate medical entity with fused information, then retrieves and encodes the candidate medical entity in the knowledge base to obtain the type information representation of the candidate medical entity, and finally splices the representation of the candidate medical entity with fused information and the type information representation of the candidate medical entity to obtain an enhanced representation of the candidate medical entity;

[0040] (3) The interaction module interacts the enhanced context representation of medical entity mentions and the enhanced representation of candidate medical entities obtained by the knowledge fusion module through a cross-attention mechanism to obtain an interaction representation, thereby enhancing the interaction between the two;

[0041] (4) The matching module performs click operation on the interaction representation obtained by the interaction module and uses the softmax function to obtain a matching score, and selects the candidate medical entity with the highest score, that is, the most matching candidate medical entity.

[0042] Embodiment 2:

[0043] As shown in the attached Figure 2 For the medical entity disambiguation method based on relationship fusion and information fusion of the present invention, the method comprises the following steps:

[0044] S1. Construct a medical entity disambiguation knowledge base: Download the medical data sets published on the official website as the original data for medical entity disambiguation;

[0045] Illustrative example: There are many publicly available medical data sets on the network, such as the BC5CDR data set. The data format in the BC5CDR data set is as follows:

[0046]

[0047]

[0048] Among them, 354896 is the PMID, that is, the ID of the text; |t| is the title, that is, the title; |a| is the Abstract, that is, the text; 0, 9 are the start position index and end position index of the medical entity mention; 18, 34, 90, 99 are the same as above; Lidocaine, cardiac asystole, lidocaine are medical entity mentions; Chemical, Disease are the types of medical entity mentions; D008012, D006323 are entity concept IDs.

[0049] S2. Construct a medical entity disambiguation model data set: Preprocess the original data set downloaded in S1 and convert the data into json format data;

[0050] Illustrative example: The preprocessed data is as follows:

[0051] {"mention_id":"354896_0_0","start_index":0,"end_index":9,"text":"Lidocaine","type":"Chemical","content_document_id":"354896_0","label_candidate_id":"D008012"},{"mention_id":"354896_0_1","start_index":18,"end_index":34,"text":"cardiac asystole","type":"Disease","content_document_id":"354896_0","label_candidate_id":"D006323"},{"mention_id":"354896_0_2","start_index":90,"end_index":99,"text":"lidocaine","type":"Chemical","content_document_id":"354896_0","label_candidate_id":"D008012"}。

[0052] Among them, "mention_id" is the medical entity mention ID; "start_index" is the start position index of the medical entity mention, "end_index" is the end position index of the medical entity mention; "text" is the medical entity mention; "type" is the type of the medical entity mention; "content_document_id" is the text ID; "label_candidate_id" is the candidate medical entity ID.

[0053] S3. Build a medical entity disambiguation model: Build an encoding module, build a knowledge fusion module, build an interaction module, build a matching module. The specific steps are as follows:

[0054] S301. Build an encoding module: The encoding module converts the preprocessed data obtained in step S2 into token vectors, that is, converts the context of the medical entity mention and the candidate medical entity into token vectors, and then uses an encoder to encode the token vectors respectively to obtain the context representation of the medical entity mention and the representation of the candidate medical entity. The specific steps are as follows:

[0055] First, a medical entity mention m k ={c i ,…,c jThe context C = {c1, …, c l} and the candidate medical entity E = {e1, ..., e m} are respectively transformed into token vectors by the tokenizer of BioBERT, that is Then, the BioBERT encoder is used to encode them respectively to obtain the context representation H c of the medical entity mention and the representation H e of the candidate medical entity; the subscript k represents the k-th medical entity mention; the subscripts i and j respectively represent the start and end positions of the words of the medical entity mention; the subscript l represents the length of the words in the context of the medical entity mention; the subscript m represents the number of candidate medical entities; the specific formula is described as follows:

[0056]

[0057]

[0058] where the subscript L in formula (1) represents the length of the token vector; the superscript c represents the context; the M in formula (2) represents the length of the token vector; the superscript e represents the entity;

[0059] For example: In Pytorch, the code implementation for the above description is as follows:

[0060] tokenizer = AutoTokenizer.from_pretrained("dmis-lab / biobert")

[0061] model = AutoModel.from_pretrained("dmis-lab / biobert")

[0062] mention_context = config["mentions.json"]

[0063] candidate_entities = config["entity.txt"]

[0064] args.config = config

[0065] mention_context_tokens = tokenizer.encode(mention_context,

[0066] add_special_tokens=True, truncation=True, padding=True)

[0067] mention_context_encoded = model(torch.tensor([mention_context_tokens]))[0]

[0068] candidate_entities_encoded = []

[0069] for entity in candidate_entities:

[0070] entity_tokens = tokenizer.encode(entity, add_special_tokens=True,

[0071] truncation=True, padding=True)

[0072] entity_encoded = model(torch.tensor([entity_tokens]))[0]

[0073] candidate_entities_encoded.append(entity_encoded)

[0074] Among them, mention_context represents the context of a medical entity mention; candidate_entities represents the set of candidate entities; mention_context_tokens represents the context representation of a medical entity mention; entity_tokens represents the representation of a candidate medical entity; dmis-lab / biobert represents the pre-trained language model BioBERT loaded, and the open source address is https: / / huggingface.co / dmis-lab / biobert-v1.1;

[0075] S302. Build a knowledge fusion module: mainly including building a relationship fusion module and an information fusion module; the specific steps are as follows:

[0076] S30201. Build a relationship fusion module: As shown in the appendix Figure 4 shown, the relationship fusion module first processes all mentions m = {m1,..., m in the context C of a medical entity mention N} Retrieve in the knowledge graph to obtain all relations R = {r1,..., r b}, and then encode all relations R through the relational graph convolutional neural network to obtain the relational information representation between medical entity mentions Finally, the context representation H of the medical entity mentions obtained in S301 c is concatenated with the relational information representation H between medical entity mentions r Then, the enhanced context representation H of the medical entity mentions is obtained through a linear layer c,r , where the subscript N represents the number of medical mentions, the subscript b represents the number of relations, and the superscript r represents the relation; the specific formula description is as follows:

[0077] R = GE(m) (3)

[0078]

[0079] H c,r = W[H c ; H r + b (5)

[0080] Among them, GE represents the embedding lookup table of the knowledge graph, retrieving the relation set of all candidate medical entities in the knowledge graph; represents the relation v between medical entities i The hidden state representation at the l-th layer of the relational graph convolutional neural network; σ represents the sigmoid activation function; are learnable parameters; represents the set of neighbor indices of the i-th medical entity mention in relation R; represents the normalization constant; W, b represent learnable parameters; other symbols have been described before and will not be repeated;

[0081] For example: In Pytorch, the code implementation for the above description is as follows:

[0082] class RelationGCN(nn.Module):

[0083] def __init__(self, input_dim, output_dim):

[0084] super(RelationGCN, self).__init__()

[0085] self.linear = nn.Linear(input_dim, output_dim)

[0086] def forward(self,relation_embeddings):

[0087] relation_encoded = self.linear(relation_embeddings)

[0088] return relation_encoded

[0089] gcn = RelationGCN(embedding_dim,output_dim)embedding_dim,

[0090] relation_embeddings = []

[0091] for mention in mention_context:

[0092] if mention in knowledge_graph:

[0093] relations = knowledge_graph[mention]

[0094] for relation in relations:

[0095] relation_tokens = tokenizer.encode(relation,add_special_tokens = True,

[0096] truncation = True,padding = True)

[0097] relation_encoded = model(torch.tensor([relation_tokens]))[0]

[0098] relation_embeddings.append(relation_encoded)

[0099] if len(relation_embeddings)>0:

[0100] relation_embeddings = torch.stack(relation_embeddings)

[0101] relation_encoded = gcn(relation_embeddings)

[0102] enhanced_mention_context = torch.cat((mention_context_encoded,

[0103] relation_encoded), dim = 1)

[0104] linear_layer = nn.Linear(embedding_dim + output_dim,

[0105] enhanced_embedding_dim)

[0106] enhanced_mention_context = linear_layer(enhanced_mention_context)

[0107] Among them, RelationGCN() represents the relation graph convolutional neural network class; relation_embeddings represents the relation information between medical entity mentions; relation_encoded represents the representation of the relation information between medical entity mentions; enhanced_mention_context represents the concatenation of the context representation of medical entity mentions and the representation of relation information; cat() represents the concatenation operation; enhanced_mention_context represents the enhanced context representation of medical entity mentions;

[0108] S30202. Build an information fusion module: The information fusion module first retrieves and encodes the candidate medical entity in the knowledge base to obtain the synonym information representation H s of the candidate medical entity, and then uses the attention mechanism to fuse the representation H e of the candidate medical entity obtained in S301 with the synonym information representation H s of the candidate medical entity to obtain the representation H e,s of the candidate medical entity with the fused information; again, it retrieves and encodes the candidate medical entity in the knowledge base to obtain the type representation information H t of the candidate medical entity, and then concatenates the representation H e,s of the candidate medical entity with the fused information with the type information representation H t of the candidate medical entity to obtain the enhanced representation H e,s,t of the candidate medical entity; the superscript s represents synonyms, and the superscript t represents types. The specific formula description is as follows:

[0109]

[0110] H e,s,t = [H e,s ; H t (7)

[0111] Among them, Attention in formula (6) represents the attention mechanism; softmax represents the normalized exponential function; Q represents the query matrix of candidate medical entities, which is obtained by multiplying the input matrix formed by the query vector q of candidate medical entities with a trainable parameter matrix and then through linear transformation; K represents the synonym matrix of candidate medical entities, which is obtained by multiplying the input matrix formed by the synonym vector k of candidate medical entities with a trainable parameter matrix and then through linear transformation; V represents the value matrix of candidate medical entities, which is obtained by multiplying the input matrix formed by the value vector v of candidate medical entities with a trainable parameter matrix and then through linear transformation; the query vector q of candidate medical entities is the e token vector of H s ; the synonym vector k of candidate medical entities is the k token vector of H; the value vector of candidate medical entities is the token vector corresponding to the candidate medical entity query vector v; d

[0112] For example: In Pytorch, the code implementation for the above description is as follows:

[0113] synonyms_info = {}

[0114] entity_types_info = {}

[0115] class Attention(nn.Module):

[0116] def __init__(self, input_dim):

[0117] super(Attention, self).__init__()

[0118] self.linear = nn.Linear(input_dim, 1)

[0119] def forward(self, candidate_entity_repr, synonyms_repr):

[0120] combined_repr = torch.cat((candidate_entity_repr, synonyms_repr), dim=1)

[0121] attention_weights = F.softmax(self.linear(combined_repr), dim=1)

[0122] fused_repr = torch.matmul(attention_weights.transpose(1, 0),

[0123] synonyms_repr).squeeze(0)

[0124] return fused_repr

[0125] attention = Attention(embedding_dim)

[0126] synonyms_embeddings = []

[0127] for candidate_entity in synonyms_info:

[0128] synonyms = synonyms_info[candidate_entity]

[0129] for synonym in synonyms:

[0130] synonym_tokens = tokenizer.encode(synonym, add_special_tokens=True,

[0131] truncation=True, padding=True)

[0132] synonym_encoded = model(torch.tensor([synonym_tokens]))[0]

[0133] synonyms_embeddings.append(synonym_encoded)

[0134] if len(synonyms_embeddings)>0:

[0135] synonyms_embeddings = torch.stack(synonyms_embeddings)

[0136] attention(candidate_entity_repr, synonyms_embeddings)

[0137] type_embeddings = []

[0138] for candidate_entity in entity_types_info:

[0139] entity_type = entity_types_info[candidate_entity]

[0140] type_tokens = tokenizer.encode(entity_type, add_special_tokens=True, truncation=True, padding=True)

[0141] type_encoded = model(torch.tensor([type_tokens]))[0]

[0142] type_embeddings.append(type_encoded)

[0143] if len(type_embeddings) > 0:

[0144] type_embeddings = torch.stack(type_embeddings)

[0145] enhanced_candidate_entity_repr = torch.cat((fused_repr, type_embeddings), dim=1)

[0146] Among them, synonyms_info represents the set of synonym information; entity_types_info represents the set of types; Attention() represents the attention function; synonyms_embeddings represents the representation of synonym information; fused_repr represents the representation of the candidate medical entity with fused information; attention_weights represents the attention weights; matmul() represents the matrix multiplication operation; type_embeddings represents the type representation information of the candidate medical entity; enhanced_candidate_entity_repr represents the representation of the enhanced candidate medical entity. For the detailed content of the attention mechanism, please refer to the paper "Attention is all you need".

[0147] S303. Construct an interaction module: The interaction module takes the context representation H of the enhanced medical entity mention obtained in S30201 c,r and the representation H of the enhanced candidate medical entity obtained in S30202 e,s,t as inputs. First, calculate the similarity matrix S between the two, and then calculate the interaction between the two through the cross-attention mechanism to obtain the interaction representation A. The specific formula is described as follows:

[0148]

[0149]

[0150]

[0151]

[0152]

[0153] A = [a1,..., a u (13)

[0154] Among them, s in formula (8) ij represents the element of the similarity matrix S. The subscript i in ij represents the i-th token vector of the context representation of the enhanced medical entity mention, and the subscript j in ij represents the j-th token vector of the representation of the enhanced candidate medical entity; represents the learnable parameter; respectively represent the elements of the context representation H of the enhanced medical entity mention c,r and the representation H of the enhanced candidate medical entity e,s,t ; ⊙ represents the dot product operation; S in formula (9) α represents H c,r paying attention to He,s,t The attention matrix, where row represents applying softmax to the rows of the similarity matrix S; S in formula (10) β represents H e,s,t Focus on H c,r The attention matrix, where col represents applying softmax to the columns of the similarity matrix S; in formula (11) represents H c,r Focus on H e,s,t The attention scores, represents H e,s,t Focus on H c,r The attention scores, with subscript j representing the j-th element; a in formula (12) z represents the z-th interactive representation; A in formula (13) is the set of obtained interactive representations, with subscript u representing the number of interactive representations; other symbols have been described before and will not be repeated;

[0155] For example: In Pytorch, the code implementation for the above description is as follows:

[0156] class CrossAttention(nn.Module):

[0157] def __init__(self, input_dim):

[0158] super(CrossAttention, self).__init__()

[0159] self.linear1 = nn.Linear(input_dim, input_dim)

[0160] self.linear2 = nn.Linear(input_dim, 1)

[0161] def forward(self, enhanced_mention_context, enhanced_candidate_entity_repr):

[0162] attention_weights =

[0163] self.linear2(torch.tanh(self.linear1(enhanced_mention_context) +

[0164] enhanced_candidate_entity_repr))

[0165] normalized_weights = F.softmax(attention_weights, dim = 1)

[0166] interact_repr = torch.matmul(normalized_weights.transpose(1, 0),

[0167] enhanced_mention_context).squeeze(0)

[0168] return interact_repr

[0169] cross_attention = CrossAttention(enhanced_embedding_dim)

[0170] similarity_matrix = torch.matmul(enhanced_mention_context,

[0171] enhanced_candidate_entity_repr.t())

[0172] Among them, CrossAttention() represents the cross-attention class; enhanced_mention_context, mention_context_repr represent the context representation of the enhanced medical entity mention; enhanced_candidate_entity_repr represents the representation of the enhanced candidate medical entity; interact_repr represents the obtained interaction representation; similarity_matrix represents the similarity matrix;

[0173] S304, Construct a matching module: The matching module takes the interaction representation A obtained in S303 as input, obtains the matching scores of each candidate medical entity through dot product operation and uses the softmax function, and selects the candidate medical entity with the highest score; the specific formula description is as follows:

[0174] y i = softmax(a j T ·a i ) (14)

[0175] Among them, y i in formula (14) represents the matching score of each candidate medical entity; a jThe context representation of the j-th medical entity mention representing the interaction; a i The representation of the i-th candidate medical entity of the interaction;

[0176] For example: In Pytorch, the code implementation described above is as follows:

[0177] class MatchingModule(nn.Module):

[0178] def __init__(self, input_dim, num_candidates):

[0179] super(MatchingModule, self).__init__()

[0180] self.linear = nn.Linear(input_dim, num_candidates)

[0181] def forward(self, interaction_repr):

[0182] scores = self.linear(interaction_repr)

[0183] probabilities = F.softmax(scores, dim = 0)

[0184] return probabilities

[0185] num_candidates = []

[0186] matching_module = MatchingModule(input_dim, num_candidates)

[0187] probabilities = matching_module(interaction_repr)

[0188] max_score, max_index = torch.max(probabilities, dim = 0)

[0189] Among them, num_candidates represents the number of candidate medical entities; linear() represents the linear layer function; score represents the matching score; probabilities represents the matching probability score; max_score represents the highest matching score, and max_index represents the index position.

[0190] S4. Train the medical entity disambiguation model: Train the medical entity disambiguation model constructed in step S3 on the training dataset of the medical entity disambiguation model obtained in step S2. As shown in the appendix Figure 6 as follows, the specific steps are as follows:

[0191] S401. Construct the loss function: As known from step S304, y i is the score of each candidate medical entity processed by the medical entity disambiguation model. y is the true label, representing the correct candidate medical entity corresponding to the medical entity mention. Here, cross-entropy is used as the loss function. The specific formula is as follows:

[0192]

[0193] S402. Optimize the model: Use the AdamW optimization function as the optimization function of this model, and the learning rate is 1e-5. On the training dataset, optimize and train the medical entity disambiguation model. When the model of this method has not been fully trained, it is necessary to further perform step S4 on the training dataset for training to optimize the model parameters. When the model training is completed, the score of each candidate medical entity can be predicted by step S304. According to the score, the model selects the correct medical entity;

[0194] For example: In Pytorch, the code implementation described above is as follows:

[0195] loss_function = nn.CrossEntropyLoss()

[0196] optimizer = optim.AdamW(entity_disambiguation_model.parameters(), lr = 1e-5)

[0197] num_epochs = 10

[0198] for epoch in range(num_epochs):

[0199] optimizer.zero_grad()

[0200] y_pred = entity_disambiguation_model(y_scores)

[0201] loss = loss_function(y_pred.unsqueeze(0), y_true.unsqueeze(0))

[0202] loss.backward()

[0203] optimizer.step()

[0204] print(f"Epoch:{epoch + 1}, Loss:{loss.item()}")

[0205] y_pred = entity_disambiguation_model(y_scores)

[0206] predicted_label = torch.argmax(y_pred).item()

[0207] Among them, CrossEntropyLoss() represents the binary cross-entropy loss function; lr represents the learning rate, which is set to 1e-5 here; num_epochs represents the number of training rounds of the model, which is set to 10 here; backward() represents the backpropagation function; y_pred represents the scores of each candidate medical entity; y_true represents the true label;

[0208] The model proposed by the present invention has achieved better results than the current advanced models on the BC5CDR dataset. The comparison of the experimental results is shown in Table 1 specifically,

[0209] Table 1: Experimental Results on BC5CDR

[0210] Method Precision Recall F1-score Cross Encoder 83.6 84.3 85.8 Dual Encoder 86.4 89.5 87.1 LATTE 89.69 88.48 87.33 Our model 90.0 88.9 88.4

[0211] The model of the present invention is compared with the existing models. The experimental results show that the method of the present invention has been greatly improved. Among them, the first three rows are the experimental results of the models of the existing methods, and the last row is the experimental result of the model of the present invention. It can be seen from this that the present invention has a greater improvement than the existing models.

[0212] Example 3:

[0213] As shown in the appendix Figure 3 shown, based on the medical entity disambiguation device of relationship fusion and information fusion in Example 2, this device includes,

[0214] The medical entity disambiguation knowledge base construction unit, the medical entity disambiguation model dataset construction unit, the medical entity disambiguation model construction unit, and the medical entity disambiguation model training unit respectively implement the functions of steps S1, S2, S3, and S4 in the medical entity disambiguation method based on relationship fusion and information fusion. The specific functions of each unit are as follows:

[0215] The medical entity disambiguation knowledge base construction unit downloads the medical datasets published on the official website as the original data for medical entity disambiguation.

[0216] The medical entity disambiguation model dataset construction unit preprocesses the downloaded original dataset, converts the data into json format data, and obtains the training dataset for the model.

[0217] The medical entity disambiguation model construction unit is used to construct an encoding module, a knowledge fusion module, an interaction module, and a matching module.

[0218] The medical entity disambiguation model training unit is used to construct the loss function and optimization function required during model training and complete the training of the model.

[0219] Example 4:

[0220] Based on the storage medium of Example 2, which stores multiple instructions. The instructions are loaded by the processor and execute the steps of the medical entity disambiguation method based on relationship fusion and information fusion in Example 2.

[0221] Example 5:

[0222] Based on the electronic device of Example 4, the electronic device includes: the storage medium of Example 4; and

[0223] A processor for executing the instructions in the storage medium of Example 4.

[0224] Finally, it should be noted that: the above embodiments are only used to illustrate the technical solutions of the present invention, rather than to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those of ordinary skill in the art should understand that: they can still modify the technical solutions described in the foregoing embodiments, or perform equivalent replacements on some or all of the technical features; and these modifications or replacements do not make the essence of the corresponding technical solutions deviate from the scope of the technical solutions of the embodiments of the present invention.

Claims

1. A medical entity disambiguation method based on relationship fusion and information fusion, characterized in that, The method is as follows: S1 Construct a medical entity disambiguation knowledge base: Download the medical datasets published on the official website as the original data for medical entity disambiguation; S2 Construct a medical entity disambiguation model dataset: Preprocess the original dataset obtained in S1 and convert the data into json format data; S3 Construct a medical entity disambiguation model: Construct an encoding module, a knowledge fusion module, an interaction module, and a matching module; Among them, the encoding module converts the preprocessed data into token vectors, that is, converts the context of the medical entity mention and the candidate medical entity into token vectors, and then uses an encoder to encode the token vectors respectively to obtain the context representation of the medical entity mention and the representation of the candidate medical entity; The knowledge fusion module includes a relationship fusion module and an information fusion module. Among them, the relationship fusion module first retrieves all mentions m = {m1,..., m N} in the context C of the medical entity mentions in the knowledge graph, and obtains all relationships R = {r1,..., r b} retrieved from the knowledge graph. Then, all relationships R are encoded through a relational graph convolutional neural network to obtain the relationship information representation between medical entity mentions Finally, the context representation H c of the obtained medical entity mentions is concatenated with the relationship information representation H r between medical entity mentions, and then the enhanced context representation H c,r of the medical entity mentions is obtained through a linear layer. The subscript N represents the number of medical mentions, the subscript b represents the number of relationships, and the superscript r represents the relationship. The information fusion module first retrieves and encodes the candidate medical entities in the knowledge base to obtain the synonym information representation H s of the candidate medical entities. Then, the obtained representation H e of the candidate medical entities is fused with the synonym information representation H s of the candidate medical entities through an attention mechanism to obtain the representation H e,s of the candidate medical entities with the fused information. Again, the candidate medical entities are retrieved and encoded in the knowledge base to obtain the type representation information H t of the candidate medical entities. Then, the representation H e,s of the candidate medical entities with the fused information is concatenated with the type information representation H t of the candidate medical entities to obtain the enhanced representation H e,s,t of the candidate medical entities. The superscript s represents synonyms, and the superscript t represents types; The interaction module takes the context representation H of the obtained enhanced medical entity mentions c,r and the representation H of the obtained enhanced candidate medical entities e,s,t as inputs. First, calculate the similarity matrix S between the two, and then calculate the interaction between the two through the cross-attention mechanism to obtain the interaction representation A; The matching module takes the obtained interaction representation A as input, obtains the matching score of each candidate medical entity through dot product operation and uses the softmax function, and selects the candidate medical entity with the highest score; S4 Train the medical entity disambiguation model: Train the medical entity disambiguation model constructed in step S3 on the training dataset of the medical entity disambiguation model obtained in step S2.

2. The medical entity disambiguation method based on relationship fusion and information fusion according to claim 1, characterized in that The construction process of the encoding module is as follows: First, a medical entity mention m k ={c i ,..., c j} and its context C = {c1,..., c l} and candidate medical entities E = {e1,..., e m} are respectively transformed into token vectors by the tokenizer of BioBERT, that is Then, the BioBERT encoder is used to encode them respectively to obtain the context representation H c of the medical entity mention and the representation H e of the candidate medical entity; the subscript k represents the k-th medical entity mention; the subscripts i and j represent the start and end positions of the words of the medical entity mention respectively; the subscript l represents the word length of the context of the medical entity mention; the subscript m represents the number of candidate medical entities; the specific formula is described as follows: Among them, the subscript L in formula (1) represents the length of the token vector; the superscript c represents the context; the M in formula (2) represents the length of the token vector; the superscript e represents the entity.

3. The medical entity disambiguation method based on relationship fusion and information fusion according to claim 1, characterized in that Construct a relationship fusion module, and the specific formula description is as follows: R = GE(m) (3) H c,r = W[H c ; H r + b (5) Among them, GE represents the embedding lookup table of the knowledge graph, and retrieves the set of relationships of all candidate medical entities in the knowledge graph; represents the relationship v between medical entities i is the hidden state representation of the l-th layer of the relational graph convolutional neural network; σ represents the sigmoid activation function; W r (l-1) and W0 (l-1) are learnable parameters; represents the set of neighbor indices of the i-th medical entity mention in the relationship R; represents the normalization constant; W and b represent learnable parameters; other symbols have been described before and will not be repeated; Construct an information fusion module, and the specific formula description is as follows: H e,s,t = [H e,s ; H t (7) Among them, Attention in formula (6) represents the attention mechanism; softmax represents the normalized exponential function; Q represents the candidate medical entity query matrix, which is obtained by multiplying the input matrix formed by the query vector q of the candidate medical entity with the trainable parameter matrix and through linear transformation; K represents the synonym matrix of the candidate medical entity, which is obtained by multiplying the input matrix formed by the synonym vector k of the candidate medical entity with the trainable parameter matrix and through linear transformation; V represents the value matrix of the candidate medical entity, which is obtained by multiplying the input matrix formed by the value vector v of the candidate medical entity with the trainable parameter matrix and through linear transformation; the query vector q of the candidate medical entity is the token vector of H e ; the synonym vector k of the candidate medical entity is the token vector of H s ; the value vector of the candidate medical entity is the token vector corresponding to the candidate medical entity query vector v; d k represents the vector length; the symbols in formula (7) have been described before and will not be repeated here.

4. The medical entity disambiguation method based on relationship fusion and information fusion according to claim 1, wherein The specific formula description of the interaction module is as follows: A = [a1,..., a u (13) Among them, s in formula (8) ij represents the element of the similarity matrix S. In the subscript ij, i represents the i-th token vector of the context representation of the enhanced medical entity mention, and j in the subscript ij represents the j-th token vector of the representation of the enhanced candidate medical entity; represents the learnable parameter; respectively represent the context representation H of the enhanced medical entity mention c,r and the representation H of the enhanced candidate medical entity e,s,t of the elements; ⊙ represents the dot product operation; S in formula (9) α represents the attention matrix of H c,r focusing on H e,s,t where row represents performing softmax on the rows of the similarity matrix S; S in formula (10) β represents the attention matrix of H e,s,t focusing on H c,r where col represents performing softmax on the columns of the similarity matrix S; in formula (11) represents the attention score of H c,r focusing on H e,s,t ; represents the attention score of H e,s,t focusing on H c,r where the subscript j represents the j-th token vector of the representation of the enhanced candidate medical entity; a in formula (12) z represents the z-th interactive representation; A in formula (13) is the set of the obtained interactive representations, and the subscript u represents the number of interactive representations; other symbols have been described before and will not be repeated here.

5. The medical entity disambiguation method based on relationship fusion and information fusion according to claim 1, wherein The specific formula description of the matching module is as follows: y i = softmax(a j T ·a i ) (14) Among them, y in formula (14) i represents the matching score of each candidate medical entity; a j represents the context representation of the j-th medical entity mention in the interaction; a i represents the representation of the i-th candidate medical entity in the interaction.

6. The medical entity disambiguation method based on relationship fusion and information fusion according to claim 1, characterized in that Train the medical entity disambiguation model, as follows: Construct the loss function: y i is the score of each candidate medical entity obtained after being processed by the medical entity disambiguation model, and y is the true label, representing the correct candidate medical entity corresponding to the medical entity mention; here, cross-entropy is used as the loss function; the specific formula is as follows: Construct an optimization function: Use the AdamW optimization function as the optimization function. Except that its learning rate is set to 1e-5, other hyperparameters of AdamW are selected with the default values in pytorch; When the medical entity disambiguation model has not been trained, it needs to be further trained on the training dataset to optimize the parameters of the medical entity disambiguation model; when the medical entity disambiguation model is trained, the scores of each candidate medical entity can be predicted, and according to the scores, the model selects the correct medical entity.

7. A medical entity disambiguation device based on relationship fusion and information fusion, characterized in that, The device includes a medical entity disambiguation knowledge base construction unit, a medical entity disambiguation model dataset construction unit, a medical entity disambiguation model construction unit, and a medical entity disambiguation model training unit, which respectively implement the steps of the medical entity disambiguation method based on relationship fusion and information fusion described in any one of claims 1-6.

8. A storage medium storing multiple instructions, characterized in that, The instructions are loaded by the processor and execute the steps of the medical entity disambiguation method based on relationship fusion and information fusion described in any one of claims 1-6.

9. An electronic device, characterized in that, The electronic device includes: The storage medium described in claim 8; and a processor for executing the instructions in the storage medium.

Images