Small sample medical knowledge graph completion method based on implicit relationship attention network

Abstract

A small sample medical knowledge graph completion method based on implicit relation attention network, first, the feature of task medical relation is extracted in the medical knowledge graph, and the information of one-hop neighbor is captured, the enhanced embedding of task medical entity is realized, then the similarity and dynamic relation embedding of head and tail entity neighbor relation embedding of task medical entity are obtained, finally, the two are fused to obtain more semantic fine-grained relation representation, the score between the head entity, the relation and the tail entity is calculated to realize the completion of the medical knowledge graph. The invention considers the neighbor information and relation embedding information of the medical knowledge graph, and fuses them, in the small sample scene, the completion effect is high in correctness and good in effect.

Description

Technical Field

[0001] This invention relates to the field of medical knowledge graphs, and in particular to a method for completing small-sample medical knowledge graphs based on implicit relational attention networks. Background Technology

[0002] With the rapid development of medical fields, an increasing amount of cutting-edge medical data and knowledge is being generated. Knowledge graphs can integrate medical data and knowledge to form a structured, graphical representation of knowledge. Current medical knowledge graphs cover various medical topics, such as treatment methods, drugs, and diagnostic methods. However, current medical knowledge graphs also have many problems, such as diverse terminology, fragmented knowledge, and difficulties in maintenance. Therefore, constructing a more effective medical knowledge graph has become a new challenge.

[0003] Medical knowledge graph completion uses automated methods to fill in missing information in a knowledge graph, thereby improving its structure, enhancing knowledge usability, and discovering new connections. In current medical knowledge graphs, different types of relationships often exhibit a significant long-tail distribution, meaning that many relationships have only a small number of entity pairs. Traditional completion methods fail to achieve satisfactory results. Therefore, researching small-sample medical knowledge graph completion has become an urgent need. Summary of the Invention

[0004] To overcome the shortcomings of current medical knowledge graph completion methods, such as low accuracy and low efficiency in small sample scenarios, this invention proposes a small sample medical knowledge graph completion method based on implicit relational attention networks. This method considers and fuses the neighbor information and relational embedding information of the medical knowledge graph, achieving high accuracy and good results in small sample scenarios.

[0005] The technical solution adopted by this invention to solve its technical problem is:

[0006] A few-sample medical knowledge graph completion method based on implicit relation attention network is proposed. First, feature extraction of task medical relations is performed in the medical knowledge graph, and information of one-hop neighbors is captured to achieve enhanced embedding of task medical entities. Then, the similarity of head and tail entity neighbor relationship embeddings and dynamic relation embedding are obtained for task medical entities. Finally, the two are fused to obtain a more semantically fine-grained relation representation. Medical knowledge graph completion is achieved by calculating the scores between head entities, relations and tail entities.

[0007] Furthermore, the method includes the following steps:

[0008] Step 1: A medical knowledge graph Depend on Three pairs Composition, in which, , , , These represent the head entities, relationships, and tail entities in the medical knowledge graph. This represents a collection of entities that include drugs, diseases, symptoms, medical devices, transmission routes, examination methods, and treatments. Represents the set of inherent relationships between entities. and These represent the number of entities and relations, respectively; the relations corresponding to a small sample task are: , targeting medical entities Its first-order neighbor is represented as ,in, Representative background image, For the relationship between medical entities and their neighbors, For neighboring entities;

[0009] Step 2: Use the TransE algorithm to obtain a medical knowledge graph Initial entity embedding vector set and relation embedding vector set ;

[0010] Step 3: For medical knowledge graphs Small sample task relationship Through the relationship of medical small sample tasks Forward entity extraction Each entity pair forms a reference set. Extract from the remaining positive entity pairs Each entity pair forms a positive query set. Then, by randomly selecting from the entity set, a negative query set is formed. The corresponding vector satisfies the condition ; , and All entities in the task are called task-medical entities, and all small-sample task relationships and entity-neighbor relationships are called task-medical relationships.

[0011] Step 4: Randomly select a task-medical relationship Feature extraction, residual connections, and normalization are performed to obtain enhanced feature relation embeddings.

[0012] ;

[0013] Including enhanced task relationship embedding and embedding of enhanced entity neighbor relationships , Includes enhanced head entity neighbor relationship embedding and enhanced tail entity neighbor relationship embedding ,in, For GeLU activation function, and All parameters are learnable; traverse all task-medical relationships and compute the enhanced feature relationship embeddings of the task-medical relationships;

[0014] Step 5: Select any medical entity for the task. Calculate the corresponding enhanced task relation embedding and embedding of enhanced entity neighbor relationships similarity

[0015] ;

[0016] Traverse all task-medical entity pairs and calculate the similarity score between the corresponding enhanced task relation embedding and the enhanced entity neighbor relation embedding;

[0017] Step 6: Randomly select a medical entity for the task. Computational Neighbor-Aware Embedding

[0018] ;

[0019] ;

[0020] Iterate through all medical entities in the task and calculate the neighbor-aware embeddings between all different medical entities and their one-hop neighbors;

[0021] Step 7: Randomly select a medical entity for the task. Embedded medical entities and entity neighbor perception embedding Coupled to obtain enhanced embeddings of task medical entities

[0022] ;

[0023] in, For ReLU functions, These are learnable parameters; traverse all task-specific medical entities and compute the enhanced embeddings of all task-specific medical entities;

[0024] Step 8: Randomly select a pair of task medical entities from the medical knowledge graph. Calculate the similarity of head and tail entity neighbor relationship embeddings.

[0025] ;

[0026] By traversing all task medical entity pairs and calculating the similarity of the head and tail entity neighbor relationship vectors, the query set neighbor relationship similarity set can be obtained. Relevance set of neighbor relationships with reference set ;

[0027] Step 9: Randomly select a pair of task medical entities from the medical knowledge graph. Each entity pair and its relation is used as the input sequence of the Transformer encoder. ,in, These represent head entity augmentation embedding and tail entity augmentation embedding, respectively. The similarity of the head and tail entity neighbor vectors is used to calculate the embedding vector containing location information.

[0028] ;

[0029] in Represents positional encoding;

[0030] Step 10: Calculate the enhanced triplet vector sequence

[0031] ;

[0032] in, Indicates after the first Layer Transformer Blocks The hidden state, the final hidden state As the dynamic relation embedding of the triples; traverse all task medical entity pairs, repeat steps 9-10, and calculate their corresponding dynamic relation embeddings to obtain the query set triple dynamic relation embedding set. Dynamic Relationship Embedded Set of Reference Set Triples ;

[0033] Step 11: In the query set Choose any triplet In the reference set Choose any triplet Calculate the embedding correlation of the dynamic triplet relationship between the two. Correlation with the similarity of neighbor relationships between the two ,in

[0034] ;

[0035] ;

[0036] Traverse the query set and reference set, and calculate the dynamic relation embedding relevance of each query set element to the reference set element and the relevance of their neighbor relations similarity.

[0037] Step 12: In the query set Choose any triplet Calculate its relation embedding Relevance embedding of neighbor relationships ,in

[0038] ;

[0039] ;

[0040] in

[0041] ;

[0042] ;

[0043] ;

[0044] Traverse the query set and reference set to calculate the relation embedding and neighbor relation relevance embedding of different query triples;

[0045] Step 13: In the query set Choose any triplet Calculate its fusion relationship embedding

[0046] ;

[0047] in, For learnable parameters, Adjustments are made based on the degree of loss of neighbor information; the higher the relevance of neighbor information, the better. The higher the value, the lower the relevance of neighbor information. The smaller the value, the more likely it is to iterate through the query set and calculate all fusion relationship embeddings.

[0048] Step 14: In the query set Choose any triplet Calculate the semantic similarity between it and the corresponding fusion relationship embedding.

[0049] ;

[0050] Iterate through all query entity pairs and calculate the semantic similarity between all query triples and the corresponding fusion relation embeddings;

[0051] Step 15: Calculate the loss:

[0052] ;

[0053] in, It is the safety boundary distance, achieved by minimizing... This aims to maximize the similarity score between the positive sample query set and the reference set.

[0054] Step 16: Update the model parameters using stochastic gradient descent; repeat steps 4 to 16, when... The calculation ends when the value falls below the specified minimum loss.

[0055] Step 17: For tasks with a relationship The missing triples in the medical knowledge graph ,in, To represent the missing tail entity, calculate by traversing the entity set in the medical knowledge graph. Fractions, of which This represents any entity in the medical entity set. Then, the scores are sorted, and the medical entity with the highest score is used as the tail entity to complete the missing triple.

[0056] Step 18: Traverse all triples in the medical knowledge graph and repeat step 17 to obtain a complete medical knowledge graph.

[0057] The technical concept of this invention is as follows: In medical knowledge graphs, for small sample task relationships, by mining the similarity of neighbor relationships between triple entity pairs, and integrating triple dynamic relationship embedding and entity pair neighbor relationship correlation information, the relationship embedding of the query set to the reference set is enhanced, thereby improving the accuracy of small sample knowledge graph completion.

[0058] The beneficial effects of this invention are: it can enrich the relationship embedding information from two aspects, namely, the dynamic relationship embedding of triples and the correlation information of entity-neighbor relationships, so as to achieve high accuracy and good effect in medical knowledge graph completion. Attached Figure Description

[0059] Figure 1 This is a schematic diagram of a few-sample medical knowledge graph completion method based on implicit relation attention networks. Solid circles represent entities in the medical knowledge graph, hollow circles represent one-hop neighbors of entities, and arrows between solid and hollow circles indicate the relationship between them. The task relationship is defined by… This indicates that the solid circles connected by the dashed lines are... Represents a complete medical entity. Detailed Implementation

[0060] The invention will now be further described with reference to the accompanying drawings.

[0061] Reference Figure 1A few-sample medical knowledge graph completion method based on implicit relation attention network is proposed. First, feature extraction of task medical relations is performed in the medical knowledge graph, and information of one-hop neighbors is captured to realize enhanced embedding of task medical entities. Then, the similarity of head and tail entity neighbor relationship embeddings and dynamic relationship embedding are obtained for task medical entities. Finally, the two are fused to obtain a more semantically fine-grained relation representation. By calculating the scores between head entities, relations and tail entities, medical knowledge graph completion is achieved.

[0062] The method includes the following steps:

[0063] Step 1: A medical knowledge graph Depend on Three pairs Composition, in which, , , , These represent the head entities, relationships, and tail entities in the medical knowledge graph. This represents a collection of entities that include drugs, diseases, symptoms, medical devices, transmission routes, examination methods, and treatments. Represents the set of inherent relationships between entities. and These represent the number of entities and relations, respectively; the relations corresponding to a small sample task are: , targeting medical entities Its first-order neighbor is represented as ,in, Representative background image, For the relationship between medical entities and their neighbors, For neighboring entities;

[0064] Step 2: Use the TransE algorithm to obtain a medical knowledge graph Initial entity embedding vector set and relation embedding vector set ;

[0065] Step 3: For medical knowledge graphs Small sample task relationship Through the relationship of medical small sample tasks Forward entity extraction Each entity pair forms a reference set. Extract from the remaining positive entity pairs Each entity pair forms a positive query set. Then, by randomly selecting from the entity set, a negative query set is formed. The corresponding vector satisfies the condition ; , and All entities in the task are called task-medical entities, and all small-sample task relationships and entity-neighbor relationships are called task-medical relationships.

[0066] Step 4: Randomly select a task-medical relationship Feature extraction, residual connections, and normalization are performed to obtain enhanced feature relation embeddings.

[0067] ;

[0068] Including enhanced task relationship embedding and embedding of enhanced entity neighbor relationships , Includes enhanced head entity neighbor relationship embedding and enhanced tail entity neighbor relationship embedding ,in, For GeLU activation function, and All parameters are learnable; traverse all task-medical relationships and compute the enhanced feature relationship embeddings of the task-medical relationships;

[0069] Step 5: Select any medical entity for the task. Calculate the corresponding enhanced task relation embedding and embedding of enhanced entity neighbor relationships similarity

[0070] ;

[0071] Traverse all task-medical entity pairs and calculate the similarity score between the corresponding enhanced task relation embedding and the enhanced entity neighbor relation embedding;

[0072] Step 6: Randomly select a medical entity for the task. Computational Neighbor-Aware Embedding

[0073] ;

[0074] ;

[0075] Iterate through all medical entities in the task and calculate the neighbor-aware embeddings between all different medical entities and their one-hop neighbors;

[0076] Step 7: Randomly select a medical entity for the task. Embedded medical entities and entity neighbor perception embedding Coupled to obtain enhanced embeddings of task medical entities

[0077] ;

[0078] in, For ReLU functions, These are learnable parameters; traverse all task-specific medical entities and compute the enhanced embeddings of all task-specific medical entities;

[0079] Step 8: Randomly select a pair of task medical entities from the medical knowledge graph. Calculate the similarity of head and tail entity neighbor relationship embeddings.

[0080] ;

[0081] By traversing all task medical entity pairs and calculating the similarity of the head and tail entity neighbor relationship vectors, the query set neighbor relationship similarity set can be obtained. Relevance set of neighbor relationships with reference set ;

[0082] Step 9: Randomly select a pair of task medical entities from the medical knowledge graph. Each entity pair and its relation is used as the input sequence of the Transformer encoder. ,in, These represent head entity augmentation embedding and tail entity augmentation embedding, respectively. The similarity of the head and tail entity neighbor vectors is used to calculate the embedding vector containing location information.

[0083] ;

[0084] in, Represents positional encoding;

[0085] Step 10: Calculate the enhanced triplet vector sequence

[0086] ;

[0087] in, Indicates after the first Layer Transformer Blocks The hidden state. The final hidden state. As the dynamic relation embedding of the triples; traverse all task medical entity pairs, repeat steps 9-10, and calculate their corresponding dynamic relation embeddings to obtain the query set triple dynamic relation embedding set. Dynamic Relationship Embedded Set of Reference Set Triples ;

[0088] Step 11: In the query set Choose any triplet In the reference set Choose any triplet Calculate the embedding correlation of the dynamic triplet relationship between the two. Correlation with the similarity of neighbor relationships between the two ,in

[0089] ;

[0090] ;

[0091] Traverse the query set and reference set, and calculate the dynamic relation embedding relevance of each query set element to the reference set element and the relevance of their neighbor relations similarity.

[0092] Step 12: In the query set Choose any triplet Calculate its relation embedding Relevance embedding of neighbor relationships ,in

[0093] ;

[0094] ;

[0095] in

[0096] ;

[0097] ;

[0098] ;

[0099] Traverse the query set and reference set to calculate the relation embedding and neighbor relation relevance embedding of different query triples;

[0100] Step 13: In the query set Choose any triplet Calculate its fusion relationship embedding

[0101] ;

[0102] in, For learnable parameters, Adjustments are made based on the degree of loss of neighbor information; the higher the relevance of neighbor information, the better. The higher the value, the lower the relevance of neighbor information. The smaller the value, the more likely it is to iterate through the query set and calculate all fusion relationship embeddings.

[0103] Step 14: In the query set Choose any triplet Calculate the semantic similarity between it and the corresponding fusion relationship embedding.

[0104] ;

[0105] Iterate through all query entity pairs and calculate the semantic similarity between all query triples and the corresponding fusion relation embeddings;

[0106] Step 15: Calculate the loss:

[0107] ;

[0108] in, It is the safety boundary distance, achieved by minimizing... This aims to maximize the similarity score between the positive sample query set and the reference set.

[0109] Step 16: Update the model parameters using stochastic gradient descent; repeat steps 4 to 16, when... The calculation ends when the value falls below the specified minimum loss.

[0110] Step 17: For tasks with a relationship The missing triples in the medical knowledge graph ,in, To represent the missing tail entity, calculate by traversing the entity set in the medical knowledge graph. Fractions, of which Let each entity in the medical entity set represent any entity. Then, sort the scores and select the medical entity with the highest score as the tail entity to complete the missing triples. Figure 1 As shown, the head entity is represented by a solid circle. Relationship with the task indicated by the dashed line The highest-scoring tail entity was calculated. Finally, the medical triad was completed. ;

[0111] Step 18: Traverse all triples in the medical knowledge graph and repeat step 17 to obtain a complete medical knowledge graph.

[0112] This embodiment considers and fuses neighbor information and relationship embedding information from the medical knowledge graph. In small sample scenarios, the completion effect is highly accurate and effective.

[0113] The embodiments described in this specification are merely examples of implementations of the inventive concept and are for illustrative purposes only. The scope of protection of this invention should not be considered limited to the specific forms described in these embodiments; rather, it extends to equivalent technical means conceived by those skilled in the art based on the inventive concept.

Claims

1. A small sample medical knowledge graph completion method based on implicit relationship attention network, characterized in that, The method comprises the following steps: Step 1: The medical knowledge graph is composed of triples, and the entities of the triples include drugs, diseases, symptoms, medical devices, transmission routes, examination methods and treatment methods; Step 2: Obtain the initial entity embedding vector set and the relationship embedding vector set of the medical knowledge graph; Step 3: Extract entity pairs from the positive entity pairs of the medical small sample task relationship to form a reference set and a positive query set, and extract from the entity set to form a negative query set, all entities of the above sets are called medical entities, and all small sample task relationships and entity neighbor relationships are called medical relationships; Step 4: Calculate the enhanced feature relationship embedding of all medical relationships, including enhanced task relationship embedding and enhanced entity neighbor relationship embedding; Step 5: Calculate the similarity score of the enhanced task relationship embedding and the enhanced entity neighbor relationship embedding of all medical entity pairs; Step 6: Calculate the neighbor-aware embedding between all medical entities and one-hop neighbors; Step 7: Coupling the medical entity embedding and the entity neighbor-aware embedding to obtain the enhanced embedding of all medical entities; Step 8: Calculate the similarity of the head and tail entity neighbor relationship vectors to obtain a query set neighbor relationship similarity set and a reference set neighbor relationship similarity set; Step 9: Take each entity pair and its relationship as an input sequence of the encoder to calculate an embedding vector containing position information; Step 10: Calculate the dynamic relationship embedding of all medical entity pairs to obtain a query set triple dynamic relationship embedding set and a reference set triple dynamic relationship embedding set; Step 11: Calculate the triple dynamic relationship embedding relevance and the neighbor relationship similarity relevance of each query set element to the reference set element; Step 12: Calculate the relationship embedding and the neighbor relationship relevance embedding of different query triples; Step 13: Traverse the query set to calculate the fusion relationship embedding according to the relationship embedding and the neighbor relationship relevance embedding; Step 14: Calculate the semantic similarity of all query triples and the corresponding fusion relationship embedding; Step 15: Calculate the loss value; Step 16: Train the model using the stochastic gradient descent method; Step 17: For the missing triples, traverse the entity set in the medical knowledge graph, calculate the semantic similarity, and take the highest medical entity to complete the tail entity of the missing triple; Step 18: Traverse all triples in the medical knowledge graph, repeat step 17 to obtain a complete medical knowledge graph.

2. The small sample medical knowledge graph completion method based on implicit relationship attention network according to claim 1, wherein, The triplets in step 1 are , , , , respectively represent the head entity, relation and tail entity in the medical knowledge graph, represents the entity set containing drugs, diseases, symptoms, medical devices, transmission routes, examination methods and treatment methods, represents the set of internal association relationships between entities, and respectively represent the number of entities and relations; the corresponding relation of a small sample task is , for medical entities , the first-order neighbor is represented as , wherein represents the background graph, is the relationship between the medical entity and the neighbor, is the neighbor entity; In step 2, the initial entity embedding vector set of the medical knowledge graph is obtained using the TransE algorithm ;​​ In step 3, for medical knowledge graphs Small sample task relationship Through the relationship of medical small sample tasks Forward entity extraction Each entity pair forms a reference set. Extract from the remaining positive entity pairs Each entity pair forms a positive query set. Then, by randomly selecting from the entity set, a negative query set is formed. The corresponding vector satisfies the condition ; Step 4: Arbitrarily select one medical relation Feature extraction, residual connection and normalization operation are performed to obtain enhanced feature relation embedding ； including enhanced task relation embeddings and enhanced entity neighbor relation embeddings , including enhanced head entity neighbor relation embeddings and enhanced tail entity neighbor relation embeddings wherein, is a gelu activation function, and are learnable parameters; traversing all medical relations, computing enhanced feature relation embeddings of the medical relations; In step 5, an arbitrary medical entity is selected , the similarity of the corresponding enhanced task relation embedding and enhanced entity neighbor relation embedding is calculated ； In step 6, any one medical entity is selected Compute neighbor-aware embeddings: ； ； In step 7, an arbitrary medical entity is chosen Coupling the medical entity embedding and the entity neighbor-aware embedding to obtain an enhanced embedding of the medical entity: ； wherein, is a Relu function, is a learnable parameter; In step 8, a pair of medical entities in the medical knowledge graph is selected at random , the similarity of the head and tail entity neighbor relationship embedding is calculated: ； In step 9, a pair of medical entities in the medical knowledge graph is selected at will Each entity pair and its relationship is taken as an input sequence of a Transformer encoder wherein, represent the head entity enhanced embedding and the tail entity enhanced embedding, respectively, represent the similarity of the head-tail entity neighbor relationship vector, and the embedding vector containing position information is calculated ； wherein represents a position encoding; In step 10, the enhanced triple vector sequence is calculated: ； wherein, denotes the passage through the layer Transformer Blocks hidden state of the last hidden state dynamic relation embedding as a triple; traversing all medical entity pairs, repeating steps 9-10, calculating the corresponding dynamic relation embedding, can obtain the query set of triple dynamic relation embedding set and the reference set of triple dynamic relation embedding set ; In step 11, in the query set arbitrarily select one triple , in the reference set arbitrarily select one triple , calculate the correlation of the triple dynamic relation embedding of the two and the correlation of the neighbor relation similarity of the two , wherein ​ ； ； Traverse the query set and the reference set to calculate the triple dynamic relationship embedding relevance and the neighbor relationship similarity relevance of each query set element to the reference set element; In step 12, for each query set An arbitrary triple is chosen , its relation embedding is computed and neighbor relation relevance embedding where ； ； Wherein ； ； ； Traverse the query set and the reference set to calculate the relationship embedding and the neighbor relationship relevance embedding of different query triples; In step 13, the query set An arbitrary triple is selected , and its fused relation embedding is calculated ； wherein, is a learnable parameter, , according to the neighbor information loss situation, when the neighbor information correlation is higher, the value is larger, the neighbor information correlation is lower, the value is smaller; In step 14, the semantic similarity between the query set Any one of the triplets is selected is calculated with the semantic similarity of the corresponding fusion relationship embedding ； Traverse all query entity pairs to calculate the semantic similarity of all query triples and the corresponding fusion relationship embedding; In step 15, the loss is calculated: ； wherein, is a safety margin distance, by minimizing to make the similarity score of the positive sample query set with the reference set as high as possible; In the step 16, the model parameters are updated by using a stochastic gradient descent method; and the steps 4 to 16 are repeated until The calculation is ended when the loss value is less than a specified minimum loss value. In step 17, the tail entity of the missing triple is determined by calculating the semantic similarity score between the missing tail entity and each entity in the entity set of the medical knowledge graph, and then ranking the scores and selecting the entity with the highest score as the tail entity of the missing triple. ​​​​​

Images