A method for automatically constructing medical common sense knowledge graphs based on meta-learning

Abstract

The present invention discloses a method for automatically constructing a medical common sense knowledge graph based on meta-learning, comprising: 1) obtaining an existing medical common sense knowledge graph and constructing medical common sense knowledge triples as training data for the model; 2) splitting the medical common sense knowledge triple data into background knowledge graphs; 3) constructing an automated construction model for the medical common sense knowledge graph and selecting a loss function to optimize the model parameters and feature vectors; 4) using the automated construction model for the medical common sense knowledge graph to score and predict the likelihood of future interactions between head and tail entities that have not interacted, and finally obtaining the knowledge construction results in the order of scores. The present invention adopts a knowledge graph construction technology based on meta-learning, and by effectively learning the vector representation of medical common sense knowledge triples, it enables the model to accurately generate new medical common sense knowledge using only a small amount of training data, thereby making contributions in the fields of medical diagnosis, clinical decision-making, and knowledge question-answering.

Description

Technical Field

[0001] The present invention belongs to the field of natural language processing technology, and specifically is a method for automatically constructing a medical common sense knowledge graph based on meta-learning. Background Art

[0002] In today's world, with the advent of the information age, massive amounts of data have been generated, and knowledge graphs have become an effective tool for integrating and representing domain knowledge. A knowledge graph is a method of representing knowledge that organizes knowledge into graphs in the form of triples (head entity, relationship, tail entity), making the connections between knowledge clearly visible. In the medical field, building a medical knowledge graph can help doctors, researchers, and decision makers better understand and utilize medical knowledge. The main application directions of medical knowledge graphs are semantic search, knowledge question answering, and clinical decision support. At the same time, knowledge graphs are gradually showing their advantages in some new fields, such as assisting drug development and public health event response. However, the traditional method of manually constructing knowledge graphs requires a lot of manpower and time, and it is difficult to keep up with the update and expansion of knowledge in the face of the ever-growing medical literature and clinical data.

[0003] Therefore, people are considering using models to learn knowledge from existing medical common sense knowledge graphs to generate new possible medical common sense knowledge, which is the current hot issue of knowledge graph completion. Knowledge graph completion technology can automatically infer missing knowledge points from incomplete knowledge graphs, thereby better helping people learn common sense knowledge. However, a major problem facing current knowledge graph completion technology is the lack of training data. Knowledge entities and relationships require a large amount of manual annotation to obtain, a process that is very time-consuming and labor-intensive. In addition, due to the continuous updating and expansion of knowledge graphs, new knowledge entities and relationships need to be continuously added, which will also lead to the existence of many unlabeled entities and relationships in the knowledge graph. Therefore, the lack of training data is a major challenge facing current knowledge graph completion technology. To solve this problem, some researchers have proposed semi-supervised learning and remote supervised learning methods, which use existing knowledge graphs and other external knowledge to assist in completing unlabeled entities and relationships. However, it relies too much on labeled data, and once the data is inaccurately labeled, it will affect the effect of knowledge graph completion. In addition, some researchers have proposed methods based on generative adversarial networks and reinforcement learning to expand training data by generating virtual knowledge graph data, but this training method is too complicated. Summary of the Invention

[0004] In order to address the shortcomings of the above-mentioned existing technologies, the present invention proposes a method for automatically constructing medical common sense knowledge graphs based on meta-learning, so that only a small amount of training data can be used to train the model to accurately realize the automatic construction of medical common sense knowledge graphs, and accurately generate new medical common sense knowledge, thereby providing favorable support for medical research, medical education, and medical knowledge management.

[0005] In order to achieve the above-mentioned object, the present invention adopts the following technical solutions:

[0006] The method for automatically constructing a medical knowledge graph based on meta-learning is characterized by the following steps:

[0007] Step 1: Obtain a medical knowledge graph and construct a medical knowledge graph G = {(h, r, t)∈ε×R×ε} based on the entities in the medical knowledge graph and the relationships between the entities, where h represents the head entity node, r represents the relationship node, and t represents the tail entity node; ε and R are the entity node set and the relationship node set corresponding to the medical knowledge graph G, respectively;

[0008] Step 2: Extract some triples from the medical knowledge graph G as the background knowledge graph G′. The background knowledge graph G′ contains some relationship nodes in the relationship node set R. The triples corresponding to the remaining relationship nodes in R are used as the training knowledge graph G. train ;

[0009] Step 3: From the training knowledge graph G train Randomly select the i-th relationship node r i , and select the i-th relationship node r i The corresponding partial triples are used to construct the relationship node r i The support set S i , using the i-th relationship node r i The remaining triples form the query set Q i ; Then the query set Q i All tail entities in are replaced with entities that have no relationship with their head entities, thus constructing a negative query set

[0010] Step 4: Construct an interactive attention encoder and interactively train the support set S i and query set Q i Encode the semantic information of the head and tail entities to generate more discriminative entity pair representations;

[0011] Step 4.1 Encode the entity semantic information to obtain the head entity The final vector representation of and tail entity The final vector representation of

[0012] Step 4.2 Use formula (7) to get the entity pair Vector representation of

[0013]

[0014] In formula (7), and is the parameter to be trained, and tanh is the activation function;

[0015] Step 4.3 Follow the process of step 4.1-step 4.2 to get the information about the i-th relationship node r i The query set Q i Entity pair vector representation

[0016] Step 5: Build an adaptive prototype network to generate the relation prototype representation P by aggregating entity pair vector representations i ;

[0017] Step 5.1 Use formula (8) to get the query entity pair and the j-th reference entity pair in the support set The similarity score α i,j :

[0018]

[0019] In formula (8), ⊙ is the dot product operation, Represents the support set S i The l-th reference entity pair in Representing entity pairs Vector representation of ;

[0020] Step 5.2: Use formula (9) to get the prototype representation P related to the query set. i :

[0021]

[0022] In formula (9), α i,j Is the query entity and support set S i The j-th reference entity pair in Similarity score of

[0023] Step 6: The medical common sense knowledge model is constructed by the interactive attention encoder and the adaptive prototype network. Based on the medical common sense knowledge graph G and the background knowledge graph G′, the medical common sense knowledge model is trained using the back propagation algorithm. At the same time, the loss function L shown in formula (10) is calculated to update the model parameters. When the loss function L continues to decrease until convergence, the optimal medical common sense knowledge model is obtained:

[0024]

[0025] In formula (10), γ is the marginal hyperparameter, For the query set Q i Entity Pairs The vector representation of Negative query set Entity Pairs Vector representation of ; [·] + is the hinge loss function;

[0026] Step 7: Input the existing medical common sense knowledge graph into the optimal medical common sense knowledge model, thereby outputting the credibility probability of the new medical common sense knowledge triples generated by the model, and outputting the medical common sense knowledge triples with higher probabilities, thereby realizing the automatic construction of the medical common sense knowledge graph.

[0027] The method for automatically constructing a medical knowledge graph based on meta-learning according to the present invention is also characterized in that step 4.1 is performed as follows:

[0028] Step 4.1.1 Generate the i-th relationship node r i The support set S i The j-th entity pair in Neighbor in, Represents the header entity neighbors, and Represents the tail entity neighbors, and in, Representation and header entity There are related entities in the background knowledge graph G′, r h Represents the header entity Neighbors with head entity The relationship between Represents the tail entity There are related entities in the background knowledge graph G′, r t Represents the tail entity Neighbors with tail entity the relationship between;

[0029] Step 4.1.2 Counting Header Entities The number of neighbors Set the dimension of the latent semantic feature vector to d and construct it using random initialization method dimensional neighbor relationship vector matrix dimensional neighbor entity vector matrix

[0030] Step 4.1.3 Calculate the head entity Neighbor attention weight matrix

[0031] Generate head entities separately using random initialization and tail entity Vector representation of and

[0032] Use formula (1) to generate the i-th relationship node r i The initialization vector represents Ur i :

[0033]

[0034] Calculate the head entity using formula (2) Neighbor attention weight matrix

[0035]

[0036] In formula (2), are two weight matrices;

[0037] Step 4.1.4 Generate header entity The weighted neighbor vector representation of

[0038] Calculate the head entity using formula (3) The vector matrix of the fused semantic information of the neighbors

[0039]

[0040] In formula (3), tanh is the activation function, || is the connection operation, and are the parameter matrices to be trained;

[0041] Generate the head entity using formula (4) The weighted neighbor vector representation of

[0042]

[0043] In formula (4), is the weight matrix to be trained, V k Is the head entity The vector representation of the fused semantic information of the kth neighbor, α k Is the head entity The kth neighbor of the head entity The attention score is obtained by formula (5);

[0044]

[0045] In formula (5), A q Is the head entity The qth attention weight in the neighbor attention weight matrix, A k Is the head entity

[0046] The kth attention weight in the neighbor attention weight matrix of ;

[0047] Step 4.1.5 Use formula (6) to get the head entity The final vector representation of

[0048]

[0049] In formula (6), are two weight matrices to be trained, and tanh is the activation function;

[0050] Step 4.1.6: Follow the process from step 4.1.2 to step 4.1.5 to get the tail entity The final vector representation of

[0051] The electronic device of the present invention includes a memory and a processor, and is characterized in that the memory is used to store a program that supports the processor to execute the method for automatically constructing the medical common sense knowledge graph, and the processor is configured to execute the program stored in the memory.

[0052] The present invention provides a computer-readable storage medium, and the computer-readable storage medium stores a computer program, which is characterized in that when the computer program is run by a processor, the steps of the method for automatically constructing a medical common sense knowledge graph are executed.

[0053] Compared with the prior art, the present invention has the following beneficial effects:

[0054] 1. To address the lack of medical common sense knowledge, this invention fully utilizes the knowledge information of the existing medical common sense knowledge graph and integrates the semantic information of common sense knowledge into the vector representation. Especially when faced with multiple triples involving different semantic relationships, it can more accurately capture the features related to the medical common sense knowledge that the model needs to generate, thereby achieving the goal of more accurate and targeted generation of medical common sense knowledge;

[0055] 2. To address the problem of insufficient training data, this paper uses a meta-learning-based approach, enabling the learning of useful information using only a small number of common sense knowledge triplets, thereby accurately generating new medical common sense knowledge with high credibility, greatly improving the overall effectiveness of the learning model and thus achieving more effective medical common sense knowledge reasoning and application;

[0056] 3. The present invention designs an interactive attention encoder and an adaptive prototype network to generate vector representations of entity pairs and relational prototypes adapted to different query triples, respectively. This allows for more accurate capture of the underlying relational semantics of entity pairs and enables the model to focus more on features relevant to the triples to be generated, thereby providing more accurate and reliable medical knowledge for problem solving and decision support in practical applications.

[0057] 4. Meta-learning aims to enable the model to quickly adapt to new tasks or new environments through prior knowledge learned from a small number of samples. In the construction of medical common sense knowledge graphs, meta-learning can be used to automatically learn and infer rules and patterns between medical entities and relationships. The present invention can realize the automatic construction of medical common sense knowledge graphs by combining meta-learning algorithms with medical literature, clinical data and expert knowledge. The automatic construction method of medical common sense knowledge graphs based on meta-learning has the following advantages: First, it can greatly reduce the workload of manual work and improve the efficiency of knowledge graph construction. Second, through prior knowledge learned from a small number of samples, it can quickly adapt to new medical fields and new knowledge. Finally, it can improve the accuracy and completeness of the knowledge graph by integrating multiple data sources and expert knowledge. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] Figure 1 Flowchart of the method of the present invention. DETAILED DESCRIPTION

[0059] In this embodiment, a method for automatically constructing a medical common sense knowledge graph based on meta-learning is to represent medical common sense knowledge in the form of triples (head entity, relationship, tail entity), and project the head entity, relationship and tail entity and related background knowledge graph information into a latent semantic vector space with the same dimension, and then model the head entity and tail entity through an interactive attention encoder to generate a vector representation of the entity pair, and finally generate a relationship prototype that adapts to different query triples through an adaptive prototype network, calculate the credibility probability of different new common sense knowledge, thereby achieving the effect of generating new common sense knowledge and realizing the automatic construction of the medical common sense knowledge graph. Specifically, if Figure 1 As shown, the steps are as follows:

[0060] Step 1: Obtain the medical knowledge graph and construct a medical knowledge graph G = {(h, r, t)∈ε×R×ε} based on the entities in the medical knowledge graph and the relationships between the entities. For example, (pertussis, symptoms, chest tightness), where h represents the head entity node, r represents the relationship node, and t represents the tail entity node; ε and R are the entity node set and relationship node set corresponding to the medical knowledge graph G, respectively.

[0061] Step 2: Extract some triples from the medical knowledge graph G as the background knowledge graph G′. The background knowledge graph G′ contains some relationship nodes in the relationship node set R. The triples corresponding to the remaining relationship nodes in R are used as the training knowledge graph G. train ;

[0062] Step 3: From the training knowledge graph G train Randomly select the i-th relationship node r i , and select the i-th relationship node r i The corresponding partial triples are used to construct the relationship node r i The support set S i , using the i-th relationship node r i The remaining triples form the query set Q i ; Then the query set Q i All tail entities in are replaced with entities that have no relationship with their head entities, thus constructing a negative query set

[0063] For example, if the support set is (pertussis, symptoms, chest tightness) and the corresponding query set is (benzene poisoning, symptoms, nausea), the model needs to learn the vector representation of the support set so that, for the head entity and relation (benzene poisoning, symptoms) of the given query set, it can generate the corresponding tail entity: nausea.

[0064] Step 4: Construct an interaction attention encoder that can simulate the interaction between the head and tail entities at both the entity and relation levels. In this way, the resulting entity pair representation incorporates the relevant semantics between the head and tail entities, which is conducive to discovering the underlying semantic relationship of the entity pair and generating more discriminative entity pair representations;

[0065] Step 4.1 Encode entity semantic information:

[0066] Step 4.1.1 Generate the i-th relationship node r i The support set S i The j-th entity pair in Neighbor in, Represents the header entity neighbors, and Represents the tail entity neighbors, and in, Representation and header entity There are related entities in the background knowledge graph G′, r h Represents the header entity Neighbors with head entity The relationship between Represents the tail entity There are related entities in the background knowledge graph G′, r t Represents the tail entity Neighbors with tail entity the relationship between;

[0067] For example, for the entity pair (pertussis, chest tightness) related to symptoms, we can get the neighbors of the head entity pertussis and the relationship between the two: (pertussis, belongs to, acute respiratory infectious disease), (pertussis, course of disease, 2 to 3 months);

[0068] Step 4.1.2 Counting Header Entities The number of neighbors Set the dimension of the latent semantic feature vector to d and construct it using random initialization method dimensional neighbor relationship vector matrix dimensional neighbor entity vector matrix

[0069] Step 4.1.3 Calculate the head entity Neighbor attention weight matrix

[0070] Generate head entities separately using random initialization and tail entity Vector representation of and

[0071] Use formula (1) to generate the i-th relationship node r i The initialization vector represents Ur i :

[0072]

[0073] Calculate the head entity using formula (2) Neighbor attention weight matrix

[0074]

[0075] In formula (2), are two weight matrices;

[0076] Step 4.1.4 Generate header entity The weighted neighbor vector representation of

[0077] Calculate the head entity using formula (3) The vector matrix of the fused semantic information of the neighbors

[0078]

[0079] In formula (3), tanh is the activation function, || is the connection operation, and are the parameter matrices to be trained;

[0080] Generate the head entity using formula (4) The weighted neighbor vector representation of

[0081]

[0082] In formula (4), is the weight matrix to be trained, V k Is the head entity The vector representation of the fused semantic information of the kth neighbor, α k Is the head entity The kth neighbor of the head entity The attention score is obtained by formula (5);

[0083] In formula (5), A q Is the head entity The qth attention weight in the neighbor attention weight matrix, A k Is the head entity The kth attention weight in the neighbor attention weight matrix of ;

[0084]

[0085] For example, for the triple (pertussis, symptoms, chest tightness), the neighbors and relationships of the head entity (belonging to, acute respiratory infectious disease), (disease course, 2 to 3 months), it is obvious that the neighbor acute respiratory infectious disease is more important for modeling the head entity pertussis (because respiratory infectious disease is closer to the symptom information that the triple wants to express);

[0086] Step 4.1.5 Use formula (6) to get the head entity The final vector representation of

[0087]

[0088] In formula (6), are two weight matrices to be trained, and tanh is the activation function;

[0089] Step 4.1.6: Follow the process from step 4.1.2 to step 4.1.5 to get the tail entity The final vector representation of

[0090] Step 4.2 Use formula (7) to get the entity pair Vector representation of

[0091]

[0092] In formula (7), and is the parameter to be trained, and tanh is the activation function;

[0093] Step 4.3 Follow the process of steps 4.1-4.2 to obtain the relationship node r of the i-th relationship node. i The query set Q i Entity pair vector representation

[0094] Step 5: Construct an adaptive prototype network and assign different weights to reference entity pairs. This allows the model to pay more attention to reference entity pairs related to the query set entity pairs and generate relation prototype representations by aggregating entity pair representations.

[0095] Step 5.1 Use formula (8) to get the query entity pair and the j-th reference entity pair in the support set The similarity score α i,j :

[0096]

[0097] In formula (8), ⊙ is the dot product operation, Represents the support set S i The l-th reference entity pair in Representing entity pairs The vector representation of Representative query set Q i Entity Pairs The vector representation of Represents the support set S i The j-th reference entity pair in Vector representation of ;

[0098] For example, the support set is: (pertussis, belongs to, acute respiratory infectious diseases) (deer antler, belongs to, Northeast China)

[0099] The query set is: (hypertension, belongs to, circulatory system diseases). To generate the correct answer 'circulatory system diseases', the information expressed by the support set (pertussis, belongs to, acute respiratory infectious diseases) is obviously more important than the geographical location information expressed by (deer antler, belongs to, Northeast China);

[0100] Step 5.2: Use formula (9) to get the prototype representation P related to the query set. i :

[0101]

[0102] In formula (9), α i,j Is the query entity and support set S i The j-th reference entity pair in The similarity score of is the support set S i The jth reference entity pair in Vector representation of ;

[0103] Step 6: The medical common sense knowledge model is constructed by the interactive attention encoder and the adaptive prototype network. Based on the medical common sense knowledge graph G and the background knowledge graph G′, the medical common sense knowledge model is trained using the back propagation algorithm. At the same time, the loss function L shown in formula (10) is calculated to update the model parameters. When the loss function L continues to decrease until convergence, the optimal medical common sense knowledge model is obtained:

[0104]

[0105] In formula (10), [x] + =max(0,x) is a hinge loss function, γ is a marginal hyperparameter, is the vector representation of the query set entity pairs, is the vector representation of the negative query set entity pair;

[0106] Step 7: Input the existing medical common sense knowledge graph into the optimal medical common sense knowledge model, thereby outputting the credibility probability of the new medical common sense knowledge triples generated by the model, and outputting the medical common sense knowledge triples with higher probabilities, thereby realizing the automatic construction of the medical common sense knowledge graph.

[0107] In this embodiment, an electronic device includes a memory and a processor, wherein the memory is used to store a program that supports the processor to execute the above method, and the processor is configured to execute the program stored in the memory.

[0108] In this embodiment, a computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the steps of the above method are executed.

Claims

1. A method for automatically constructing a medical common sense knowledge graph based on meta-learning, characterized in that: The steps are as follows: Step 1: Obtain a medical knowledge graph and construct a medical knowledge graph G = {(h, r, t)∈ε×R×ε} based on the entities in the medical knowledge graph and the relationships between the entities, where h represents the head entity node, r represents the relationship node, and t represents the tail entity node; ε and R are the entity node set and the relationship node set corresponding to the medical knowledge graph G, respectively; Step 2: Extract some triples from the medical knowledge graph G as the background knowledge graph G′. The background knowledge graph G′ contains some relationship nodes in the relationship node set R. The triples corresponding to the remaining relationship nodes in R are used as the training knowledge graph G. train ; Step 3: From the training knowledge graph G train Randomly select the i-th relationship node r i , and select the i-th relationship node r i The corresponding partial triples are used to construct the relationship node r i The support set S i , using the i-th relationship node r i The remaining triples form the query set Q i ; Then the query set Q i All tail entities in are replaced with entities that have no relationship with their head entities, thus constructing the negative query set Q i - ; Step 4: Construct an interactive attention encoder and interactively train the support set S i and query set Q i Encode the semantic information of the head and tail entities to generate more discriminative entity pair representations; Step 4.1: Encode entity semantic information to obtain the head entity The final vector representation of and tail entity The final vector representation of Step 4.2: Use formula (7) to get the entity pair Vector representation of In formula (7), and is the parameter to be trained, and tanh is the activation function; Step 4.3: Follow the process of step 4.1-step 4.2 to get the relationship node r of the i-th relationship node i The query set Q i Entity pair vector representation Step 5: Build an adaptive prototype network to generate the relation prototype representation P by aggregating entity pair vector representations i ; Step 5.1: Use formula (8) to get the query entity pair and the j-th reference entity pair in the support set The similarity score α i,j : In formula (8), ⊙ is the dot product operation, Represents the support set S i The l-th reference entity pair in Representing entity pairs Vector representation of ; Step 5.2: Use formula (9) to obtain the prototype representation P related to the query set i : In formula (9), α i,j Is the query entity and support set S i The j-th reference entity pair in Similarity score of Step 6: The medical common sense knowledge model is constructed by the interactive attention encoder and the adaptive prototype network. Based on the medical common sense knowledge graph G and the background knowledge graph G′, the medical common sense knowledge model is trained using the back propagation algorithm. At the same time, the loss function L shown in formula (10) is calculated to update the model parameters. When the loss function L continues to decrease until convergence, the optimal medical common sense knowledge model is obtained: In formula (10), γ is the marginal hyperparameter, For the query set Q i Entity Pairs The vector representation of Negative query set Entity Pairs Vector representation of ; [·] + is the hinge loss function; Step 7: Input the existing medical common sense knowledge graph into the optimal medical common sense knowledge model, thereby outputting the credibility probability of the new medical common sense knowledge triples generated by the model, and outputting the medical common sense knowledge triples with higher probabilities, thereby realizing the automatic construction of the medical common sense knowledge graph.

2. The method for automatically constructing a medical common sense knowledge graph based on meta-learning according to claim 1, characterized in that: The step 4.1 is carried out as follows: Step 4.1.1: Generate the i-th relationship node r i The support set S i The j-th entity pair in Neighbor in, Represents the header entity neighbors, and Represents the tail entity neighbors, and in, Representation and header entity There are related entities in the background knowledge graph G′, r h Represents the header entity Neighbors with head entity The relationship between Represents the tail entity There are related entities in the background knowledge graph G′, r t Represents the tail entity Neighbors with tail entity the relationship between; Step 4.1.2: Count header entities The number of neighbors Set the dimension of the latent semantic feature vector to d and construct it using random initialization method dimensional neighbor relationship vector matrix dimensional neighbor entity vector matrix Step 4.1.3: Calculate the head entity Neighbor attention weight matrix Generate head entities separately using random initialization and tail entity Vector representation of and Use formula (1) to generate the i-th relationship node r i The initialization vector represents Ur i : Calculate the head entity using formula (2) Neighbor attention weight matrix In formula (2), are two weight matrices; Step 4.1.4: Generate the header entity The weighted neighbor vector representation of Calculate the head entity using formula (3) The vector matrix of the fused semantic information of the neighbors In formula (3), tanh is the activation function, || is the connection operation, and are the parameter matrices to be trained; Generate the head entity using formula (4) The weighted neighbor vector representation of In formula (4), is the weight matrix to be trained, V k Is the head entity The vector representation of the fused semantic information of the kth neighbor, α k Is the head entity The kth neighbor of the head entity The attention score is obtained by formula (5); In formula (5), A q Is the head entity The qth attention weight in the neighbor attention weight matrix, A k Is the head entity The kth attention weight in the neighbor attention weight matrix of ; Step 4.1.5: Use formula (6) to get the head entity The final vector representation of In formula (6), are two weight matrices to be trained, and tanh is the activation function; Step 4.1.6: Follow the process from step 4.1.2 to step 4.1.5 to get the tail entity The final vector representation of 3. An electronic device comprising a memory and a processor, characterized in that: The memory is used to store a program that supports the processor to execute the method for automatically constructing the medical common sense knowledge graph according to claim 1 or 2, and the processor is configured to execute the program stored in the memory.

4. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method for automatically constructing a medical common sense knowledge graph according to claim 1 or 2 are executed.

Images