Recursive path query encoding method for answering complex queries in medical knowledge graphs

Abstract

The present invention relates to a recursive path query encoding method for answering complex queries on medical knowledge graphs. The method comprises the following steps: obtaining and preprocessing a relevant dataset to obtain a structured set of logical queries and corresponding answer entities; decomposing the query into several path queries by branch based on a tree-like computational graph of an input first-order logic query; serializing the path queries to obtain a path query sequence; using a bidirectional Transformer Encoder as a path query encoder, recursively encoding all path query sequences according to the computational graph until a query embedding is obtained; and optimizing the network and related parameters using negative sampling and a cross-entropy loss based on the query embedding and the embeddings of positive and negative answer entity samples. This method leverages future query context information to explicitly model the implicit dependencies between the various components of the path query.

Description

Technical Field

[0001] The present invention belongs to the field of answering complex logical queries on medical knowledge graphs in knowledge graph reasoning, and specifically relates to a recursive path query encoding method for answering complex queries on medical knowledge graphs. Background Art

[0002] Healthcare issues remain a top concern in people's lives. With the rapid development of artificial intelligence (AI), the integration of healthcare and AI holds significant practical and application value. Medical knowledge graphs are a product of this integration. Medical knowledge graphs organize and store vast amounts of expert medical knowledge. Specialists can use the medical information in these graphs to comprehensively consider a patient's condition and aid in diagnosis. The general public can also use these graphs to search for relevant information corresponding to symptoms or the precise usage and dosage of medications. Consequently, medical knowledge graphs have garnered widespread attention and adoption across the industry.

[0003] When users query the medical knowledge graph, they may ask complex queries. For example, for a query like this: "Which drugs can treat diseases related to protein X but not protein Y?", it can be converted into a corresponding first-order logic query expression, such as Answering this kind of first-order logic query on an incomplete knowledge graph is a complex logic query problem.

[0004] To address the complex logical query problem of knowledge graphs, query embedding methods are currently mainly used at home and abroad. Specifically, these methods embed the entities and relations of the knowledge graph into a specific vector space, and then use well-defined parameterized neural logic operations to obtain the computational graph based on the first-order logic query expression corresponding to the given query, and calculate the final query embedding step by step from the anchor node in sequence. By comparing the distance between the query embedding and the candidate entity embedding, the entities with the smallest distance and the closest proximity are found as the answer to the query. These methods can solve the missing problem of knowledge graphs and are feasible in terms of computational complexity, so they have achieved good results, but they still have limitations. They only consider the previous query context information but not the future context information during the encoding process, which makes it difficult for them to model the implicit dependencies between the various parts of the query.

[0005] To address the above issues, there is an urgent need to propose a recursive path query encoding method for answering complex queries in medical knowledge graphs. Summary of the Invention

[0006] The present invention aims to address the aforementioned shortcomings of the prior art by providing a recursive path query encoding method for answering complex queries on medical knowledge graphs. This method, based on the tree-like nature of the query computation graph, decomposes complex first-order logic queries into several path queries, which are then recursively encoded. By introducing a bidirectional multi-head self-attention mechanism into the path query encoder, this method can fully leverage future query context to explicitly model the implicit dependencies between the various components of the path query.

[0007] The purpose of the present invention can be achieved by taking the following technical solutions:

[0008] A recursive path query encoding method for answering complex queries on medical knowledge graphs is proposed. This method uses a bidirectional Transformer Encoder to recursively encode path queries, thereby explicitly modeling the implicit dependencies between the various parts of the path query to achieve better learning results, thereby better answering complex logical queries on medical knowledge graphs. The method includes the following steps:

[0009] S1. Obtain a medical knowledge graph dataset, preprocess the data, and obtain a structured first-order logic query set and its corresponding answer entity set;

[0010] S2. Decompose the complex logic query into several path queries according to the tree-like computation graph corresponding to the input first-order logic query;

[0011] S3. Serialize the path query to obtain a path query sequence;

[0012] S4. Use the bidirectional Transformer Encoder as the path query encoder. According to the computation graph, recursively use the path query encoder to encode all path query sequences until the query embedding is obtained.

[0013] S5. Use negative sampling to optimize the network and related parameters using cross entropy loss based on the query embedding and the embedding of positive and negative answer entity samples;

[0014] S6. Inputting the complex logical query to be queried into the optimized network to obtain an embedding of the complex logical query;

[0015] S7. Compare the embedding of the complex logical query with the embedding of each entity based on distance, and retrieve the entity with the closest distance as the answer to the complex query.

[0016] Furthermore, the first-order logic query in step S1 is defined as follows:

[0017] A knowledge graph is a heterogeneous graph that contains a large number of nodes (entities) and edges (relationships), which stores real-world knowledge through links composed of these entities and relationships.

[0018] Knowledge Graph Defined as in Represents an entity set, each entity Represents a set of relations, where each relation r∈R represents a binary Boolean function: in Represented in the knowledge graph There is a directed edge of relationship type r from entity e i Pointing to entity e j , False means that There is no such directed edge of relationship type r from entity e i Pointing to entity e j .

[0019] Complex logical queries on knowledge graphs are queries that contain existential quantifiers. Conjunction ∧, Disjunction ∨, and Logical Negation First-order logic queries including the four logical operators.

[0020] It is worth noting that first-order logic queries do not take into account universal quantifiers This is because in real-world knowledge graphs there is no situation where one entity links to all other entities in the graph.

[0021] A first-order logic query q consists of a known set of anchor entities An unknown target answer entity variable And a series of intermediate entity variables A first-order logic query q is defined as the following disjunctive normal form, that is, the disjunction of several conjunctions:

[0022] Among them, c1, c2, ..., c i ,...,c n Represents the n conjunction expressions contained in the query q, each of which represents the conjunction of several literals a, c i =a i1 ∧a i2 ∧...∧a im , c i Indicates the i-th conjunction expression contained in the query q, a i1 ,a i2 ,...,a im represents the i-th conjunction expression c iEach of the m literals a represents a relational expression, which is the above binary Boolean function r or the negation of r. Depending on the input parameters of the function r, there are four cases:

[0023]

[0024] in, Represents an anchor entity, V∈{V ？ ,V1,V2,...,V k},V′∈{V1,V2,...,V k}, and V≠V′, “or” means that a can be any of the four relational expressions.

[0025] Furthermore, the computation graph in step S2 is defined as follows:

[0026] A first-order logic query yields a computational graph with a directed acyclic graph structure that defines the entire logical reasoning process. This graph defines the order of logical operations. Because of its directed acyclic nature and tree structure, the computational graph can be decomposed by branch.

[0027] like Figure 2 As shown in the figure, based on the input first-order logic query, the computational graph corresponding to the query can be obtained. The computational graph has a tree structure, and the edges in the graph are all directed edges. The directions of these directed edges define the order of logical reasoning. During the logical reasoning process, starting from the anchor node (that is, the known entity in the query), the corresponding logical operations of the corresponding edges are executed in sequence until the final representation of the target variable node is obtained, that is, the answer to the input query. Due to the tree-like characteristics of the first-order logic query computational graph, complex first-order logic queries can be decomposed into several path queries by branch. This divide-and-conquer decomposition method makes it possible to naturally introduce a bidirectional Transformer Encoder to encode each path query, thereby fully modeling the complex implicit dependencies within each path query.

[0028] Each node in the computation graph represents an unknown variable node or a known anchor node, and each edge in the computation graph represents a logical operation on an entity set defined as follows:

[0029] (1) Relationship projection: given an entity subset and a relationship The relationship projection operation returns a new entity set Each entity in One or more entities in are connected by an edge of relationship type r, that is,

[0030] (2) Intersection: Given n entity sets The intersection operation returns the intersection of n entity sets

[0031] (3) Union: Given n entity sets The intersection operation returns the intersection of n entity sets

[0032] (4) Negation: Given an entity set, the negation operation returns the complement of the entity set

[0033] Furthermore, the query decomposition process in step S2 is as follows:

[0034] S21. Obtain a computation graph corresponding to the query from the input first-order logic query;

[0035] S22. According to the predefined order of the computation graph, the anchor node in the computation graph is used as the starting node to perform graph traversal to decompose the above first-order logic query to obtain a path query. According to the node type encountered during the graph traversal, the path query acquisition process is divided into the following two cases:

[0036] (1) When the traversed node is a single-branch node, if the node is not the root node, the traversal process continues; if the node is the root node, the path acquisition process ends with the node as the end point, and a path query from the starting node to the root node is obtained;

[0037] (2) When the traversed node is a multi-branch node, the node is not considered. Instead, all branch paths from the starting node to the child nodes of the multi-branch node are obtained. If the multi-branch node is the root node, the path acquisition process is terminated; if not, the multi-branch node is used as the new starting node and the subsequent path queries are continued.

[0038] After the above graph traversal process, the input first-order logic query is decomposed into several path queries according to the branches in the computation graph. Figure 2 As shown, after the above query decomposition process, Figure 2 The computation graph in is decomposed into three path queries.

[0039] Furthermore, the path query serialization process in step S3 is as follows:

[0040] S31. In order to introduce a bidirectional Transformer Encoder to fully utilize the query context information to explicitly model the implicit dependencies between the various parts of the path query, it is necessary to convert the path query into a sequence form that can be processed by the bidirectional Transformer Encoder. The path query containing unknown variable nodes is represented in the form of a sequence, and the known starting node n of the path query is used. s Followed by the corresponding known logical operation operator sequence l1, l2, ..., l k Represents a path query, that is, sequence n s ,l1,l2,…,l k , and denote the sequence as S o , the above representation can uniquely determine a path query:

[0041]

[0042] in, Represents a path query, n s Indicates the starting node, Represents one or more unknown variable nodes in the path query, l1,l2,...,l k is the corresponding sequence of logical operators. The logical operators of path query involve relational projection and logical negation, k≥1;

[0043] S32, in order to encode the entire path query accompanied by the logical negation operation and balance the above path query sequence S o The semantic relationship between each part, define three types of word nodes: [Node], [Projection], [Negation], use these three types of word nodes to modify the sequence S in the previous step o , add a type word node [Node] before the starting node, add a type word node [Projection] before each relationship projection operator, and then replace the logical negation operator in the original position with the type word node [Negation] to obtain a new path query representation sequence S n The introduction of these three types of word nodes enables the natural use of bidirectional Transformer Encoder to encode path queries with logical negation;

[0044] S33, embed the three types of word nodes, the entities corresponding to all known anchor nodes, and the relations corresponding to all relational projection operations into the d-dimensional vector space R d , sequence S n can be converted into the input embedding sequence E1, E2, ..., E for the path query encoder in step S4 m, where m represents the number of sequence elements, and the input embedding sequence is recorded as E in , the embedded sequence E in Each element in is a d-dimensional vector. Since the bidirectional Transformer Encoder processes vector input, this substep embeds all elements in the input path query sequence into the vector space for further processing in subsequent steps.

[0045] Furthermore, the recursive path query encoding process in step S4 is as follows:

[0046] S41. According to the computation graph corresponding to the input query, the path query encoder defined by the following formula is used to first obtain the embedding E of the path query sequence containing the known anchor entity nodes in sequence: pq :

[0047]

[0048] Among them, E in is the input embedding sequence, It is a k1-layer bidirectional TransformerEncoder that embeds the input sequence E in , that is, E1, E2, ..., E m , the updated sequence representation E1′, E2′, ..., E m ′, each element in the updated sequence representation is still a d-dimensional vector, and MP() is an average pooling layer, which represents the updated sequence E1′, E2′, ..., E m ′ Take the average value and get a d-dimensional vector E from m d-dimensional vectors pq .

[0049] The above bidirectional Transformer Encoder comes from the literature: Ashish Vaswani, Noam Shazeer, NikiParmar, Jakob Uszkoreit, Llion Jones, Aidan N Gomez, Kaiser, and IlliaPolosukhin. Attention is all you need. Advances in neural information processing systems, 30, 2017. This paper proposes the Transformer model, which is the most important and hottest basic technology in the field of deep learning and is widely used. The original Transformer is an Encoder-Decoder architecture, that is, an encoder-decoder architecture, consisting of an Encoder and a Decoder. In order to introduce a bidirectional attention mechanism to fully utilize future query context information, only the Encoder part of the Transformer is needed, because the Encoder part uses bidirectional attention, while the Decoder part uses unidirectional attention, and the present invention does not need to use the characteristics and functions of the Decoder part. Therefore, the present invention only uses the Encoder part of the Transformer, and the bidirectional Transformer Encoder used in the present invention has the same architecture as the Encoder part of the Transformer proposed in the above-mentioned literature.

[0050] Since the path queries obtained after the decomposition of the input first-order logic query in step S2 may include path queries with unknown starting nodes, such as Figure 2 Path query 3 in . These path query sequences including unknown elements cannot be directly encoded by the path query encoder, so this sub-step first obtains the embedding of the path query whose starting node is a known anchor entity node, such as Figure 2 Path query 1 and path query 2;

[0051] S42. In order to complete subsequent path queries containing unknown starting nodes (i.e., variable nodes) to encode them, based on the characteristic that these unknown starting nodes must be multi-branch nodes, each known branch path query of these multi-branch nodes is used to embed to obtain the representation of these unknown multi-branch nodes, that is, the embedding of the path query sequence obtained in step S41 is used to obtain the representation of the subsequent unknown multi-branch variable starting nodes corresponding to these branch paths.

[0052] For example, for Figure 2Path query 3 in the example above is a path query with an unknown starting node. Since the unknown starting node is a two-branch node, and both branch path queries of this two-branch node start with known anchor nodes, embeddings for both branch path queries have been obtained after the previous sub-step S41. Therefore, the embeddings of path queries 1 and 2 can be aggregated to obtain a representation of the starting node of path query 3, thereby completing path query 3 and enabling it to continue to be processed by the path query encoder.

[0053] Specifically, we use the disjunctive normal form to transform disjunctive queries into conjunctive queries. This allows us to define a single encoder to aggregate branch path embeddings to complete subsequent paths. This is because the disjunctive normal form transforms complex first-order logic queries into the disjunctive form of conjunctions, meaning we only need to consider conjunctive queries. Conjunctive queries only need to consider intersection operations, not union operations, so only a single encoder is needed.

[0054] Since different multi-branch nodes have different numbers of branches, we only consider the intersection operation of two inputs here, because multiple branch inputs can be converted into the form of superposition of two inputs. A k2-layer multi-layer perceptron is used as the encoder to aggregate the branch path embedding and obtain the aggregate output corresponding to the unknown multi-branch variable node. The formula is as follows:

[0055] in, represents a multilayer perceptron with k2 layers, are the two branch path query embeddings of the corresponding multi-branch node, represents the splicing operation, E v ∈R d is the output obtained by aggregating the two branch path query embeddings.

[0056] The above-mentioned multilayer perceptron is a structure based on a feedforward neural network, consisting of an input layer, a hidden layer, and an output layer, where the hidden layer can have several layers. The neurons in each layer are connected to the neurons in the adjacent layer. By continuously adjusting the weights between neurons, learning and prediction of complex problems can be achieved. Since multilayer perceptrons can have multiple neuron layers, they are also called deep neural networks, which are now a classic technology in the field of deep learning. For a vector input X, each layer in the multilayer perceptron will multiply the input X by the weight matrix W corresponding to the layer, and optionally add a bias term b to the product. Finally, a nonlinear activation function F() is used to obtain the output of the layer as the input of the next layer. For example, for the i-th layer of the multilayer perceptron, the update rule of the layer is:

[0057] X i+1 =F(W i X i +bi )

[0058] Among them, X i is the output of the previous layer, that is, the output of the i-1th layer, corresponding to the input of this layer, W i is the weight matrix of this layer, b i is the bias term corresponding to this layer, X i+1 It is the output of this layer and can continue to be used as the input of the next layer of the multilayer perceptron. The activation function F() is used to introduce nonlinear factors to each layer. Otherwise, each layer of the multilayer perceptron is just a simple linear mapping. The activation function F() can be a ReLU function, a Sigmoid function, or a Tanh function.

[0059] S43. Obtain the representation of the unknown starting variable node of the subsequent path query by aggregating the known branch embeddings, recursively use the path query encoder to encode all subsequent path query sequences, and obtain the final query embedding by encoding the entire query computation graph.

[0060] This is because the completion operation in sub-step S42 can repeatedly complete subsequent path queries based on the currently obtained path query embedding, thereby continuing to use the path query encoder to encode these subsequent path queries, and repeating the cycle until the final query embedding is obtained. In essence, this is a recursive path query encoding process.

[0061] Furthermore, the network optimized in step S5 includes the path query encoder defined in step S41 and the multi-layer perceptron defined in step S42 for aggregating branch path embeddings to obtain corresponding unknown multi-branch variable node representations, and the weights of the path query encoder and the multi-layer perceptron are the objects of optimization;

[0062] The network-related parameters include the embeddings of the three types of word nodes defined in step S32: [Node], [Projection], and [Negation], as well as the embeddings of all entities and relationships in the knowledge graph. These parameters are learnable and will be optimized and updated.

[0063] Furthermore, the optimization objective based on negative sampling in step S5 is as follows:

[0064]

[0065] Among them, σ() is the sigmoid function, q is a given complex logical query, [q] represents all answer entities for query q, that is, the answer entity set corresponding to query q, v∈[q] is the answer entity corresponding to query q, corresponding to the positive sample, is the uth random negative sample, which corresponds to the answer to query q. γ is a fixed boundary value that acts as a constraint. By adjusting the appropriate value, the overfitting effect of network optimization can be reduced. t is the number of negative samples. The distance function Dist() calculates the Euclidean distance between the embeddings corresponding to the two inputs:

[0066] In the above formula, x1 and x2 are the two inputs of the distance function Dist(). Indicates taking the embedding of x1, It means taking the embedding of x2.

[0067] This optimization objective aims to simultaneously train the network's weight parameters, the embeddings of the three types of word nodes, and the embeddings of all entities and relations in the knowledge graph dataset. The objective function aims to minimize the distance between the query embedding and the embedding of the answer entity corresponding to the query, while maximizing the distance between the query embedding and the embedding of non-answer entities (i.e., negative samples). These negative samples are randomly sampled from the non-answer entity set, with t negative samples randomly sampled for each query training. For the distance function Dist, since the embedding vector space is a d-dimensional single-point vector space, a simple Euclidean distance is sufficient to measure the distance between embeddings.

[0068] Furthermore, the distance function used in step S7 is the distance function Dist() defined in claim 8. Because the network and related parameters are optimized according to the distance function, the same distance function should also be used to answer complex queries using the optimized model.

[0069] The present invention has the following advantages and effects compared to the prior art:

[0070] 1) This paper designs a recursive path query encoding method for answering complex queries on medical knowledge graphs. By decomposing complex queries into several path query sequences, this method naturally incorporates a bidirectional Transformer Encoder as a sequence model to encode these path query sequences. Through the bidirectional self-attention mechanism of the bidirectional Transformer Encoder, this method can fully utilize future query context information to explicitly model the implicit dependencies between the components within each path query.

[0071] 2) The present invention uses a special path query serialization scheme to enable the recursive path query encoding method to process all first-order logic query operators including logical negation operators. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] The drawings described herein are used to provide a further understanding of the present invention and constitute a part of this application. The exemplary embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation of the present invention. In the drawings:

[0073] Figure 1 is a flowchart of a recursive path query encoding method disclosed in the present invention for answering complex queries on a medical knowledge graph;

[0074] Figure 2 This is a schematic diagram of the process of step S2 in Example 1 of the present invention;

[0075] Figure 3 This is a schematic diagram of the process of step S2 in Example 2 of the present invention;

[0076] Figure 4 It is a schematic diagram of the model of the present invention;

[0077] Figure 5 It is a schematic diagram of the calculation graph structure of various complex logical queries used in Example 3 of the present invention. DETAILED DESCRIPTION

[0078] To make the objectives, technical solutions, and advantages of the embodiments of the present invention more clear, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts shall fall within the scope of protection of the present invention.

[0079] Example 1

[0080] Figure 1 This is a flowchart of the recursive path query encoding method disclosed in the present invention for answering complex queries on the medical knowledge graph, such as Figure 1 As shown, the method includes the following steps:

[0081] S1. Obtain a medical knowledge graph dataset and preprocess the data to obtain a structured first-order logic query set and its corresponding answer entity set. The knowledge graph data can be in Chinese or English, but English data is used as an example here.

[0082] S21. Obtain the computational graph corresponding to the query from the input first-order logic query. Figure 2As shown in Figure 2, given an English query “Which drugs can treat diseases related to protein X but not related to protein Y?”, the query can be converted into a first-order logic query expression:

[0083] In this formula, V is the intermediate entity variable, V ？ is the answer entity variable, which corresponds to the query answer that the user expects. Figure 2 As shown, the corresponding tree-structured computational graph can be obtained from the above first-order logic query expression;

[0084] S22, according to the order defined in advance in the computation graph, the anchor nodes in the computation graph (such as Figure 2 The anchor nodes protein X and protein Y in the graph are used as the starting nodes to perform graph traversal to decompose the above first-order logic query to obtain a path query. According to the node types encountered during the graph traversal, the path query acquisition process is divided into the following two cases:

[0085] (1) When the traversed node is a single-branch node, if the node is not the root node, the traversal process continues; if the node is the root node, the path acquisition process ends with the node as the end point, and a path query from the starting node to the root node is obtained;

[0086] (2) When the traversed node is a multi-branch node, the node is not considered. Instead, all branch paths from the starting node to the child nodes of the multi-branch node are obtained. If the multi-branch node is the root node, the path acquisition process is terminated; if not, the multi-branch node is used as the new starting node and the subsequent path queries are continued.

[0087] After the above graph traversal process, the input first-order logic query is decomposed into several path queries according to the branches in the computation graph. Figure 2 As shown, the above complex query is decomposed into three path queries.

[0088] S31, express the path query containing unknown variable nodes in the form of a sequence, and use the known starting node n of the path query s Followed by the corresponding known logical operation operator sequence l1, l2, ..., l k Represents a path query, that is, sequence n s ,l1,l2,…,l k , and denote the sequence as S o , the above representation can uniquely determine a path query:

[0089]

[0090] in, Represents a path query, n s Indicates the starting node, Represents one or more unknown variable nodes in the path query, l1,l2,...,l k is the corresponding sequence of logical operators. The logical operators of path query involve relational projection and logical negation, k≥1;

[0091] S32. Define three types of word nodes: [Node], [Projection], and [Negation]. Use these three types of word nodes to modify the sequence S in the previous step. o , add a type word node [Node] before the starting node, add a type word node [Projection] before each relationship projection operator, and then replace the logical negation operator in the original position with the type word node [Negation] to obtain a new path query representation sequence S n .like Figure 4 As shown, the corresponding path sequence of path query 2 is: [Node], [e2:ProteinY], [Projection], [r1:RelatedTo], [Negation];

[0092] S33, embed the three types of word nodes, the entities corresponding to all known anchor nodes, and the relations corresponding to all relational projection operations into the d-dimensional vector space R d , sequence S n Transformed into the input embedding sequence E1, E2, ..., E for the path query encoder in step S4 m , where m represents the number of sequence elements, and the input embedding sequence is recorded as E in , the embedded sequence E in Each element in is a d-dimensional vector. Figure 4 As shown, {·} represents the embedding of ·, for the path query sequence of path query 2:

[0093] [Node],[e2:ProteinY],[Projection],[r1:RelatedTo],[Negation],

[0094] Embed each element of the sequence into the d-dimensional vector space R dIn the example, {[Node]},{[e2:ProteinY]},{[Projection]},{[r1:RelatedTo]},{[Negation]} is used to input the path query encoder in the subsequent steps, which can only process vector inputs.

[0095] S41. According to the computation graph corresponding to the input query, the path query encoder defined by the following formula is used to first obtain the embedding E of the path query sequence containing the known anchor entity nodes in sequence: pq :

[0096]

[0097] Among them, E in is the input embedding sequence, It is a k1-layer bidirectional TransformerEncoder that embeds the input sequence E in , that is, E1, E2, ..., E m , the updated sequence representation E1′, E2′, ..., E m ′, each element in the updated sequence representation is still a d-dimensional vector, and MP() is an average pooling layer, which represents the updated sequence E1′, E2′, ..., E m ′ Take the average value and get a d-dimensional vector E from m d-dimensional vectors pq .like Figure 4 As shown in Figure 2, path query 1 and path query 2 are started with known anchor entity nodes, so the path query encoder is first used to encode these two queries to obtain their embeddings;

[0098] S42, for Figure 4 Path query 3 in [ 1 ] cannot be directly encoded using the path query encoder because its starting node is an unknown multi-branch variable node V. To complete and encode subsequent path queries containing unknown starting nodes (i.e., variable nodes), based on the fact that these unknown starting nodes are necessarily multi-branch nodes, each known branch path query embedding of these multi-branch nodes is used to obtain representations of these unknown multi-branch nodes. That is, the embeddings of the path query sequence obtained in step S41 are used to obtain representations of the subsequent unknown multi-branch variable starting nodes corresponding to these branch paths. For example, the representation of multi-branch variable node V can be obtained by aggregating the embeddings of path queries 1 and 2, thereby completing path query 3 so that it can be independently encoded by the path query encoder.

[0099] Specifically, we use the disjunctive normal form to transform the disjunctive query problem into a conjunctive query problem. This allows us to define only one encoder to aggregate the branch path embeddings to complete the subsequent path. Since different multi-branch nodes have different numbers of branches, we only consider the intersection operation of two inputs here, as multiple branch inputs can be converted into a form of superposition of two inputs. We use a k2-layer multi-layer perceptron as an encoder to aggregate the branch path embeddings and obtain the aggregate output corresponding to the unknown multi-branch variable node. The formula is as follows:

[0100] in, represents a multilayer perceptron with k2 layers, are the two branch path query embeddings of the corresponding multi-branch node, represents the splicing operation, E v ∈R d is the output obtained by aggregating the two branch path query embeddings. Figure 4 In the example, the embedding of path query 1 and the embedding of path query 2 are input into the above multi-layer perceptron to obtain the representation of the unknown starting node V of path query 3;

[0101] S43, by aggregating known branch embeddings to obtain the representation of the unknown starting variable node of the subsequent path query, recursively use the path query encoder to encode all subsequent path query sequences, and obtain the final query embedding by encoding the entire query calculation graph. Figure 4 As shown, after the completion operation of sub-step S42, the path query encoder can be used to continue encoding the final path query 3, thereby obtaining the embedding of the input first-order logic query. In essence, this is a process of recursively encoding the path query.

[0102] S5. Use negative sampling to optimize the network and related parameters using cross entropy loss based on the query embedding and the embedding of positive and negative answer entity samples;

[0103] S6. Inputting the complex logical query to be queried into the optimized network to obtain an embedding of the complex logical query;

[0104] S7. Compare the query embedding with the embeddings of each entity based on distance, and retrieve the entity with the closest distance as the answer to the complex query.

[0105] Example 2

[0106] Based on the recursive path query encoding method for answering complex queries on medical knowledge graphs disclosed in Example 1, this embodiment continues to use Chinese data as an example to disclose a recursive path query encoding method for answering complex queries on medical knowledge graphs, such as Figure 1 As shown, the method includes the following steps:

[0107] S1. Obtain a medical knowledge graph dataset and preprocess the data to obtain a structured first-order logic query set and its corresponding answer entity set. The knowledge graph data can be in Chinese or English, but Chinese data is used as an example here.

[0108] S21. Obtain the computational graph corresponding to the query from the input first-order logic query. Figure 2 As shown in the figure, given a Chinese query "Which diseases can be treated that are related to protein X and protein Y but not to protein Z?", the query can be converted into a first-order logic query expression: In this formula, V is the intermediate entity variable, V ？ is the answer entity variable, which corresponds to the query answer that the user expects. Figure 3 As shown, the corresponding tree-structured computational graph can be obtained from the above first-order logic query expression;

[0109] S22, according to the order defined in advance in the computation graph, the anchor nodes in the computation graph (such as Figure 3 The anchor nodes X protein, Y protein, and Z protein in the graph are used as the starting nodes to perform graph traversal to decompose the above first-order logic query to obtain a path query. According to the node types encountered during the graph traversal, the path query acquisition process is divided into the following two cases:

[0110] (1) When the traversed node is a single-branch node, if the node is not the root node, the traversal process continues; if the node is the root node, the path acquisition process ends with the node as the end point, and a path query from the starting node to the root node is obtained;

[0111] (2) When the traversed node is a multi-branch node, the node is not considered. Instead, all branch paths from the starting node to the child nodes of the multi-branch node are obtained. If the multi-branch node is the root node, the path acquisition process is terminated; if not, the multi-branch node is used as the new starting node and the subsequent path queries are continued.

[0112] After the above graph traversal process, the input first-order logic query is decomposed into several path queries according to the branches in the computation graph. Figure 3 As shown, the above complex query is decomposed into four path queries.

[0113] S31, express the path query containing unknown variable nodes in the form of a sequence, and use the known starting node n of the path query s Followed by the corresponding known logical operation operator sequence l1, l2, ..., l k To represent a path query, that is, the sequence n s ,l1,l2,…,l k, and denote the sequence as S o , the above representation can uniquely determine a path query:

[0114]

[0115] in, Represents a path query, n s Indicates the starting node, Represents one or more unknown variable nodes in the path query, l1,l2,...,l k is the corresponding sequence of logical operators. The logical operators of path query involve relational projection and logical negation, k≥1;

[0116] S32. Define three types of word nodes: [Node], [Projection], and [Negation]. Use these three types of word nodes to modify the sequence S in the previous step. o , add a type word node [Node] before the starting node, add a type word node [Projection] before each relationship projection operator, and then replace the logical negation operator in the original position with the type word node [Negation] to obtain a new path query representation sequence S n .For example, Figure 3 The corresponding path sequence of path query 3 in is: [Node], [e3: Z protein], [Projection], [r1: Correlation], [Negation];

[0117] S33, embed the three types of word nodes, the entities corresponding to all known anchor nodes, and the relations corresponding to all relational projection operations into the d-dimensional vector space R d , sequence S n Transformed into the input embedding sequence E1, E2, ..., E for the path query encoder in step S4 m , where m represents the number of sequence elements, and the input embedding sequence is recorded as E in , the embedded sequence E in Each element in is a d-dimensional vector.

[0118] S41. According to the computation graph corresponding to the input query, the path query encoder defined by the following formula is used to first obtain the embedding E of the path query sequence containing the known anchor entity nodes in sequence: pq :

[0119]

[0120] Among them, E in is the input embedding sequence, It is a k1-layer bidirectional TransformerEncoder that embeds the input sequence E in , that is, E1, E2, ..., E m , the updated sequence representation E1′, E2′, ..., E m ′, each element in the updated sequence representation is still a d-dimensional vector, and MP() is an average pooling layer, which represents the updated sequence E1′, E2′, ..., E m ′ Take the average value and get a d-dimensional vector E from m d-dimensional vectors pq .like Figure 3 As shown, path query 1, path query 2 and path query 3 are started by known anchor entity nodes, so the two queries are first encoded using the path query encoder to obtain their embeddings.

[0121] S42, for Figure 3 Path query 4 in [ 4 ] cannot be directly encoded using the path query encoder because its starting node is an unknown multi-branch variable node V. To complete and encode subsequent path queries containing unknown starting nodes (i.e., variable nodes), based on the characteristic that these unknown starting nodes are necessarily multi-branch nodes, each known branch path query embedding of these multi-branch nodes is used to obtain representations of these unknown multi-branch nodes. That is, the embeddings of the path query sequence obtained in step S41 are used to obtain representations of the subsequent unknown multi-branch variable starting nodes corresponding to these branch paths. For example, the representation of multi-branch variable node V can be obtained by aggregating the embeddings of path queries 1, 2, and 3, thereby completing path query 4 so that it can be independently encoded by the path query encoder.

[0122] Specifically, we use the disjunctive normal form to transform the disjunctive query problem into a conjunctive query problem. This allows us to define only one encoder to aggregate the branch path embeddings to complete the subsequent path. Since different multi-branch nodes have different numbers of branches, we only consider the intersection operation of two inputs here, as multiple branch inputs can be converted into a form of superposition of two inputs. We use a k2-layer multi-layer perceptron as an encoder to aggregate the branch path embeddings and obtain the aggregate output corresponding to the unknown multi-branch variable node. The formula is as follows:

[0123] in, represents a multilayer perceptron with k2 layers, are the two branch path query embeddings of the corresponding multi-branch node, represents the splicing operation, E v ∈R dis the output obtained by aggregating the two branch path query embeddings. Figure 3 In the example of Example 1, similar to Example 1, the embedding of path query 1, the embedding of path query 2, and the embedding of path query 3 are input into the above multi-layer perceptron to obtain the representation of the unknown starting node V of path query 4;

[0124] S43: Aggregate the known branch embeddings to obtain a representation of the unknown starting variable node for the subsequent path query. Recursively use the path query encoder to encode all subsequent path query sequences, and obtain the final query embedding by encoding the entire query computation graph. Similar to the process in Example 1, after the completion operation in sub-step S42, the path query encoder can be used to encode the final path query 4, thereby obtaining the embedding of the input first-order logic query. This is essentially a recursive encoding process for path queries.

[0125] S5. Use negative sampling to optimize the network and related parameters using cross entropy loss based on the query embedding and the embedding of positive and negative answer entity samples;

[0126] S6. Inputting the complex logical query to be queried into the optimized network to obtain an embedding of the complex logical query;

[0127] S7. Compare the query embedding with the embeddings of each entity based on distance, and retrieve the entity with the closest distance as the answer to the complex query.

[0128] Example 3

[0129] The experimental results of the recursive path query encoding method for answering complex queries on medical knowledge graphs disclosed in this invention and other query embedding methods on three general knowledge graph complex query answering task benchmark datasets (FB15k, FB15k-237, and NELL995) are shown in Table 1:

[0130] Table 1. Experimental results of the present invention and other query embedding methods on benchmark datasets

[0131]

[0132]

[0133] In Table 1, the evaluation index used is MRR (%), which is used to measure the ability of the method to retrieve the correct answer to the corresponding complex query. The higher the index, the better the performance of the method. 1p, 2p, 3p, 2i, 3i, ip, pi, 2u, up, 2in, 3in, inp, pin, and pni in Table 1 represent the computational graph structures of different complex queries, such as Figure 5As shown in the table, p represents relational projection, i represents intersection, u represents union, and n represents logical negation. The values ​​in these columns are the MRR values ​​of each method answering this query structure separately. Avg_p represents the average MRR values ​​of the nine query structures 1p, 2p, 3p, 2i, 3i, ip, pi, 2u, and up, which is used to measure the ability of the method for complex queries that do not contain logical negation operations; while Avg_n represents the average MRR values ​​of the five query structures 2in, 3in, inp, pin, and pni, which is used to measure the ability of the method for complex queries that contain logical negation operations. "N / A" in Table 1 means that the method is not applicable to these query structures and therefore has no corresponding MRR value.

[0134] From the comparison of experimental results in Table 1, it can be seen that compared with previous query embedding methods (GQE, Q2B, BetaE and MLP), the recursive path query encoding method disclosed in the present invention for answering complex queries on medical knowledge graphs can achieve better results on various benchmark datasets.

[0135] These previous query embedding methods all start from the anchor node based on the computation graph, and only process a single link in the computation graph at a time, thereby gradually obtaining the final query embedding through parameterized neural logic operations. However, this approach makes it impossible for these methods to fully utilize future query context information to jointly encode queries, and it is difficult to model the implicit dependencies between the various parts of the query. The present invention, by taking advantage of the tree-like structure characteristics of the query computation graph, decomposes complex queries into several path queries for processing, and introduces a bidirectional self-attention mechanism by using a bidirectional Transformer Encoder to encode path queries to fully utilize future query context information, thereby better modeling the complex relationships within the path queries. The experimental results in Table 1 verify the effectiveness and practicality of the present invention.

[0136] The above embodiments are preferred implementations of the present invention, but the implementations of the present invention are not limited to the above embodiments. Any other changes, modifications, substitutions, combinations, and simplifications that do not deviate from the spirit and principles of the present invention should be considered equivalent replacement methods and are included in the scope of protection of the present invention.

Claims

1. A recursive path query encoding method for answering complex queries on medical knowledge graphs, characterized in that: The recursive path query encoding method comprises the following steps: S1. Obtain a medical knowledge graph dataset, preprocess the data, and obtain a structured first-order logic query set and its corresponding answer entity set; S2. Decompose the complex logic query into several path queries according to the tree-like computation graph corresponding to the input first-order logic query. The computation graph in step S2 is defined as follows: A first-order logic query generates a computational graph with a directed acyclic graph structure that defines the entire logical reasoning process. The computational graph defines the order of logical operations and is decomposed by branches. Each node in the computational graph represents an unknown variable node or a known anchor node. Each edge in the computational graph represents a logical operation on an entity set defined as follows: (1) Relationship projection: given an entity subset and a relationship The relationship projection operation returns a new entity set Each entity in One or more entities in are connected by an edge of relationship type r, that is, (2) Intersection: Given n entity sets The intersection operation returns the intersection of n entity sets (3) Union: Given n entity sets The intersection operation returns the intersection of n entity sets (4) Negation: Given an entity set, the negation operation returns the complement of the entity set The query decomposition process in step S2 is as follows: S21. Obtain a computation graph corresponding to the query from the input first-order logic query; S22. According to the predefined order of the computation graph, the anchor node in the computation graph is used as the starting node to perform graph traversal to decompose the above first-order logic query to obtain a path query. According to the node type encountered during the graph traversal, the path query acquisition process is divided into the following two cases: (1) When the traversed node is a single-branch node, if the node is not the root node, the traversal process continues; if the node is the root node, the path acquisition process ends with the node as the end point, and a path query from the starting node to the root node is obtained; (2) When the traversed node is a multi-branch node, the node is not considered. Instead, all branch paths from the starting node to the child nodes of the multi-branch node are obtained. If the multi-branch node is the root node, the path acquisition process is terminated; if not, the multi-branch node is used as the new starting node and the subsequent path queries are continued. After the above graph traversal process, the input first-order logic query is decomposed into several path queries according to the branches in the computation graph; S3. Serialize the path query to obtain a path query sequence; S4. Use the bidirectional Transformer Encoder as the path query encoder. According to the computation graph, recursively use the path query encoder to encode all path query sequences until the query embedding is obtained. S5. Use negative sampling to optimize the network and related parameters using cross entropy loss based on the query embedding and the embedding of positive and negative answer entity samples; S6. Inputting the complex logical query to be queried into the optimized network to obtain an embedding of the complex logical query; S7. Compare the embedding of the complex logical query with the embedding of each entity based on distance, and retrieve the entity with the closest distance as the answer to the complex query.

2. The recursive path query encoding method for answering complex queries on medical knowledge graphs according to claim 1 is characterized in that: The first-order logic query is defined as follows: Knowledge Graph Defined as in Represents an entity set, each entity Represents a set of relations, each relation Represents a binary Boolean function: {True,False}, where True means in the knowledge graph There is a directed edge of relationship type r from entity e i Pointing to entity e j , False means that There is no such directed edge of relationship type r from entity e i Pointing to entity e j ; Complex logical queries on knowledge graphs are queries that contain existential quantifiers. Conjunction ∧, Disjunction ∨, and Logical Negation First-order logic query including four logical operators. A first-order logic query q includes a known anchor entity set An unknown target answer entity variable And a series of intermediate entity variables A first-order logic query q is defined as follows in disjunctive normal form: Among them, c1, c2, …, c i ,…,c n Represents the n conjunction expressions contained in the query q, each of which represents the conjunction of several literals a, c i =a i1 ∧a i2 ∧…∧a im , c i Indicates the i-th conjunction expression contained in the query q, a i1 ,a i2 ,…,a im represents the i-th conjunction expression c i Each of the m literals a represents a relational expression, which is the negation of the above binary Boolean function r or r. Depending on the input parameters of the function r, there are four cases: in, Represents an anchor entity, V∈{V ？ ,V1,V2,…,V k }, V′∈{V1,V2,…,V k }, and V≠V′, "or" means that a can be any of the four relational expressions.

3. The recursive path query encoding method for answering complex queries on medical knowledge graphs according to claim 1 is characterized in that: The path query serialization process in step S3 is as follows: S31, express the path query containing unknown variable nodes in the form of a sequence, and use the known starting node n of the path query s Followed by the corresponding known logical operation operator sequence l1, l2, ..., l k Represents a path query, that is, sequence n s ,l1,l2,…,l k , and denote the sequence as S o , the above representation can uniquely determine a path query: in, Represents a path query, n s Indicates the starting node, Represents one or more unknown variable nodes in the path query, l1, l2, ..., l k is the corresponding sequence of logical operators. The logical operators of path query involve relational projection and logical negation, k≥1; S32. Define three types of word nodes: [Node], [Projection], and [Negation]. Use these three types of word nodes to modify the sequence S in the previous step. o , add a type word node [Node] before the starting node, add a type word node [Projection] before each relationship projection operator, and then replace the logical negation operator in the original position with the type word node [Negation] to obtain a new path query representation sequence S n ; S33, embed the three types of word nodes, the entities corresponding to all known anchor nodes, and the relations corresponding to all relational projection operations into the d-dimensional vector space R d , sequence S n Transformed into the input embedding sequence E1, E2, ..., E for the path query encoder in step S4 m , where m represents the number of sequence elements, and the input embedding sequence is recorded as E in , the embedded sequence E in Each element in is a d-dimensional vector.

4. The recursive path query encoding method for answering complex queries on medical knowledge graphs according to claim 3 is characterized in that: The recursive path query encoding process in step S4 is as follows: S41. According to the computation graph corresponding to the input query, the path query encoder defined by the following formula is used to first obtain the embedding E of the path query sequence containing the known anchor entity nodes in sequence: pq : Among them, E in is the input embedding sequence, It is a k1-layer bidirectional Transformer Encoder that embeds the input sequence E in , that is, E1, E2, …, E m , the updated sequence representation E1′, E2′, …, E m ′, each element in the updated sequence representation is still a d-dimensional vector, and MP() is an average pooling layer, which represents the updated sequence E1′, E2′, …, E m ′ Take the average value and get a d-dimensional vector E from m d-dimensional vectors pq ; S42. Use a k2-layer multilayer perceptron as the encoder to aggregate the branch path embedding and obtain the aggregate output corresponding to the unknown multi-branch variable node. The formula is as follows: in, represents a multilayer perceptron with k2 layers, are the two branch path query embeddings of the corresponding multi-branch node, represents the splicing operation, E v ∈R d It is the output obtained by aggregating the two branch path query embeddings; S43. Obtain the representation of the unknown starting variable node of the subsequent path query by aggregating the known branch embeddings, recursively use the path query encoder to encode all subsequent path query sequences, and obtain the final query embedding by encoding the entire query computation graph.

5. The recursive path query encoding method for answering complex queries on medical knowledge graphs according to claim 4 is characterized in that: The network optimized in step S5 includes the path query encoder defined in step S41 and the multi-layer perceptron defined in step S42 for aggregating branch path embeddings to obtain corresponding unknown multi-branch variable node representations, and the weights of the path query encoder and the multi-layer perceptron are the objects of optimization; The network-related parameters include the embeddings of the three types of word nodes defined in step S32: [Node], [Projection], and [Negation], as well as the embeddings of all entities and relationships in the knowledge graph.

6. The recursive path query encoding method for answering complex queries on medical knowledge graphs according to claim 1, characterized in that: The optimization objective based on negative sampling in step S5 is as follows: Among them, σ() is the sigmoid function, q is a given complex logical query, [q] represents all answer entities for query q, that is, the answer entity set corresponding to query q, v∈[q] is the answer entity corresponding to query q, corresponding to the positive sample, is the uth random negative sample, which corresponds to the answer to query q. γ is a fixed boundary value that acts as a constraint. By adjusting the appropriate value, the overfitting effect of network optimization can be reduced. t is the number of negative samples. The distance function Dist() calculates the Euclidean distance between the embeddings corresponding to the two inputs: In the above formula, x1 and x2 are the two inputs of the distance function Dist(). Indicates taking the embedding of x1, It means taking the embedding of x2.

7. The recursive path query encoding method for answering complex queries on medical knowledge graphs according to claim 6, characterized in that: The distance function used in step S7 is the distance function Dist().

Images