Medical knowledge graph question and answer method based on GAT and probabilistic reasoning

Abstract

The invention discloses a medical knowledge graph question answering method based on GAT and probabilistic reasoning, and relates to the technical field of natural language processing. According to the technical scheme, problem coding and instruction decoding are executed; initializing an answer entity probability; a GAT network is constructed and iterated; probability filtering and neighbor selection are carried out; iterative training and probability updating are executed; and carrying out final answer entity probability sorting. The medical knowledge graph question-answering method based on GAT and probabilistic reasoning has the advantages that intelligent weighting processing is conducted on nodes and edges in a medical field knowledge graph, a graph attention network and a probabilistic reasoning mechanism are introduced, the accuracy of a question-answering system is remarkably improved, the robustness and adaptability of the system are enhanced, and the method is suitable for popularization and application. The interpretability and user friendliness of the system are improved, the answer generation process is optimized, the calculation efficiency is improved, and innovation and development of the medical knowledge graph question answering technology are promoted.

Description

Technical Field

[0001] The present invention belongs to the technical field of natural language processing, relates to a multi-hop question-answering system in a medical knowledge graph, proposes a multi-level semantic analysis model based on deep learning, and specifically relates to a medical knowledge graph question-answering method based on GAT and probabilistic reasoning. Background Art

[0002] With the rapid development of information technology, the medical field is facing the challenge of information "explosion". The continuous emergence of massive medical literature, patient data and clinical guidelines has increased the demand of medical staff and patients for accurate health information. How to efficiently extract valuable knowledge from this massive amount of information and transform it into actionable information has become an important topic in current medical research and practice. In this context, the question-answering system based on knowledge graph is particularly important. It can intuitively display the relationship between concepts such as diseases, symptoms, treatment plans, and drugs, thereby helping users better understand and use this information.

[0003] Although knowledge graph-based question-answering systems have shown great potential in the medical field, the development of existing technologies still faces a series of challenges and technical problems that directly affect the effectiveness and accuracy of the systems in practical applications.

[0004] In the existing technology, most medical question-answering systems based on knowledge graphs rely on traditional information retrieval and matching algorithms, which perform well when dealing with simple and clear questions, but often have difficulty providing accurate and comprehensive answers when faced with complex and ever-changing medical queries. This is because traditional algorithms lack sufficient flexibility and deep understanding capabilities when dealing with nodes and relationships in graphs, and have difficulty capturing the hidden and complex semantic information in graphs.

[0005] In addition, existing knowledge graph question-answering systems often lack detailed analysis of the relationships between nodes and dynamic weight allocation during the answer generation process. In the medical field, the relationships between concepts such as diseases, symptoms, and drugs are intricate, and the strength and importance of the associations between different nodes vary. If these relationships cannot be accurately evaluated, it will be difficult for the system to generate accurate and targeted answers. Summary of the invention

[0006] In order to solve the above-mentioned technical problems existing in the prior art, the present invention provides a medical knowledge graph question-answering method based on GAT and probabilistic reasoning, which aims to optimize the answer generation process of the knowledge graph question-answering system. The framework generates subgraphs and analyzes the relationship and similarity between neighbor nodes by making full use of the information in the knowledge graph. Specifically, the present invention uses the Graph Attention Network (GAT) in the graph neural network to dynamically assign weights to the generated subgraph neighbor nodes, thereby enhancing the feature expression ability between nodes. In the screening process of candidate answer entities, the present invention introduces a probabilistic filter to accurately evaluate candidate entities based on the weights and similarities of the nodes.

[0007] To achieve the above object, the present invention provides the following technical solutions:

[0008] A medical knowledge graph question answering method based on GAT and probabilistic reasoning, the steps are as follows:

[0009] S1, execution question encoding and instruction decoding;

[0010] S2, initialize the answer entity probability;

[0011] S3, build and iterate the GAT network;

[0012] S4, perform probability filtering and neighbor selection;

[0013] S5, perform iterative training and probability update;

[0014] S6. Final answer entity probability ranking.

[0015] Furthermore, in step S1, the BERT model encodes the input text sequence through the self-attention mechanism and maps each word into a dense vector space. After encoding the natural language question using the BERT model, the generation process of its vector representation is expressed by the following formula:

[0016] X=BERT(P)

[0017] Among them, P is the input natural language question, and X is the vector representation obtained after BERT encoding;

[0018] h=Pooling(X)

[0019] h is the final vector representation extracted from the output of the BERT model, which is obtained by performing a pooling operation on all word vectors. After obtaining the encoding of the question, the encoding results are deeply analyzed to identify and extract the various instructions contained in the question.

[0020] Furthermore, in step S2, before the question-answering reasoning begins, the probabilities of all possible answer entities are initialized to zero:

[0021] P A =0

[0022] Among them, P A is a vector whose length is equal to the total number of answer entities and each element corresponds to the initial probability of an answer entity.

[0023] Furthermore, in step S3, a network model dominated by GAT is constructed, which updates the probability of the answer entity in an iterative manner; in each iteration, the GAT network adaptively learns the weight between each node and its neighboring nodes. The core formula of GAT is expressed as:

[0024] h i ′=σ(∑ j∈Ni α ij Whj)

[0025] Among them, h i ′ is the new feature vector of node i, Nii is the set of neighbor nodes of node i, α ij is the attention weight of node j to node i, W is the shared weight matrix, and σ is the activation function.

[0026] Furthermore, in step S4, in each iteration, the neighboring nodes with higher correlation with the target node are selected through the probability filtering mechanism; this step is achieved by adjusting the attention weight α ij To achieve this, the network pays more attention to those neighbor nodes that have actual contributions to the current task:

[0027]

[0028] Among them, e ij is the unnormalized attention score between nodes i and j, computed via a feed-forward neural network.

[0029] Furthermore, in step S5, the probability distribution of the answer entity is continuously updated through three loop iterations; in each iteration, the network updates the probability of each answer entity according to the current probability distribution and the information of the neighboring nodes:

[0030]

[0031] in, is the probability distribution of the answer entity after the tth iteration, and X is the feature matrix of the node.

[0032] Furthermore, in step S6, after three iterations, the final answer entity probability distribution is obtained; by sorting, the answer entity with the highest probability is selected as the final output: is the probability distribution of the answer entity after three iterations.

[0033] Beneficial effects of the present invention:

[0034] Compared with the prior art, the medical knowledge graph question-answering method based on GAT and probabilistic reasoning described in the present invention has the following technical features or beneficial effects:

[0035] (1) Significantly improve the accuracy of the question-answering system:

[0036] By introducing the graph attention network (GAT), the present invention can dynamically learn and assign the importance weights of nodes and edges in the knowledge graph, which greatly enhances the system's ability to understand complex medical information. Compared with the traditional static weight assignment method, GAT can more accurately capture the correlation and difference between nodes, thereby improving the accuracy of answer generation.

[0037] (2) Enhance the robustness and adaptability of the system:

[0038] The introduction of the probabilistic reasoning mechanism makes the present invention more robust when processing uncertain information. By iteratively updating the probability distribution of the answer entity, the system can gradually converge to the most likely answer, effectively reducing the risk of misjudgment. At the same time, this mechanism also enables the system to better adapt to the question-answering needs in different situations, improving the versatility and practicality of the system.

[0039] (3) Improve the interpretability and user-friendliness of the system:

[0040] This invention provides a more transparent and explainable question-answering process by combining graph attention networks and probabilistic reasoning. Users can not only get the final answer, but also understand how the system arrives at the answer based on the input question and the information in the knowledge graph. This helps to enhance users' trust in the system and promotes further optimization and improvement of the system.

[0041] (4) Optimize the answer generation process and improve computational efficiency:

[0042] By building a network model dominated by GAT and combining it with a probabilistic filtering mechanism, the present invention can effectively reduce the amount of calculation and improve the response speed of the system while ensuring the accuracy of the answer. This is of great significance for processing massive medical data and complex query requests, and can provide users with more timely and efficient services.

[0043] (5) Promote the innovation and development of medical knowledge graph question-answering technology:

[0044] The invention not only solves the problems existing in the prior art, but also provides new ideas and methods for the development of medical knowledge graph question answering technology. By combining advanced technologies such as graph attention networks and probabilistic reasoning, the invention provides a useful reference and reference for research and practice in related fields, and helps promote innovation and development in the entire field.

[0045] In summary, the present invention significantly improves the accuracy and robustness of the medical knowledge graph question-answering system by introducing graph attention networks and probabilistic reasoning mechanisms, while enhancing the interpretability and user-friendliness of the system, providing a new solution for the intelligent processing and utilization of medical information. BRIEF DESCRIPTION OF THE DRAWINGS

[0046] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the present invention will be described in detail below in combination with the accompanying drawings and detailed embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:

[0047] Figure 1 It is a schematic diagram of the GAP structure of the present invention;

[0048] Figure 2 It is the interpretable reasoning graph of the present invention. DETAILED DESCRIPTION

[0049] In order to make the purpose, technical solution and advantages of the present invention more clearly understood, the present invention is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present invention and are not intended to limit the present invention. Figure 1-2 The medical knowledge graph question-answering method based on GAT and probabilistic reasoning is further explained, and the technical solutions in the embodiments of the present application are clearly and completely described; obviously, the described embodiments are only part of the embodiments of the present application, not all of the embodiments, and based on the embodiments in the present application, all other embodiments obtained by ordinary technicians in this field without making creative work are within the scope of protection of this application.

[0050] Example 1

[0051] This invention combines the graph attention network (GAT) and the probability mechanism to intelligently weight nodes and edges in the medical knowledge graph, which can more accurately capture the information relationship across nodes, thereby improving the accuracy and reasoning ability of multi-hop question answering. The specific scheme is as follows:

[0052] This embodiment proposes a question-answering framework (GAP) based on GAT and probabilistic reasoning, including:

[0053] Step 1: Question encoding and instruction decoding. BERT encodes the input text sequence through the self-attention mechanism and maps each word into a dense vector space. After encoding the natural language question using the BERT model, the generation process of its vector representation can be expressed by the following formula:

[0054] X=BERT(P)

[0055] Among them, P is the input natural language question, and X is the vector representation obtained after BERT encoding.

[0056] h=Pooling(X)

[0057] h is the final vector representation extracted from the BERT output, which is usually obtained by performing a pooling operation on all word vectors. After obtaining the encoding of the question, the present invention conducts an in-depth analysis of the encoding results to identify and extract various instructions contained in the question.

[0058] Step 2: Initialize the probability of the answer entity. Before the question-answering reasoning begins, the probabilities of all possible answer entities are initialized to zero. This means that without any evidence to support it, the present invention does not make assumptions about any answer entity:

[0059] P A =0

[0060] Among them, P A is a vector whose length is equal to the total number of answer entities and each element corresponds to the initial probability of an answer entity.

[0061] Step 3: GAT network construction and iteration. Build a GAT-dominated network model that will update the probability of the answer entity in an iterative manner. In each iteration, the GAT network will adaptively learn the weights between each node and its neighbor nodes to better capture the local relationships in the graph. The core formula of GAT can be expressed as:

[0062] h i ′=σ(∑ j∈Ni α ij Whj)

[0063] Among them, h i ′ is the new feature vector of node i, Nii is the set of neighbor nodes of node i, α ij is the attention weight of node j to node i, W is the shared weight matrix, and σ is the activation function.

[0064] Step 4: Probabilistic filtering and neighbor selection. In each iteration, the probabilistic filtering mechanism is used to select neighbor nodes with high correlation with the target node. This step can be achieved by adjusting the attention weight α ij This allows the network to pay more attention to neighbor nodes that have actual contributions to the current task:

[0065]

[0066] Among them, e ij is the unnormalized attention score between nodes i and j, which can be calculated by a feed-forward neural network.

[0067] Step 5: Iterative training and probability update. Through three cycles of iteration, the probability distribution of the answer entity is continuously updated. In each iteration, the network will update the probability of each answer entity based on the current probability distribution and the information of the neighboring nodes:

[0068]

[0069] in, is the probability distribution of the answer entity after the tth iteration, and X is the feature matrix of the node.

[0070] Step 6: Final answer entity probability sorting. After three iterations, the final answer entity probability distribution is obtained. Through sorting, the answer entity with the highest probability is selected as the final output: is the probability distribution of the answer entity after three iterations.

[0071] Compared with the prior art, the above technical solution adopted by the present invention has the following advantages:

[0072] 1) Reliability. The graph attention network allows intelligent weighting of nodes and edges in the knowledge graph, thereby dynamically adjusting the importance of different information and improving the relevance and reliability of the reasoning results.

[0073] 2) Robustness. The introduction of a probabilistic mechanism can effectively handle uncertainty, optimize the reasoning process, and improve the robustness of the system.

[0074] 3) Interpretability. Compared with ReaRev, the whole process is easier to visualize and understand by humans.

[0075] Example 2

[0076] like Figure 1As shown, in order to prove the effectiveness of the present invention, the GAP network structure is built using Python's deep learning library pytorch framework, and two public English knowledge graph multi-hop reasoning datasets are mainly used: the English dataset WebQSP (WebQuestionsSP) and the complex dataset CWQ (ComplexWebQuestions). The WebQSP dataset contains 4737 natural language questions, which are designed to be answered using a subset of the Freebase knowledge graph (KG). Freebase KG contains approximately 164.6 million facts and 24.9 million entities. Questions in this dataset typically require two-hop reasoning within the scope of the knowledge graph. Specifically, for about 30% of the questions, the model needs to aggregate two or more KG facts; about 7% of the questions require constraint reasoning; and the remaining questions only require a single KG fact to answer. The CWQ dataset is an extension of WebQSP, and more complex multi-hop questions are constructed by expanding the question entities or adding constraints to the answers. The dataset contains 34,689 questions covering four question types: constitutive (45%), conjunction (45%), comparative (5%), and superlative (5%). These questions require reasoning up to four hops on the knowledge graph, which has the same scope as the WebQSP dataset.

[0077] The specific training and testing process includes:

[0078] Step 1: Use Python's deep learning library pytorch framework to build the GAP network structure;

[0079] Step 2: Build the basic GAP network structure, determine the basic parameters such as batch_size, maximum number of iterations epoch and step size of the data in the network, and input the pre-processed data sets into the GAP network. The parameters in the method proposed by the present invention are:

[0080] Table 1 GAP algorithm initial parameters

[0081]

[0082] Step 3: Use Bert encoding to semantically encode the questions raised by users and convert them into high-dimensional vector form.

[0083] Step 4: Semantic decoding stage. The high-dimensional semantic vector output by the BERT model is further converted into specific operation instructions through the decoder. This step ensures that the understanding of the problem can be converted into executable instructions, laying the foundation for the subsequent reasoning process.

[0084] Step 5: Probability initialization: Before the reasoning process begins, all potential answer entities are assigned an initial probability value, i.e., zero probability, to exclude the influence of any prior assumptions on the reasoning process.

[0085] Step 6: A graph attention network-based model is constructed, which dynamically adjusts the representation of node features by iteratively learning the relationships between nodes and their neighboring nodes in the graph to capture the complex relationship between question and answer entities.

[0086] Step 7: Introduce a probabilistic filtering mechanism to optimize the selection process of neighbor nodes. This mechanism evaluates the correlation between nodes and gives priority to neighbor nodes that are closely related to the target node, thereby improving the model's sensitivity to key information.

[0087] Step 8: Through multiple iterations, the model continuously optimizes the probability distribution of the answer entity. In each iteration, the model combines the current probability distribution and the information of neighboring nodes to update the probability of the answer entity to approach the true answer.

[0088] Step 9: After multiple rounds of iterations, the model outputs the final probability distribution of the answer entity. By sorting the probabilities, the most likely answer entity is selected as the final output, ensuring that the question-answering system can provide highly accurate answers.

[0089] Table 2 compares the performance of various baseline methods for multi-hop question answering on the WebQSP and CWQ datasets. Compared with existing models, GAP has a significant optimization effect.

[0090] Table 2: Performance comparison of different methods on WebQSP and CWQ datasets Table 2 Performance comparison of different methods (Hits@1 or F1 scores in%). Bold fonts

[0091] denote the best methods.

[0092] Method WebqspH@1 / F1 CWQH@1 KV-Mem (Miller et al., 2016) 46.7 / 38.6 21.1 SGReader (Xiong et al., 2019) 67.2 / 57.3 - EmbedKGQA (Saxena et al., 2020) 66.6 / - - GrafNet (Sun et al., 2018) 66.7 / 62.4 32.8 PullNet (Sun et al., 2019) 68.1 / - 45.9 TransferNet (Shi et al., 2021) 71.4 / - 48.6 Rigel (Sen et al., 2021) 73.3 / - 48.7 NSM (He et al., 2021) 68.7 / 62.8 47.6 NSM-distill (He et al., 2021) 74.3 / 67.4 48.8 EmQL (Sun et al., 2020) 75.5 / - - SQALER (Atzeni et al., 2021) 70.6 / - - SQALER+GNN (Atzeni et al., 2021) 76.1 / - - Rev (Mavromatis, et al, 2022) 76.4 / 70.9 52.9 Ours 78.72 / 71.0 53.04

[0093] The present invention will combine background knowledge in the medical field to ensure that the generated questions are not only scientific and accurate, but also consistent with actual clinical scenarios. Through technologies such as deep learning, the model can extract information from historical cases and clinical trial data to generate questions related to the patient's condition. This process will help medical professionals obtain relevant information more efficiently and make more accurate decisions when faced with complex cases. By building a question-and-answer system based on a knowledge graph, the present invention hopes to not only improve the efficiency of information retrieval in the medical field, but also improve the user experience.

[0094] The above description is only a preferred specific implementation manner of the present invention, but the protection scope of the present invention is not limited thereto. Any technician familiar with the technical field can make equivalent replacements or changes according to the technical scheme and inventive concept of the present invention within the technical scope disclosed by the present invention, which should be covered by the protection scope of the present invention.

Claims

1. A medical knowledge graph question answering method based on GAT and probabilistic reasoning, characterized in that: Here are the steps: S1, execution question encoding and instruction decoding; S2, initialize the answer entity probability; S3, build and iterate the GAT network; S4, perform probability filtering and neighbor selection; S5, perform iterative training and probability update; S6. Final answer entity probability ranking.

2. The medical knowledge graph question-answering method based on GAT and probabilistic reasoning as claimed in claim 1, characterized in that: In step S1, the BERT model encodes the input text sequence through the self-attention mechanism and maps each word into a dense vector space. After encoding the natural language question using the BERT model, the generation process of its vector representation is expressed by the following formula: X=BERT(P) Among them, P is the input natural language question, and X is the vector representation obtained after BERT encoding; h=Pooling(X) h is the final vector representation extracted from the output of the BERT model, which is obtained by performing a pooling operation on all word vectors. After obtaining the encoding of the question, the encoding results are deeply analyzed to identify and extract the various instructions contained in the question.

3. The medical knowledge graph question-answering method based on GAT and probabilistic reasoning as claimed in claim 1, characterized in that: In step S2, before question-answering reasoning begins, the probabilities of all possible answer entities are initialized to zero: P A ＝0 Among them, P A is a vector whose length is equal to the total number of answer entities and each element corresponds to the initial probability of an answer entity.

4. The medical knowledge graph question-answering method based on GAT and probabilistic reasoning as claimed in claim 1, characterized in that: In step S3, a network model dominated by GAT is constructed, which updates the probability of the answer entity in an iterative manner; in each iteration, the GAT network adaptively learns the weight between each node and its neighboring nodes. The core formula of GAT is expressed as: h i ′=σ(∑ j∈Ni a ij Whj) Among them, h i ′ is the new feature vector of node i, Nii is the set of neighbor nodes of node i, α ij is the attention weight of node j to node i, W is the shared weight matrix, and σ is the activation function.

5. The medical knowledge graph question answering method based on GAT and probabilistic reasoning as claimed in claim 1, characterized in that: In step S4, in each iteration, the neighbor nodes with higher correlation with the target node are selected through the probability filtering mechanism; this step is achieved by adjusting the attention weight α ij To achieve this, the network pays more attention to those neighbor nodes that have actual contributions to the current task: Among them, e ij is the unnormalized attention score between nodes i and j, computed via a feed-forward neural network.

6. The medical knowledge graph question-answering method based on GAT and probabilistic reasoning as claimed in claim 1, characterized in that: In step S5, the probability distribution of the answer entity is continuously updated through three loop iterations; in each iteration, the network updates the probability of each answer entity based on the current probability distribution and the information of the neighboring nodes: in, is the probability distribution of the answer entity after the tth iteration, and X is the feature matrix of the node.

7. The medical knowledge graph question-answering method based on GAT and probabilistic reasoning as claimed in claim 1, characterized in that: In step S6, after three iterations, the final answer entity probability distribution is obtained; by sorting, the answer entity with the highest probability is selected as the final output: P A (3) is the probability distribution of the answer entity after three iterations.

Images