A knowledge graph-based enhanced large language model question and answer method

Abstract

A knowledge graph enhancement-based large language model question answering method relates to the technical field of knowledge graphs, extracts key entities from medical text, and uses knowledge graphs to construct evidence subgraphs to enhance the reasoning ability of large language models. Graph neural networks are used to extract features and reason about relationships in subgraphs, enhancing the model's understanding of medical scenarios. Global smoothing feature initialization and dynamic weight modeling are used to accurately adjust the relationships between nodes and reduce the reasoning space to the most relevant part. In addition, the subgraph is formatted into an entity chain and converted into a natural language description, which is integrated into a unified reasoning graph to provide a comprehensive perspective for the model. Finally, the model generates the final answer based on the reasoning graph and key reasoning elements, while constructing an explanation context to enhance the model's explainability.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of knowledge graph, and particularly relates to a large language model question and answer method based on knowledge graph enhancement. BACKGROUND

[0002] With the growth and complexity of medical data, and the increasing demand for precision medicine, traditional diagnostic methods are facing challenges. For this reason, large language models (LLMs) are introduced into medical auxiliary diagnosis systems. Through deep learning processing of natural language, LLMs can quickly analyze medical literature and medical records, helping doctors identify disease patterns, predict illness and provide recommendations, thereby significantly improving the efficiency of clinical decision-making. However, large language models sometimes produce "hallucinations" of seemingly reasonable but inaccurate information, which is unacceptable in the medical field that requires high accuracy and reliability. To solve this problem, knowledge graphs (KGs) are introduced, which build a structured network containing concepts such as diseases, symptoms, and drugs and their relationships, enhancing case understanding and supporting logical reasoning. However, relying solely on knowledge graphs cannot fully meet the complex needs of the medical field. KGs, while providing the relationship structure between entities, still have limitations in handling unstructured data and capturing complex contextual information, and their reasoning ability is also limited.

[0003] In order to further improve the reasoning accuracy and explainability of the system, combining graph neural networks (GNNs) has become a new development direction. GNNs can effectively process graph structure data in KGs, through the message passing mechanism between nodes, realizing deeper feature extraction and relationship reasoning. In addition, GNNs can combine unstructured text information with structured KGs, so that the system not only obtains rich semantics from text, but also uses the explicit relationships provided by KGs for accurate reasoning. This not only improves the understanding ability of the model for complex medical scenarios, but also increases the explainability of the decision-making process, significantly reducing the risk of inaccurate or misleading information in the output of the large language model, which is crucial for patient safety. SUMMARY

[0004] The present application provides a large language model question and answer method based on knowledge graph enhancement, which can realize higher level reasoning, improve the accuracy and explainability of the answer, and make up for the shortcomings of LLMs.

[0005] The technical scheme adopted by the present application to overcome the technical problems is:

[0006] A large language model question and answer method based on knowledge graph enhancement, comprising:

[0007] a) obtaining n medical questions to form a medical question dataset Q, Q = {q1, q2,..., qn};i ,...,q n}, where q i For the i-th medical problem;

[0008] b) The i-th medical problem q i Inputting the data into the Qwen2.5-14B-Instruct model, the output is the i-th medical question q. i Preliminary answer a i All n preliminary answers form a preliminary answer set A, A = {a1, a2, ..., a...} i ,...,a n};

[0009] c) From the i-th medical problem q i and the i-th medical problem q i Preliminary answer a i Extract z key entities to form a key entity set E, E = {e1, e2, ..., e...} k ,...,e z}, where e k This is the kth key entity;

[0010] d) Using the Med-BERT model, extract t triple entities from the DiseaseKG knowledge graph to obtain a triple entity set G, G = {u1, u2, ..., u...} j ,...,u t}, where u j Let j be the j-th triplet entity;

[0011] e) Utilizing the k-th key entity e k and the j-th triple entity u j Construct the set of entities with the highest similarity V q ;

[0012] f) Based on the set of entities with the highest similarity V q Constructing the evidence subgraph K q ;

[0013] g) Based on the evidence subgraph K q The final node feature matrix J′ is obtained by constructing the matrix.

[0014] h) Obtain the context enhancement feature T of the i-th neighbor node based on the final node feature matrix J′. i ctx ;

[0015] i) Based on the evidence subgraph K q Construct a new subgraph Z, input the new subgraph Z into the Qwen2.5-14B-Instruct model, and output the natural language interpretation T.m ;

[0016] j) using Apollo Server to build GraphQL interface, calling evidence subgraph K in Apollo Server's resolver through HTTP request q API makes Qwen2.5-14B-Instruct model linked to evidence subgraph K q , the context enhanced feature T i ctx and natural language explanation T m input into Qwen2.5-14B-Instruct model linked to evidence subgraph K q , the output gets the final generated answer A Q .

[0017] Further, in step a), n medical questions are obtained from the CliMedBench medical question dataset.

[0018] Further, in step c), the i-th medical question q i and the preliminary answer a i of the i-th medical question q i are input into the Med-BERT model, and the output gets z key entities.

[0019] Further, step e) includes the following steps:

[0020] e-1) input the k-th key entity e k into the 300-dimensional GloVe model, and output the 300-dimensional key entity vector m k , k∈{1,...,z}, input the j-th triple entity u j into the 300-dimensional GloVe model, and output the 300-dimensional triple entity vector g j , j∈{1,...,t} ;

[0021] e-2) input the key entity vector m k into the 12-layer Tradsformer encoder, and output the text entity embedding with context information h′ e , input the triple entity vector g j into a one-layer graph convolution network GCN, and output the knowledge graph entity embedding with structure information h′ g ;

[0022] e-3) input the text entity embedding with context information h′ e into a multi-layer perceptron MLP, and output the projected embedding h″ eEmbed knowledge graph entities with structural information into h′ g The input is fed into a multilayer perceptron (MLP), and the output is the projected embedding h″. g ;

[0023] e-4) Calculate the vector m of the key entity. k The vector g of the triple entity j The cosine similarity is used to obtain the local similarity S. local (m k ,g j );

[0024] e-5) embed the projected h″ e and the embedded h″ after projection g The input is fed into the attention mechanism to calculate the global similarity S. global (h″ e ,h″ g );

[0025] e-6) via formula The similarity S(m) was calculated. k ,g j In the formula, α and β are both weight values;

[0026] (e-7) Construct a similarity matrix S with z rows and t columns. The value in the k-th row and j-th column of the similarity matrix S is the similarity S(m k ,g j Given k∈{1,...,z}, j∈{1,...,t}, select all vectors m containing the key entity from the similarity matrix S. k The similarity is calculated, and if the selected similarity is greater than or equal to the similarity threshold τ, it is extracted. All extracted similarities are sorted in descending order, and the vectors of the triple entities in the top k similarity values ​​are extracted, where -1≤τ≤1, k=10. The m triple entities selected from the vectors of all z key entities constitute the set V of entities with the highest similarity. q V q ={g1,g2,...,g i ,...,g m}, g i Let be the vector of the i-th triplet entity selected.

[0027] Furthermore, step f) includes the following steps:

[0028] f-1) Establish a queue set D, D = φ, and establish a set of visited nodes V. visited V visited =φ, where φ is the empty set;

[0029] f-2) The set of entities with the highest similarity Vq vector g i of the ith triple entity is added into the queue set D in the form of tuple (g i , 0), i∈{1,...,m}, 0 is the initial depth of the vector g i of the ith triple entity, the vector g i of the ith triple entity is added into the visited node set V visited ;

[0030] f-3) obtaining x triple entities related to the vector g i of the ith triple entity in the DiseaseKG knowledge graph as the neighbor nodes of the vector g i of the ith triple entity, the jth neighbor node of the vector g i of the ith triple entity is g j , j∈{1,...,x}, j≠i; j the jth neighbor node g j is added into the queue set D in the form of tuple (g j , 0+1), 0+1 is the current depth of the jth neighbor node g i , all the x neighbor nodes of the vector g visited of the ith triple entity are added into the visited node set V visited , j∈{1,...,x}, j≠i;

[0031] f-4) cleaning the visited node set V visited , when cleaning, only one of the repeated nodes in the visited node set V JD is kept and the rest of the redundant repeated nodes are deleted;

[0032] f-5) establishing a subgraph node structure K JD , K GX = φ, and establishing a subgraph relationship set K GX ,

[0033] K i = φ;

[0034] f-6) taking a tuple (g i , d) from the queue set D, i∈{1,...,z}, d is the current depth of the vector g max of the ith triple entity, judging whether d is greater than or equal to the maximum depth d i , if yes, deleting the tuple (g i , d) from the queue set D, if not, checking whether the vector g visited of the ith triple entity is in the visited node set V iis not in the visited node set V visited , the vector g i of the ith triple entity is added to the visited node set V visited , and the edge constituted by the relationship between the vector g i of the ith triple entity and each neighbor node is added to the subgraph relationship set K GX , it is detected whether the jth neighbor node g i of the vector g j of the ith triple entity exists in the visited node set V visited , if yes, the jth neighbor node g j is added to the subgraph node structure K JD , and the tuple (g j , d+1) is added to the queue set D.

[0035] f-7) repeat step f-6) until the queue set D is empty.

[0036] f-8) clean the subgraph node structure K JD , in which only one of the repeated neighbor nodes in the subgraph node structure K JD is retained and the remaining redundant repeated neighbor nodes are deleted, to obtain the cleaned subgraph node structure K JD ', and clean the subgraph relationship set K GX , in which only one of the repeated edges in the subgraph relationship set K GX is retained and the remaining redundant repeated edges are deleted, to obtain the cleaned subgraph relationship set K GX ';

[0037] f-9) establish an evidence subgraph K q = (K JD ', K GX ').

[0038] Preferably, d max = 8.

[0039] Further, step g) comprises the following steps:

[0040] g-1) input the evidence subgraph K q into a BERT model, and output a node feature matrix J of N rows and N columns, N being the number of neighbor nodes in the cleaned subgraph node structure K JD ';

[0041] g-2) construct an adjacency matrix A, the value of the ith row and jth column of the adjacency matrix A being a ij , i∈{1,...,N}, j∈{1,...,N}, if the cleaned subgraph node structure K JDa ij = 1 if there is an edge between the i-th neighbor node and the j-th neighbor node in the cleaned subgraph node structure K JD = 0 if there is no edge between the i-th neighbor node and the j-th neighbor node in the cleaned subgraph node structure K ij = 0, a ii = 0, a jj = 0;

[0042] g-3) Add the adjacency matrix A to the N-order identity matrix to obtain the extended adjacency matrix A', and calculate the smoothing weight matrix A by the formula ^ , where D is the degree matrix corresponding to the extended adjacency matrix A';

[0043] g-4) Calculate the first smoothed feature by the formula , where r is a control parameter for the influence degree of neighbor nodes, and r e (0, 1);

[0044] g-5) Calculate the second smoothed feature by the formula

[0045] g-6) Repeat step g-5) t-2 times to obtain the t-th smoothed feature

[0046] g-7) Calculate the difference between the Frobenius norm of the t-th smoothed feature and the Frobenius norm of the (t-1)-th smoothed feature , if the difference is less than or equal to , execute step g-8), if the difference is greater than , execute step g-6) until the difference is less than or equal to

[0047] g-8) Calculate the high-order feature matrix J by the formula (1) , where W (1) is a weight matrix, and (·) is a nonlinear activation function, and the high-order feature matrix J (2) is calculated by the formula J (1) = (A^J (2) W (2) ), where W (2) is a weight matrix;

[0048] g-9) Perform residual connection between the t-th smoothed feature and the high-order feature matrix J (2) to obtain the final node feature matrix J' of N rows and N columns. ​​

[0049] Preferably, in step g-7) Furthermore, step h) includes the following steps:

[0050] h-1) Input the element in the i-th row and j-th column of the final node feature matrix J′ into a multi-head attention mechanism with 8 attention heads, and output the attention score. in Let J' be the attention score of the element in the i-th row and j-th column of the final node feature matrix J' output by the i-th attention head in the multi-head attention mechanism, where m∈{1,2,...,8}, i∈{1,...,N}, j∈{1,...,N}. The input is fed into the softmax function, and the output is the attention weights.

[0051] h-2) Attention weights The feature is obtained by multiplying the value vector V of the i-th attention head.

[0052] h-3) via formula Calculate the time-varying weighted features In the formula γ (m) Let γ be the time-varying weight vector of the i-th attention head, m∈{1,2,...,8}. (m) ∈(0,1), ⊙ is the element-wise multiplication operation;

[0053] h-4) via formula The final node features are calculated. In the formula, α i Let i be the i-th weight value, where i∈{1,2,...,8};

[0054] h-5) Node Features J is the node feature matrix final The element in the i-th row and j-th column of the node feature matrix J final Perform a residual connection with the final node feature matrix J′ to obtain the feature matrix J. out h-6) will transform the characteristic matrix J out The global feature vector P is obtained by averaging the N×N elements. global , the evidence subgraph K q The input is fed into the BERT model, and the output is the feature vectors of all neighboring nodes. The feature vector of the i-th neighboring node is T. i The feature vector of the j-th neighbor node is T. j ;

[0055] h-7) via formula The weighted average attention weight d is calculated ij , β i is the i-th weight value, i∈{1, 2,..., 8}, calculated by the formula

[0056] s ij =d ij ·cosine_similarity(T i ·P global )·cosine_similarity(T j ·P global )

[0057] The importance score s of the j-th neighbor node to the i-th neighbor node is calculated ij , wherein cosine_similarity(T i ·P global ) is the cosine similarity of the feature vector T i of the i-th neighbor node and the global feature vector P global , and cosine_similarity(T j ·P global ) is the cosine similarity of the feature vector T j of the j-th neighbor node and the global feature vector P global , i∈{1,..., N}, j∈{1,..., N};

[0058] h-8) Sort the importance scores of all N-1 neighbor nodes to the i-th neighbor node in descending order, and select the neighbor nodes opposite to the i-th neighbor node in the top k importance scores as the neighborhood set N k (i) of the i-th neighbor node, k=10, and calculate the context-enhanced feature T of the i-th neighbor node by the formula i ctx .

[0059] Further, the method for constructing a new subgraph Z q from the evidence subgraph K i in step i) is as follows: i-1) Construct the set of visited nodes F, F=φ, and construct the edge set R, R=φ;

[0060] i-2) Find y edges from the cleaned subgraph relationship set K GX ' with the vector g i of the i-th triple entity as the starting point, and the ending point of the q-th edge is the neighbor node g i of the vector g j of the i-th triple entity;, i∈{1,...,m}, q∈{1,...,y}, j∈{1,...,x}, add neighbor node g j to the visited node set F, add the qth edge to the edge set R;

[0061] i-3) Establish a new subgraph Z=(F,R).

[0062] The beneficial effects of the present application are: first, key entities are extracted from medical texts, and evidence subgraphs are constructed using knowledge graphs to enhance the reasoning ability of large language models. Feature extraction and relationship reasoning of the subgraph are performed using graph neural networks to enhance the understanding ability of the model in the medical field. Through global smoothing feature initialization and dynamic weight modeling, the relationship between nodes is accurately adjusted, and the reasoning space is narrowed down to the most relevant part. In addition, the subgraph is formatted into an entity chain and converted into a natural language description, which is integrated into a unified reasoning graph to provide a comprehensive perspective for the model. Finally, the model generates the final answer based on the reasoning graph and key reasoning elements, and constructs an explanation context to enhance the explainability of the model. The present application effectively solves the problems of insufficient accuracy and explainability of large language models in the medical field, and provides an innovative solution for the development of medical question and answer systems. BRIEF DESCRIPTION OF DRAWINGS

[0063] Figure 1 The method flowchart of the present application. DETAILED DESCRIPTION

[0064] The following will be described in conjunction with the accompanying Figure 1 Further description of the present application.

[0065] A large language model question and answer method based on knowledge graph enhancement, comprising:

[0066] a) Obtain n medical questions to form a medical question data set Q, Q={q1,q2,...,q i ,...,q n}, wherein q i is the ith medical question.

[0067] b) Input the ith medical question q i into the Qwen2.5-14B-Instruct model, and output the preliminary answer a i of the ith medical question q i , and all n preliminary answers form a preliminary answer set A, A={a1,a2,...,a i ,...,a n}.

[0068] c) From the ith medical question q i and the ith medical question q ithe preliminary answer a i key entities are extracted from the zth passage, forming a key entity set E, E = {e1, e2,..., e k ,...,e z}, where e k is the kth key entity.

[0069] d) t triple entities are extracted from the DiseaseKG knowledge graph using the Med-BERT model, obtaining a triple entity set G, G = {u1, u2,..., u j ,...,u t}, where u j is the jth triple entity.

[0070] e) the highest similarity entity set V k is constructed using the kth key entity e j and the jth triple entity u q ; f) the evidence subgraph K q is constructed according to the highest similarity entity set V q .

[0071] g) the final node feature matrix J' is constructed according to the evidence subgraph K q .

[0072] h) the context-enhanced feature T i ctx of the ith neighbor node is obtained according to the final node feature matrix J'.

[0073] i) a new subgraph Z is constructed according to the evidence subgraph K q , and the new subgraph Z is input into the Qwen2.5-14B-Instruct model to output the natural language explanation T m .

[0074] j) the GraphQL interface is built using the Apollo Server, and the API of the evidence subgraph K q is called through the HTTP request in the resolver of the Apollo Server to link the Qwen2.5-14B-Instruct model to the evidence subgraph K q , and the context-enhanced feature T i ctx and the natural language explanation T m are input into the Qwen2.5-14B-Instruct model linked to the evidence subgraph K q , and the final generated answer A Q is output.

[0075] Solve the problem of lack of explainability and insufficient understanding of professional field in the reasoning process of large language model (LLM). First, key entities are extracted from the input question and the preliminary answer of LLM, which are factual assertions that need to be further verified. The key information is extracted from the text by using the ability of LLM itself. Then, the extracted key entities are encoded with the entities in the external knowledge graph, a cosine similarity matrix is calculated, and a set of entities with the highest similarity score is obtained to construct an evidence subgraph.

[0076] Then, the evidence subgraph is constructed based on the extracted entities from the knowledge graph, and a compact and highly relevant evidence graph is generated.

[0077] In processing the evidence subgraph, the context-enhanced features are extracted from the evidence subgraph as key reasoning elements by using the global smoothing initialization feature method and the dynamic weight modeling local relationship method, which captures global dependency information and local relationship changes, and accurately narrows down the reasoning space to the most relevant knowledge part. All subgraphs are formatted into entity chains, and the knowledge graph information is converted into natural language explanation by using a large language model to provide a comprehensive perspective of all evidence subgraphs.

[0078] Based on the natural language explanation of the selected information and key reasoning elements, the large language model is used to verify each factual assertion and propose revision suggestions, and the preliminary answer is adjusted according to the verification result, which improves the reasoning ability and accuracy of the large language model in the medical field, and also enhances the explainability, so that users can better understand and trust the decision-making process of the model.

[0079] In an embodiment of the present application, n medical questions are obtained from the CliMedBench medical question dataset in step a).

[0080] In an embodiment of the present application, the i-th medical question q i and the preliminary answer a i of the i-th medical question q i are input into the Med-BERT model in step c), and z key entities are output.

[0081] In an embodiment of the present application, step e) includes the following steps:

[0082] e-1) input the k-th key entity e k into the 300-dimensional GloVe model to output the 300-dimensional vector m k of the key entity, k∈{1,...,z}, and input the j-th triple entity u j into the 300-dimensional GloVe model to output the 300-dimensional vector g j of the triple entity, j∈{1,...,t}.

[0083] e-2) The vector m of the key entity k The input is fed into a 12-layer Tradsformer encoder, and the output is a text entity embedding h′ with context information. e The vector g of the triplet entity j The input is fed into a single-layer graph convolutional network (GCN), and the output is a knowledge graph entity embedding h′ with structural information. g .

[0084] e-3) Embed text entities with contextual information into h′ e The input is fed into a multilayer perceptron (MLP), and the output is the projected embedding h″. e Embed knowledge graph entities with structural information into h′ g The input is fed into a multilayer perceptron (MLP), and the output is the projected embedding h″. g .

[0085] e-4) Calculate the vector m of the key entity. k The vector g of the triple entity j The cosine similarity is used to capture the matching relationship between substructures and fine-grained features, resulting in the local similarity S. local (m k ,g j );

[0086] e-5) embed the projected h″ e and the embedded h″ after projection g The input is fed into the attention mechanism to capture the matching relationships of overall semantic features, and the global similarity S is calculated. global (h″ e ,h″ g ).

[0087] e-6) via formula The similarity S(m) was calculated. k ,g j In the formula, α and β are both weight values.

[0088] (e-7) Construct a similarity matrix S with z rows and t columns. The value in the k-th row and j-th column of the similarity matrix S is the similarity S(m k ,g j Given k∈{1,...,z}, j∈{1,...,t}, select all vectors m containing the key entity from the similarity matrix S. kthe similarity if the selected similarity is greater than or equal to the similarity threshold τ, taking out the selected similarity, sorting all the taken-out similarities in descending order, taking out the vectors of the triple entity in the first k similarities after descending order sorting, -1≤τ≤1, k=10, the m triple entities screened from the vectors of all z key entities constitute a highest similarity entity set V q , V q ={g i ,...,g m}, g i is the vector of the i-th triple entity screened.

[0089] In an embodiment of the present application, step f) comprises the following steps:

[0090] f-1) establishing a queue set D, D=φ, establishing a visited node set V visited , V visited =φ, φ is an empty set.

[0091] f-2) adding the vector g q of the i-th triple entity in the highest similarity entity set V i in the form of a tuple (g i ,0) into the queue set D, i∈{1,...,m}, 0 is the initial depth of the vector g i of the i-th triple entity, and adding the vector g i of the i-th triple entity as a node into the visited node set V visited .

[0092] f-3) obtaining x triple entities having a relationship with the vector g i of the i-th triple entity in the DiseaseKG knowledge graph as neighbor nodes of the vector g i of the i-th triple entity, the j-th neighbor node of the vector g i of the i-th triple entity being g j , adding the j-th neighbor node g j in the form of a tuple (g j ,0+1) into the queue set D, 0+1 being the current depth of the j-th neighbor node g j , and adding all the x neighbor nodes of the vector g i of the i-th triple entity into the visited node set V visited , j∈{1,...,x}, j≠i.

[0093] f-4) cleaning the visited node set V visited , when cleaning, for the visited node set V visitedThe nodes with duplicates are kept only one and the rest of the redundant duplicate nodes are deleted.

[0094] f-5) Establishing the subgraph node structure K JD , K JD = φ, establishing the subgraph relation set K GX ,

[0095] K GX = φ.

[0096] f-6) Taking one tuple (g i , d) from the queue set D, i∈{1,...,z}, d is the current depth of the ith triple entity vector g i , judging whether d is greater than or equal to the maximum depth d max , if yes, deleting the tuple (g i , d) from the queue set D, if not, checking whether the ith triple entity vector g i is in the visited node set V visited , if the ith triple entity vector g i is not in the visited node set V visited , adding the ith triple entity vector g i to the visited node set V visited and adding the edge formed by the ith triple entity vector g i and the relation of each neighbor node to the subgraph relation set K GX , detecting whether the jth neighbor node g i of the ith triple entity vector g j exists in the visited node set V visited , if yes, adding the jth neighbor node g j to the subgraph node structure K JD and adding the tuple (g j , d+1) to the queue set D.

[0097] f-7) Repeating step f-6) until the queue set D is empty.

[0098] f-8) Cleaning the subgraph node structure K JD , when cleaning, keeping only one of the repeated neighbor nodes in the subgraph node structure K JD and deleting the rest of the redundant duplicate neighbor nodes, obtaining the cleaned subgraph node structure K JD ', cleaning the subgraph relation set K GX , when cleaning, keeping only one of the repeated edges in the subgraph relation set K GX and deleting the rest of the redundant duplicate edges, obtaining the cleaned subgraph relation set KGX ′.

[0099] f-9) Establishing evidence subgraph K q =(K JD ′,K GX ′).

[0100] In this embodiment, preferably, d max =8.

[0101] In one embodiment of the present application, step g) comprises the following steps:

[0102] g-1) When processing evidence subgraph K q , the method of the present application is significantly superior to traditional LLM methods and methods using only KG in terms of generation quality, reasoning accuracy and hallucination in the medical field through comparative experiments, especially in the processing of complex problems in the medical field, it can generate higher quality and more accurate answers and provide stronger explainability. Specifically, the evidence subgraph K q is input into the BERT model, and the output obtains an N by N node feature matrix J, N is the number of neighbor nodes in the cleaned subgraph node structure K JD ′.g-2) In order to propagate node feature information in the graph, an adjacency matrix A is constructed, the value of the i-th row and j-th column of the adjacency matrix A is a ij , i∈{1,...,N}, j∈{1,...,N}, if there is an edge composed of the i-th neighbor node and the j-th neighbor node in the cleaned subgraph node structure K JD ′, then a ij =1, if there is no edge composed of the i-th neighbor node and the j-th neighbor node in the cleaned subgraph node structure K JD ′, then a ij =0, a ii =0, a jj =0.

[0103] g-3) Add the adjacency matrix A and the N-order identity matrix to obtain the extended adjacency matrix A′, and calculate the smoothing weight matrix A ^ through the formula , where D is the degree matrix corresponding to the extended adjacency matrix A′, used to balance the amplitude of feature propagation.

[0104] g-4) Calculate the first smoothed feature through the formula , where r is the degree of control of the influence of neighbor nodes, r∈(0,1).

[0105] g-5) Calculate the second smoothed feature through the formula

[0106] g-6) Repeat step g-5) t-2 times to obtain the smooth feature of the t-th iteration.

[0107] g-7) Solve for the smoothing feature of the t-th iteration. The Frobenius norm and the smoothness feature of the (t-1)th order The difference of the Frobenius norm, if the difference is less than or equal to Then proceed to step g-8). If the difference is greater than... Then proceed to step g-6) until the difference is less than or equal to g-8) via formula The higher-order characteristic matrix J is calculated. (1) In the formula W (1) Let J be the weight matrix, and (·) be the nonlinear activation function, obtained through formula J. (2) =(A^J (1) W (2) The higher-order characteristic matrix J is calculated. (2) In the formula W (2) This is the weight matrix.

[0108] g-9) will smooth the feature at the t-th time. With the higher-order characteristic matrix J (2) Perform residual connections to preserve the original feature information, resulting in a final node feature matrix J′ with N rows and N columns.

[0109] In this embodiment, preferably, in step g-7)

[0110] In one embodiment of the present invention, the relationships between nodes may differ in different stages and tasks of the graph. Therefore, a method for dynamic weight modeling of local relationships is proposed to model the relationships between nodes in the evidence subgraph in greater detail. Thus, we introduce time-varying relationship modeling, where the strength of the relationship between a node and its neighbors changes dynamically as training progresses. In this way, the network can gradually adjust the weights of local relationships according to the needs of the task and the structure of the graph. Specifically, step h) includes the following steps:

[0111] h-1) Input the element in the i-th row and j-th column of the final node feature matrix J′ into a multi-head attention mechanism with 8 attention heads, and output the attention score. in Let J' be the attention score of the element in the i-th row and j-th column of the final node feature matrix J' output by the i-th attention head in the multi-head attention mechanism, where m∈{1,2,...,8}, i∈{1,...,N}, j∈{1,...,N}. Input into the softmax function, output the attention weight

[0112] h-2) multiply the attention weight with the value vector V of the i-th attention head to obtain the feature

[0113] h-3) calculate the feature adjusted by the time-varying weight by the formula where γ (m) is the time-varying weight vector of the i-th attention head, used to understand the relationship strength between nodes, m∈{1,2,...,8}, γ (m) ∈(0,1), and is the element-wise multiplication operation.

[0114] h-4) calculate the final node feature by the formula where α i is the i-th weight value, i∈{1,2,...,8}. h-5) the node feature is the i-th row and j-th column element in the node feature matrix J final final out .h-6) take the average of the N×N elements in the feature matrix J out global q , input the evidence subgraph K i into the BERT model, and output the feature vectors of all neighbor nodes, the feature vector of the i-th neighbor node is T j , and the feature vector of the j-th neighbor node is T ij .

[0115] h-7) calculate the weighted average attention weight d i by the formula

[0116] ij ij i global j global ij ​​​​​​​​​​​​​​where cosine_similarity(T i · P global ) is the cosine similarity between the feature vector of the i-th neighbor node T i and the global feature vector P global , cosine_similarity(T j · P global ) is the cosine similarity between the feature vector of the j-th neighbor node T j and the global feature vector P global , i∈{1,...,N}, j∈{1,...,N}.

[0117] h-8) Finally, in the selection of key neighborhoods and multi-layer stacking, we introduce a recursive neighborhood expansion mechanism combined with sparsification selection, which can gradually expand the neighborhood of the evidence subgraph at each layer, while ensuring more efficient information aggregation at each layer. Specifically, sort the importance scores of all N-1 neighbor nodes to the i-th neighbor node in descending order, and select the neighbor nodes opposite to the i-th neighbor node in the top k importance scores as the neighborhood set N k (i) of the i-th neighbor node, k = 10, and calculate the context-enhanced feature T i ctx of the i-th neighbor node by formula .

[0118] In an embodiment of the present application, the method for constructing a new subgraph Z q from the evidence subgraph K i in step i) is as follows:

[0119] i-1) Construct the visited node set F, F = φ, and the edge set R, R = φ.

[0120] i-2) Find y edges from the cleaned subgraph relationship set K GX ′, with the vector g i of the i-th triple entity as the starting point, and the neighbor node g i of the vector g j of the i-th triple entity as the end point of the q-th edge, i∈{1,...,m}, q∈{1,...,y}, j∈{1,...,x}, add the neighbor node g j to the visited node set F, and add the q-th edge to the edge set R.

[0121] i-3) Establish a new subgraph Z = (F, R).

[0122] To verify the effectiveness of the knowledge graph enhanced large language model medical question answering method of the present application, we evaluated the performance of the present application in complex medical question answering tasks through two medical question answering data sets GenMedGPT-5k (as shown in Table I) and CMCQA (as shown in Table II). We used two indicators, BERTScore and GPT-4Rating, for quantitative evaluation. BERTScore measures the semantic similarity between the generated answer and the reference answer. GPT4 is used to sort the quality of the answer according to the true situation, and the answer is compared according to four standards: reaction diversity and integrity, overall factual correctness, disease diagnosis correctness and drug recommendation correctness. And the illusion index is calculated, by calculating the tfidf similarity score between different output sentences. The lower the score, the more illusions there are in the answer.

[0123] Table I BERT score and GPT-4 ranking of all methods on the GenMedGPT-5k dataset

[0124] Method BERT score GPT-4 rank hallucination Inventive method 0.7896 0.8189 0.8154 GPT-3.5 0.7612 0.7945 0.8001 BM25 Retriever 0.7583 0.7816 0.7425 KG Retriever 0.7698 0.8030 0.7866 Tree-of-thought 0.7202 0.7949 0.7658 GPT-4 0.7789 0.8102 0.8065

[0125] Table II BERT score and GPT-4 ranking of all methods on the CMCQA dataset

[0126] Method BERT score GPT-4 rank hallucination Inventive method 0.9516 0.9324 0.8962 GPT-3.5 0.9226 0.9310 0.8156 BM25 Retriever 0.9102 0.9289 0.7861 KG Retriever 0.9356 0.9246 0.8068 Tree-of-thought 0.9286 0.9166 0.8234 GPT-4 0.9396 0.9331 0.8315

[0127] Through comparative experiments, the method of the present application is significantly better than the traditional LLM method and the method using only KG in terms of generation quality, reasoning accuracy and illusion in the medical field, especially in the processing of complex problems in the medical field, it can generate higher quality and more accurate answers, and provides stronger explainability.

[0128] Finally, it should be noted that: the above only describes the preferred embodiments of the present application and is not intended to limit the present application. Although the present application has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent replacements to some technical features. Any modification, equivalent replacement, improvement, etc. within the spirit and principles of the present application shall be included in the protection scope of the present application.

Claims

1. A knowledge graph enhancement-based large language model question and answer method, characterized in that, comprising: a) obtaining a medical problem, forming a medical problem dataset , wherein is a first medical problem; b) inputting the first medical question into the Qwen 2.5-14B-Instruct model, outputting a first preliminary answer to the first medical question, all first preliminary answers forming a first preliminary answer set ;​​​​​​​ c) From the A medical problem Passing the exam A medical problem Preliminary answer Extract Several key entities form a key entity set. , ,in For the first One key entity; d) extracting a triple entity from the DiseaseKG knowledge graph by using the Med-BERT model to obtain a triple entity set , wherein is the i-th triple entity .​ e) utilizing the first key entity and the second triple entity to construct a highest similarity entity set ; f) Based on the set of entities with the highest similarity Constructing evidence subgraphs , Subgraph node structure Cleaning is performed, and the subgraph node structure is cleaned during the cleaning process. Only one duplicate neighbor node is retained, and the rest of the redundant duplicate neighbor nodes are deleted, resulting in the cleaned subgraph node structure. Set the subgraph relations Cleaning is performed on the set of subgraph relations. Only one of each repeated edge is kept, and the rest of the redundant repeated edges are deleted, resulting in the cleaned subgraph relation set. ; g) According to the evidence sub-graph Constructing the final node feature matrix ; h) according to the final node feature matrix get the context enhanced features of the neighbor node ; i) constructing new subgraphs from evidence subgraphs constructing new subgraphs inputting the new subgraphs into the Qwen2.5-14B-Instruct model, outputting natural language explanations ; j) using Apollo Server to build a GraphQL interface, calling the evidence subgraph in the Apollo Server's resolver via HTTP request The API enables the Qwen2.5-14B-Instruct model to link to the evidence subgraph , inputting the context augmented features and natural language explanations into the Qwen2.5-14B-Instruct model linked to the evidence subgraph , outputting the final generated answer ; Step g) comprises the following steps: g-1) the evidence subgraph is input into the BERT model, and the output is obtained row node feature matrix , is the node structure of the cleaned subgraph the number of neighbor nodes in the subgraph g-2) constructing an adjacency matrix adjacency matrix the value of the first column of the first row of the adjacency matrix , , if there is an edge in the cleaned subgraph node structure consisting of the relationship of the first neighbor node and the first neighbor node, then if there is no edge in the cleaned subgraph node structure consisting of the relationship of the first neighbor node and the first neighbor node, then , , ; g-3) will use the adjacency matrix and Adding the identity matrices of order X together yields the extended adjacency matrix. Through formula The smoothing weight matrix is ​​calculated. In the formula To expand the adjacency matrix The corresponding degree matrix; g-4) by the formula calculating the first smoothed feature , where is a control parameter that determines the degree of influence of the neighbor node feature, ; g-5) by the formula calculating the second smoothed feature ; g-6) repeat step g-5) second time to obtain a second smoothed feature ; g-7) solving the equation Frobenius norm of the smoothed feature of order Frobenius norm of the smoothed feature of order if the difference is less than or equal to then step g-8) is performed, if the difference is greater than then step g-6) is performed until the difference is less than or equal to ;​ g-8) by the formula to obtain the high-order feature matrix , where is the weight matrix, is a nonlinear activation function, by the formula to obtain the high-order feature matrix , where is the weight matrix; g-9) the first smooth feature with the high-order feature matrix residual connection, obtaining row column final node feature matrix .

2. The knowledge graph-based enhanced large language model question answering method according to claim 1, characterized in that: the medical question from the CliMedBench medical question dataset in step a) medical question.

3. The knowledge graph-based enhanced large language model question answering method according to claim 1, characterized in that: In step c), the first A medical problem Passing the exam A medical problem Preliminary answer The input is given to the Med-BERT model, and the output is obtained. A key entity.

4. The knowledge graph-based enhanced large language model question answering method according to claim 1, characterized in that, Step e) comprises the following steps: e-1) input the first key entity into a 300-dimensional GloVe model, output a 300-dimensional vector of the key entity , e-2) input the first triple entity into a 300-dimensional GloVe model, output a 300-dimensional vector of the triple entity , ;​​​​ e-2) inputting the vector of the key entity into an encoder of a 12-layer Tradsformer, and outputting a text entity embedding with context information e-3) inputting the vector of the key entity into a one-layer graph convolution network (GCN), and outputting a knowledge graph entity embedding with structure information e-4) inputting the vector of the key entity into a one-layer graph convolution network (GCN), and outputting a knowledge graph entity embedding with structure information e-5) inputting the vector of the key entity into a one-layer graph convolution network (GCN), and outputting a knowledge graph entity embedding with structure information ; e-3) embedding of text entities with contextual information into a multi-layer perceptron, MLP, and outputting the projected embedding embedding of knowledge graph entities with structural information into a multi-layer perceptron, MLP, and outputting the projected embedding ; e-4) computing the vector of the key entity cosine similarity with the vector of the triple entity ;​ e-5) the projected embedding and the projected embedding is input into the attention mechanism to compute the global similarity ; e-6) by the formula The similarity is calculated wherein and are weight values. e-7) establishing row column of the similarity matrix , the value of the row the column of the similarity matrix is the similarity , , , select all the vectors containing key entities in the similarity matrix , if the selected similarity is greater than or equal to the similarity threshold , take it out, sort all the selected similarities in descending order, take out the vector of the triple entity in the top similarity after descending order sorting, , , all key entity vector screened the top triple entity constitutes the highest similarity entity set , the vector of the triple entity screened.​ 5. The knowledge graph-based enhanced large language model question answering method according to claim 4, characterized in that, Step f) comprises the following steps: f-1) establishing a set of queues , , establishing a set of visited nodes , , is an empty set; f-2) adding the highest similarity entity set to the queue set the vector of the i-th triple entity the vector of the i-th triple entity ​​​​​​​​​​ f-3) obtaining the vector of the first triple entity in the DiseaseKG knowledge graph ​​​​​​​​​​​​​​​​​​​​​ f-4) the set of visited nodes cleaning, when cleaning, the set of visited nodes only one of the repeated nodes is kept and the rest of the redundant repeated nodes are deleted f-5) Establishing subgraph node structure , , Establishing subgraph relationship set , ; f-6) From the queue set Take a tuple from , , For the first Vector of triplet entities Determine the current depth. Is it greater than or equal to the maximum depth? If so, then the tuple From queue collection Delete it, otherwise check the first one. Vector of triplet entities Is it in the set of visited nodes? In the middle, if the first Vector of triplet entities Not in the set of visited nodes In the middle, the first Vector of triplet entities Add to the set of visited nodes And will be by the first Vector of triplet entities The edges formed by the relationships with each neighbor node are added to the subgraph relationship set. In the middle, the test, the first Vector of triplet entities The neighboring nodes Does it exist in the set of visited nodes? In the middle, if so, then the first neighboring nodes Add to subgraph node structure And will tuple Add to queue collection middle; f-7) repeat step f-6) until the set of queues is complete empty; f-8) cleaning the subgraph node structure cleaning, in which, for the subgraph node structure only one of the repeated neighbor nodes is kept, and the rest of the redundant repeated neighbor nodes are deleted, to obtain the cleaned subgraph node structure cleaning, in which, for the subgraph relationship set cleaning, in which, for the subgraph relationship set only one of the repeated edges is kept, and the rest of the redundant repeated edges are deleted, to obtain the cleaned subgraph relationship set ; f-9) Establishing evidence subgraph .

6. The knowledge graph-based enhanced large language model question answering method according to claim 5, characterized in that: 。 7. The knowledge graph enhanced large language model question answering method according to claim 1, characterized in that: in step g-7) .

8. The knowledge graph enhanced large language model question answering method according to claim 1, characterized in that, Step h) comprises the following steps: h-1) will be the final node feature matrix The Middle Line 1 The elements of the column are input into a multi-head attention mechanism with 8 attention heads, and the output is the attention score. ,in For the multi-head attention mechanism The final node feature matrix output by each attention head. The Middle Line 1 Attention scores of the elements in the column. , , ,Will The input is fed into the softmax function, and the output is the attention weights. ; h-2) multiplying the attention weight with the value vector of the th attention head to obtain the feature ; h-3) by the formula the time-varying weight adjusted features , where is the time-varying weight vector for the th attention head, , , is an element-wise multiplication operation; h-4) by the formula calculating the final node features wherein, is the weight value, ; h-5) node features is a node feature matrix is a node feature matrix is a node feature matrix is a node feature matrix is a node feature matrix is a node feature matrix is a node feature matrix h-6) the feature matrix is averaged to obtain a global feature vector is averaged to obtain a global feature vector , the evidence subgraph is input into the BERT model, and the feature vectors of all neighbor nodes are output, the feature vector of the first neighbor node is , the feature vector of the first neighbor node is ; h-7) by the formula The weighted average attention weight is calculated , is the first weight value, by the formula The importance score of the i-th neighbor node is calculated as wherein is the cosine similarity between the feature vector of the i-th neighbor node and the global feature vector is the cosine similarity between the feature vector of the i-th neighbor node and the global feature vector ;​​​​​​​​​ h-8) all neighbor nodes are ranked in descending order of their importance scores to the neighbor node, and the top neighbor nodes with the highest importance scores relative to the neighbor node are selected as the neighborhood set of the neighbor node , The context-enhanced feature of the neighbor node is calculated by the formula .​ 9. The knowledge graph enhanced large language model question answering method according to claim 5, characterized in that, In step i) the evidence subgraph according to The method of constructing a new subgraph is: i-1 ) constructing a set of visited nodes , , constructing a set of edges , ; i-2) finding from the set of subgraph relations after cleaning ​​​​​​​​​​​​​​​ i-3) Establishing a new subgraph .

Images