A medical question-answering method and system based on knowledge enhancement and reasoning optimization

Abstract

The present invention proposes a medical question-answering method and system based on knowledge enhancement and reasoning optimization, which relates to the technical fields of artificial intelligence and medical information processing. The method includes obtaining a medical corpus, preprocessing the medical corpus to obtain a preprocessed medical corpus; using the preprocessed medical corpus to train a preset language model to obtain an initial medical question-answering model for outputting a semantic vector; using a preset knowledge graph to generate a knowledge embedding vector, fusing the knowledge embedding vector with the semantic vector to obtain a fused vector; using the fused vector as input to the initial medical question-answering model, fine-tuning the initial medical question-answering model to obtain a fine-tuned model; performing reasoning optimization on the fine-tuned model to obtain a trained medical question-answering model; and inputting a medical question to be processed into the medical question-answering model to output a predicted answer. The present invention can effectively improve the efficiency, accuracy, and reliability of medical question-answering.

Description

Technical Field

[0001] The present invention relates to the technical field of artificial intelligence and medical information processing, and in particular to a medical question-answering method and system based on knowledge enhancement and reasoning optimization. Background Art

[0002] With the rapid development of information technology, artificial intelligence technology is being used more and more widely in the medical field, especially in medical question-answering systems. By combining natural language processing technology and deep learning algorithms, the system can understand and answer medical-related questions raised by patients or medical professionals. Therefore, the study of medical question-answering is of great significance to the demand for medical services.

[0003] In the current medical information processing environment, traditional medical knowledge retrieval and question-answering systems often face multiple challenges in the face of increasingly complex and diverse medical question-answering needs. First, existing systems often rely on handcrafted rules or simple matching algorithms. This approach is prone to misunderstandings when dealing with highly specialized and complex medical questions, and fails to provide accurate and authoritative answers. Second, these systems lack effective knowledge integration and reasoning capabilities when faced with large-scale medical knowledge bases, resulting in an inability to fully utilize existing medical resources to provide high-quality question-answering services. Furthermore, doctors and patients increasingly demand accuracy and interpretability in answers, which existing systems struggle to meet. With the rapid development of artificial intelligence technologies, particularly large-scale language models, deep learning-based medical question-answering models have demonstrated strong performance in the field of natural language processing. However, existing medical question-answering models lack effective knowledge integration and deep reasoning capabilities when faced with diverse and complex medical questions, resulting in low efficiency, low accuracy, and poor reliability in medical question-answering. Summary of the Invention

[0004] In order to solve the problems of low efficiency, low accuracy and poor reliability of medical question answering in the above-mentioned existing technologies, the present invention proposes a medical question answering method and system based on knowledge enhancement and reasoning optimization, which can effectively improve the efficiency, accuracy and reliability of medical question answering.

[0005] In order to achieve the above technical effects, the technical solutions of the present invention are as follows:

[0006] A medical question answering method based on knowledge enhancement and reasoning optimization includes the following steps:

[0007] S1. Obtaining a medical corpus, and preprocessing the medical corpus to obtain a preprocessed medical corpus;

[0008] S2. Using the preprocessed medical corpus to train a preset language model, obtaining an initial medical question-answering model for outputting semantic vectors;

[0009] S3. Generate a knowledge embedding vector using a preset knowledge graph, and fuse the knowledge embedding vector with the semantic vector to obtain a fused vector;

[0010] S4. Using the fusion vector as input to the initial medical question answering model, fine-tuning the initial medical question answering model to obtain a fine-tuned model;

[0011] S5. Performing inference optimization on the fine-tuned model to obtain a trained medical question-answering model;

[0012] S6. Input the medical question to be processed into the medical question-answering model and output the answer prediction result.

[0013] Preferably, the preprocessing of the medical corpus comprises:

[0014] S101. Define the medical corpus as where d i is the i-th document, and N is the total number of documents;

[0015] S102. The medical corpus is Clean and get the cleaned corpus The calculation expression is as follows:

[0016]

[0017] Among them, valid(d) is the data verification function;

[0018] S103. Cleaned corpus Perform word segmentation to generate a set of subword units The calculation expression is as follows:

[0019]

[0020] Among them, BPE(.) is the encoding function;

[0021] S104. Group the subword units The L-th subword unit x in L Split into fixed-length input sequences X i as follows:

[0022] X i =[x1,x2,…,x L ]

[0023] Where L is the sequence length;

[0024] S105. Input sequence X i As a preprocessed medical corpus.

[0025] Preferably, the language model includes an input layer, a Transformer encoding layer and an output layer, wherein the input layer is connected to the input end of the Transformer encoding layer, the output end of the Transformer encoding layer is connected to the output layer, and the input layer converts the input sequence X into i The embedded matrix is ​​input into the Transformer encoding layer, which includes multiple layers of Transformer encoders. Each layer of Transformer encoder uses a multi-head self-attention mechanism and a feedforward neural network to encode the embedded matrix to obtain an encoded output result.

[0026] Preferably, each layer of the Transformer encoder encodes the embedding matrix using a multi-head self-attention mechanism and a feedforward neural network to obtain an encoding output result, including:

[0027] S201. The first layer Transformer encoder is added to the output of the previous layer Transformer encoder through the multi-head self-attention mechanism and then normalized to obtain the initial output result H (l) The calculation expression is as follows:

[0028] H (l) =LayerNorm(H (l-1) +Attention(Q (l) ,K (l) ,V (l) ))

[0029] Among them, H (l-1) is the output of the l-1th layer Transformer encoder, LayerNorm(.) is the layer normalization operation, Attention is the multi-head self-attention mechanism, Q (l) is the query matrix of the l-th layer Transformer encoder, K (l) is the key matrix of the l-th layer Transformer encoder, V (l) is the value matrix of the l-th layer Transformer encoder;

[0030] S202. The initial output result H (l) Input the feedforward neural network in the lth layer Transformer encoder to obtain the output of the current layer feedforward neural network, and compare the output of the current layer feedforward neural network with the initial output result H (l) Add them together and go through layer normalization to get the encoding output H′ of the lth layer Transformer encoder (l) as follows:

[0031] H′ (l) =LayerNorm(H (l) +FFN(H (l) ))

[0032] Among them, FFN(.) is a feedforward neural network, H (l) is the initial output result of the l-th layer Transformer encoder.

[0033] Preferably, the first training total loss function is used The language model is trained, and the calculation expression of the first training total loss function is:

[0034]

[0035] Among them, α is a hyperparameter used to control the weight of the masked language model task, and β is a hyperparameter used to control the weight of the next sentence prediction task; is the loss function for training the masked language model task, The loss function for training the next sentence prediction task.

[0036] Preferably, the knowledge graph is represented as a directed graph in is the node set containing medical entities; ε is the edge set used to describe the relationship between medical entities, and each edge is represented by e=(v i ,v j ,r), where r is the relationship type; generating a knowledge embedding vector using a preset knowledge graph includes:

[0037] S301. Based on the nodes in the knowledge graph, a multi-layer graph neural network is used to calculate the node embedding vectors in the knowledge graph as follows:

[0038]

[0039] in, is the node embedding calculated by the l-th layer graph neural network, σ is the activation function, is a set of relationship types, For node v i The neighbor set of is the attention parameter, W r is the projection matrix of relation type r, is the node embedding calculated by the l-1th layer of the graph neural network;

[0040] S302. Based on node embedding Calculate the final node embedding vector h i The expression is as follows:

[0041]

[0042] Where L is the total number of layers of the graph neural network;

[0043] S303. Based on the node embedding vector h i , calculate the knowledge embedding vector H G The expression is as follows:

[0044] H G =[h1,h2,…,h n ]

[0045] Where n is the total number of nodes in the knowledge graph.

[0046] Preferably, the knowledge embedding vector is fused with the semantic vector to obtain a fusion vector Z fusion The calculation expression is as follows:

[0047] Z fusion =Concat(H L ,H G )W z

[0048] Among them, H L is the knowledge embedding vector, H G is the semantic vector, Concat(·) is the vector concatenation operation, W f is the fusion mapping matrix in the fine-tuning training phase.

[0049] Preferably, the second training total loss function is used Fine-tune the initial medical question answering model, and the second training total loss function The calculation expression is as follows:

[0050]

[0051] in, The optimization objective loss function generated for the answer, is the semantic loss function;

[0052] The optimization objective loss function The calculation expression is as follows:

[0053]

[0054] Among them, P(.) is the probability distribution function, T is the total number of generated true answer words, is the tth generated true answer word in the i-th sample, Generate a sequence for the history in the i-th sample, Qi is the input sequence of the i-th medical question, θ is the model parameter;

[0055] The semantic loss function The calculation expression is as follows:

[0056]

[0057] Among them, Embed(·) is the answer semantic embedding function, N is the total number of samples, A i is the true answer to the question, The answer generated for the fine-tuned model, ||.|| represents the norm of the vector.

[0058] Preferably, performing inference optimization on the fine-tuning model includes:

[0059] S501. Initialize the reasoning path using the fusion vector and medical question after fine-tuning the fine-tuning model;

[0060] S502. Decompose the medical problem into multiple sub-problems under the reasoning path, and generate an answer corresponding to each sub-problem;

[0061] S503. Calculate the total path loss function of the reasoning path based on the answer corresponding to each sub-question and the weight adjustment item of the reasoning path;

[0062] S504. Perform path optimization on the path loss function to obtain a medical question-answering model that outputs a final answer.

[0063] The present invention also proposes a medical question-answering system based on knowledge enhancement and reasoning optimization, comprising:

[0064] A preprocessing module is used to obtain a medical corpus and preprocess the medical corpus to obtain a preprocessed medical corpus;

[0065] A model training module is used to train a preset language model using the preprocessed medical corpus to obtain an initial medical question-answering model for outputting semantic vectors;

[0066] A knowledge embedding module is used to generate a knowledge embedding vector using a preset knowledge graph, and fuse the knowledge embedding vector with the semantic vector to obtain a fusion vector;

[0067] a fine-tuning module, configured to use the fusion vector as input to the initial medical question-answering model, and fine-tune the initial medical question-answering model to obtain a fine-tuned model;

[0068] An inference optimization module, configured to perform inference optimization on the fine-tuning model to obtain a trained medical question-answering model;

[0069] The prediction module is used to input the medical question to be processed into the medical question-answering model and output the answer prediction result.

[0070] Compared with the prior art, the beneficial effects of the technical solution of the present invention are:

[0071] The present invention proposes a medical question-answering method and system based on knowledge enhancement and reasoning optimization. First, a preprocessed medical corpus is used to construct an initial medical question-answering model. By fusing the knowledge embedding vector generated by the knowledge graph and the semantic vector output by the initial medical question-answering model, the initial medical question-answering model's ability to understand and apply medical knowledge is enhanced. Then, the initial medical question-answering model is fine-tuned in a supervised manner, and the preference for detailed reasoning process is introduced for reasoning optimization to obtain a medical question-answering model. The medical question-answering model is further used to predict answers to the medical questions to be processed. The present invention not only improves the efficiency and accuracy of medical question-answering, but also ensures the reliability of the answers, enabling it to efficiently answer complex medical problems and provide strong support for clinical decision-making. BRIEF DESCRIPTION OF THE DRAWINGS

[0072] Figure 1 A flowchart showing a medical question-answering method based on knowledge enhancement and reasoning optimization proposed in an embodiment of the present invention;

[0073] Figure 2 Another flowchart of a medical question-answering method based on knowledge enhancement and reasoning optimization proposed in an embodiment of the present invention is shown;

[0074] Figure 3 A flowchart of the inference optimization process proposed in an embodiment of the present invention is shown;

[0075] Figure 4 The figure shows a structural block diagram of a medical question answering system based on knowledge enhancement and reasoning optimization proposed in an embodiment of the present invention. DETAILED DESCRIPTION

[0076] The accompanying drawings are for illustrative purposes only and are not to be construed as limiting this patent;

[0077] It is understandable to those skilled in the art that some well-known contents may be omitted in the drawings;

[0078] The technical solution of the present invention is further described below with reference to the accompanying drawings and embodiments.

[0079] Example 1

[0080] like Figure 1 and Figure 2 As shown, this embodiment proposes a medical question answering method based on knowledge enhancement and reasoning optimization, including the following steps:

[0081] S1. Obtaining a medical corpus, and preprocessing the medical corpus to obtain a preprocessed medical corpus;

[0082] In S1, the medical corpus includes clinical guidelines, medical literature, and case reports, covering language expressions and professional terms in various medical fields. The medical corpus is preprocessed through data cleaning and formatting operations to generate an input format suitable for language model training. The preprocessing of the medical corpus includes:

[0083] S101. Define the medical corpus as where d i is the i-th document, N is the total number of documents, and the contents in the medical corpus are screened based on medical professionalism;

[0084] S102. The medical corpus is Clean and get the cleaned corpus The calculation expression is as follows:

[0085]

[0086] Among them, valid(d) is a data validation function that meets the requirements of format integrity and professionalism;

[0087] During the corpus cleaning process, irrelevant non-medical content, redundant characters, and incomplete documents are removed, and the spelling and format of medical terms are standardized;

[0088] S103. Use Byte Pair Encoding (BPE) algorithm to encode the cleaned corpus Perform word segmentation, BPE gradually generates a set of subword units by merging high-frequency word pairs Generate a set of subword units The calculation expression is as follows:

[0089]

[0090] Among them, BPE(.) is the encoding function; the subword unit set is generated Used to represent medical-related professional terms and high-frequency words;

[0091] S104. Group the subword units The L-th subword unit x in L Split into fixed-length input sequences X i as follows:

[0092] X i =[x1,x2,…,xL ]

[0093] Where L is the sequence length;

[0094] S105. Input sequence X i As a preprocessed medical corpus; each document is segmented into a sequence to generate a set of subword sequences, forming a structured data format suitable for the preset language model input.

[0095] S2. Using the preprocessed medical corpus to train a preset language model, obtaining an initial medical question-answering model for outputting semantic vectors;

[0096] S3. Generate a knowledge embedding vector using a preset knowledge graph, and fuse the knowledge embedding vector with the semantic vector to obtain a fused vector;

[0097] S4. Using the fusion vector as input to the initial medical question answering model, fine-tuning the initial medical question answering model to obtain a fine-tuned model;

[0098] S5. Performing inference optimization on the fine-tuned model to obtain a trained medical question-answering model;

[0099] S6. Input the medical question to be processed into the medical question-answering model and output the answer prediction result.

[0100] In this embodiment, the existing medical question-answering system lacks effective knowledge integration and deep reasoning capabilities when facing diverse and complex medical problems, resulting in weak accuracy and interpretability of answers, that is, low efficiency, accuracy and reliability of medical question-answering. This embodiment proposes a medical question-answering method based on knowledge enhancement and reasoning optimization. First, the preprocessed medical corpus is used to construct an initial medical question-answering model. By integrating the knowledge embedding vector generated by the knowledge graph and the semantic vector output by the initial medical question-answering model, the initial medical question-answering model's ability to understand and apply medical knowledge is enhanced; then, the initial medical question-answering model is fine-tuned in a supervised manner, and the preference for detailed reasoning process is introduced for reasoning optimization to obtain a medical question-answering model with medical question-answering capabilities; the medical question-answering model is further used to predict answers to the medical questions to be processed. The present invention not only improves the efficiency and accuracy of medical question-answering, but also ensures the reliability of the answers, enabling it to efficiently answer complex medical problems and provide strong support for clinical decision-making. This method constructs a medical question-answering model to address the problems of low efficiency, low accuracy and poor reliability of medical question-answering in existing medical question-answering systems, which lack knowledge support and detailed reasoning capabilities when dealing with complex medical problems. This method thus improves the accuracy, reasoning depth and application breadth of the medical question-answering model in medical question-answering tasks.

[0101] Example 2

[0102] See also Figure 1 and Figure 2 , S2 uses the preprocessed medical corpus to train the preset language model. The input is the medical corpus after cleaning, word segmentation and formatting, which is represented by {X1,X2,…,X N}, where each input sequence X i =[x1,x2,…,x L ] contains L subword units; the language model is based on the Transformer architecture and encodes the input through a multi-layer attention mechanism; the language model includes an input layer, a Transformer encoding layer and an output layer, the input layer is connected to the input end of the Transformer encoding layer, the output end of the Transformer encoding layer is connected to the output layer, and the input layer converts the input sequence X i The embedded matrix is ​​input into the Transformer encoding layer, which includes multiple layers of Transformer encoders. Each layer of Transformer encoder uses a multi-head self-attention mechanism and a feedforward neural network to encode the embedded matrix to obtain an encoded output result.

[0103] Each layer of the Transformer encoder encodes the embedding matrix using a multi-head self-attention mechanism and a feedforward neural network. The input of the language model is represented as an embedding matrix:

[0104] E i =Embed(X i )

[0105] Where Embed(·) is an embedding function that maps the input sequence to an embedding space of fixed dimension and obtains the encoded output results, including:

[0106] S201. The first layer Transformer encoder is added to the output of the previous layer Transformer encoder through the multi-head self-attention mechanism and then normalized to obtain the initial output result H (l) The calculation expression is as follows:

[0107] H (l) =LayerNorm(H (l-1) +Attention(Q (l) ,K (l) ,V (l) ))

[0108] Among them, H (l-1)is the output of the l-1th layer Transformer encoder, LayerNorm(.) is the layer normalization operation, Attention is the multi-head self-attention mechanism, Q (l) is the query matrix of the l-th layer Transformer encoder, K (l) is the key matrix of the l-th layer Transformer encoder, V (l) is the value matrix of the l-th layer Transformer encoder;

[0109] The calculation formula of the multi-head self-attention mechanism Attention is as follows

[0110]

[0111] d k is the dimension of the attention head;

[0112] S202. The initial output result H (l) Input the feedforward neural network in the lth layer Transformer encoder to obtain the output of the current layer feedforward neural network, and compare the output of the current layer feedforward neural network with the initial output result H (l) Add them together and go through layer normalization to get the encoding output H′ of the lth layer Transformer encoder (l) as follows:

[0113] H′ (l) =LayerNorm(H (l) +FFN(H (l) ))

[0114] Among them, FFN(.) is a feedforward neural network, H (l) is the initial output result of the l-th layer Transformer encoder;

[0115] The calculation expression of FFN(.) is as follows:

[0116] FFN(H)=max(0,HW1+b1)W2+b2

[0117] Among them, W1, W2 are the weights of the feedforward neural network, b1, b2 are the biases of the feedforward neural network;

[0118] Using the first training total loss function The language model is trained, and the pre-training tasks include the Masked Language Modeling (MLM) task and the Next Sentence Prediction (NSP) task. For the MLM task, a portion of the word positions in the input sequence are randomly masked, and the mask set is recorded as M. The loss function of the masked language model task is Defined as:

[0119]

[0120] Among them, X i represents the input sequence after removing the mask position;

[0121] For the NSP task, sentence pairs are randomly generated, and the language model needs to predict whether the two sentences are continuous contexts. The goal is to minimize the loss function of the training next sentence prediction task. The calculation expression is as follows:

[0122]

[0123] Among them, X j is the sentence pair input, y j represents the label, N is the total number of samples, P(y j ∣∣X j ) is a given input X j , the language model predicts y j The probability distribution function of

[0124] The calculation expression of the first training total loss function is:

[0125]

[0126] Among them, α is a hyperparameter used to control the weight of the masked language model task, and β is a hyperparameter used to control the weight of the next sentence prediction task; is the loss function for training the masked language model task, The loss function for training the next sentence prediction task;

[0127] Finally, we obtain the initial model for medical question answering that outputs semantic vectors, which outputs the context representation vector of each input sequence, expressed as:

[0128]

[0129] Among them, h i is the context feature vector of the i-th subword in the sequence; is the sequence length;

[0130] Through the above steps, the training of the preset language model is completed to form the initial medical question-answering model.

[0131] Example 3

[0132] The knowledge graph described in S3 is represented as a directed graph in is the node set containing medical entities; ε is the edge set used to describe the relationship between medical entities, and each edge is represented by e=(v i ,v j ,r), where r is the relationship type; the initial representation of the node is generated by the embedding function, defined as:

[0133]

[0134] in, For node v i The initial embedding vector of the node is Embed(·), and Embed(·) is the node embedding function. The knowledge embedding vector is generated by using the preset knowledge graph, and the embedding is updated by a multi-layer graph neural network (GNN), including:

[0135] S301. Based on the nodes in the knowledge graph, a multi-layer graph neural network is used to calculate the node embedding vectors in the knowledge graph as follows:

[0136]

[0137] in, is the node embedding calculated by the l-th layer graph neural network, σ is the activation function, is a set of relationship types, For node v i The neighbor set of is the attention parameter, W r is the projection matrix of relation type r, is the node embedding calculated by the l-1th layer of the graph neural network;

[0138] During the embedding update process, the graph attention mechanism is used to model the relationship between the node and its neighbor nodes, and the edge weight is calculated by the following formula:

[0139]

[0140] Among them, W r is the projection matrix of relation r, a is the attention parameter, ‖ represents the vector concatenation operation; the update formula of node embedding is:

[0141]

[0142] S302. Based on node embedding Calculate the final node embedding vector h i The expression is as follows:

[0143]

[0144] Where L is the total number of layers of the graph neural network;

[0145] S303. Based on the node embedding vector h i , calculate the knowledge embedding vector H G The expression is as follows:

[0146] H G =[h1,h2,…,h n ]

[0147] Where n is the total number of nodes in the knowledge graph.

[0148] In step S3, the fusion of the knowledge embedding vector and the semantic vector is achieved by the following formula:

[0149] H fusion =Concat(H L ,H G )W f

[0150] Among them, H L represents the semantic vector generated by the initial model of medical question answering, Concat(·) represents the vector concatenation operation, and W f is the fusion mapping matrix.

[0151] S4 first uses the labeled medical question answering dataset to fine-tune the initial medical question answering model. The labeled medical question answering dataset is defined as:

[0152]

[0153] Among them, Q i is the input sequence of the i-th medical question, A i is the corresponding true answer, N is the total number of samples;

[0154] Then the fine-tuning model input is integrated with the semantic representation vector H output by the initial medical question answering model L and the knowledge graph embedding matrix H G , embed the knowledge into vector H L Fusion with the semantic vector H G , get the fusion vector Z fusion The calculation expression is as follows:

[0155] Z fusion=Concat(H L ,H G )W z

[0156] Among them, H L is the knowledge embedding vector, H G is the semantic vector, Concat(·) is the vector concatenation operation, W f is the fusion mapping matrix in the fine-tuning training phase.

[0157] Using the second training total loss function Fine-tune the initial medical question answering model, and the second training total loss function The calculation expression is as follows:

[0158]

[0159] in, The optimization objective loss function generated for the answer, is the semantic loss function;

[0160] The answer generation process is based on a sequence-to-sequence (Seq2Seq) model, and the generated answer sequence is defined as:

[0161]

[0162] Where A=[a1,a2,…,a T ] represents the generated answer, a t For the tth generated word, a ＜t is the historical generation sequence, θ is the model parameter; the optimization goal of answer generation is achieved by maximizing the log-likelihood, and the optimization goal loss function The calculation expression is as follows:

[0163]

[0164] Among them, P(.) is the probability distribution function, T is the total number of generated true answer words, is the tth generated true answer word in the i-th sample, Generate a sequence for the history in the i-th sample, Q i is the input sequence of the i-th medical question, θ is the model parameter;

[0165] To improve the semantic consistency of the answers generated by the model, the semantic loss function Add a semantic matching regularization term based on vector similarity, the semantic loss function The calculation expression is as follows:

[0166]

[0167] Among them, Embed(·) is the answer semantic embedding function, N is the total number of samples, A i is the true answer to the question, The answer generated for the fine-tuned model, ||.|| represents the norm of the vector.

[0168] The fine-tuning model is optimized for inference, see Figure 3 ,include:

[0169] S501. Initialize the reasoning path using the fusion vector and medical question after fine-tuning the fine-tuning model;

[0170] In S501, the fine-tuning model input is composed of the fine-tuned fusion representation vector Z fusion and the medical question Q, which is used to initialize the reasoning path;

[0171] S502. Decompose the medical problem into multiple sub-problems under the reasoning path, and generate an answer corresponding to each sub-problem;

[0172] In S502, the reasoning optimization decomposes the complex medical problem into a sequence of multiple sub-problems through a step-by-step reasoning strategy as follows:

[0173] Q={q1,q2,…,q K}

[0174] Among them, q k is the kth subproblem, K is the subproblem q k The total number of subproblems, the generation of subproblems is controlled by the current problem state and the fusion representation;

[0175] The state transition generated by the subproblem is defined by the following formula:

[0176] s k+1 =RNN(s k ,z k )

[0177] Among them, s k Indicates the current reasoning state, z k is the current input representation, RNN is a recursive neural network module used to model the sequence relationship of the reasoning path;

[0178] The answer to each sub-question is generated through a Seq2Seq model, defined as:

[0179]

[0180] Among them, a k =[a k,1 ,a k,2 ,…,ak,T ] is the answer sequence of the sub-questions, a k,t is the tth generated word;

[0181] S503. Calculate the total path loss function of the reasoning path based on the answer corresponding to each sub-question and the weight adjustment item of the reasoning path;

[0182] In S503, the total path loss function combines the answers to each sub-question to generate the loss and path weight adjustment term, which is defined as:

[0183]

[0184] in, Generate the loss for the answer to the k-th sub-question, where μ is the regularization coefficient of the path weight adjustment term;

[0185] S504. Optimize the path loss function to obtain a medical question-answering model that outputs a final answer;

[0186] In S504, the path loss function is optimized mainly by optimizing the path weight parameters, and the optimization of the path weight parameters is completed by the policy gradient algorithm to maximize the cumulative reward of the entire reasoning path:

[0187]

[0188] Among them, R k is the immediate reward at step k, π θ Represents the current reasoning strategy, reward function R k is defined as:

[0189]

[0190] in, is the similarity between the generated answer and the true answer, α is the path smoothing weight;

[0191] The final answer is the integration of the answers to all sub-questions:

[0192] A final =Combine({a1,a2,…,a K})

[0193] Combine(·) is the answer integration function, which combines the intermediate reasoning results of each sub-question to enable the trained medical question-answering model to generate a complete answer.

[0194] Example 4

[0195] This embodiment proposes a medical question-answering system based on knowledge enhancement and reasoning optimization, including:

[0196] A preprocessing module is used to obtain a medical corpus and preprocess the medical corpus to obtain a preprocessed medical corpus;

[0197] A model training module is used to train a preset language model using the preprocessed medical corpus to obtain an initial medical question-answering model for outputting semantic vectors;

[0198] A knowledge embedding module is used to generate a knowledge embedding vector using a preset knowledge graph, and fuse the knowledge embedding vector with the semantic vector to obtain a fusion vector;

[0199] a fine-tuning module, configured to use the fusion vector as input to the initial medical question-answering model, and fine-tune the initial medical question-answering model to obtain a fine-tuned model;

[0200] An inference optimization module, configured to perform inference optimization on the fine-tuning model to obtain a trained medical question-answering model;

[0201] The prediction module is used to input the medical question to be processed into the medical question-answering model and output the answer prediction result.

[0202] In this embodiment, in response to the fact that existing medical question-answering systems lack effective knowledge integration and deep reasoning capabilities when facing diverse and complex medical questions, resulting in weak accuracy and interpretability of answers, that is, low efficiency, accuracy, and reliability of medical question-answering, this embodiment proposes a medical question-answering system based on knowledge enhancement and reasoning optimization. First, a medical question-answering initial model is constructed using a preprocessed medical corpus. By fusing the knowledge embedding vectors generated by the knowledge graph and the semantic vectors output by the medical question-answering initial model, the medical question-answering initial model's ability to understand and apply medical knowledge is enhanced. Then, by performing supervised fine-tuning on the medical question-answering initial model and introducing a preference for detailed reasoning process for reasoning optimization, a medical question-answering model with medical question-answering capabilities is obtained. The medical question-answering model is further used to predict answers to the medical questions to be processed. The present invention not only improves the efficiency and accuracy of medical question-answering, but also ensures the reliability of the answers, enabling it to efficiently answer complex medical questions and provide strong support for clinical decision-making. This system enhances the understanding and reasoning capabilities of medical professional questions, significantly improves the accuracy and reliability of answers, and realizes autonomous processing and efficient answering of complex medical questions.

[0203] Obviously, the above embodiments of the present invention are merely examples for the purpose of clearly illustrating the present invention, and are not intended to limit the embodiments of the present invention. A person skilled in the art would be able to make other variations or modifications based on the above description. It is not necessary and impossible to enumerate all embodiments here. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention shall be included within the scope of protection of the claims of the present invention.

Claims

1. A medical question answering method based on knowledge enhancement and reasoning optimization, characterized in that: The following steps are involved: S1. Obtaining a medical corpus, and preprocessing the medical corpus to obtain a preprocessed medical corpus; S2. Using the preprocessed medical corpus to train a preset language model, obtaining an initial medical question-answering model for outputting semantic vectors; S3. Generate a knowledge embedding vector using a preset knowledge graph, and fuse the knowledge embedding vector with the semantic vector to obtain a fused vector; S4. Using the fusion vector as input to the initial medical question answering model, fine-tuning the initial medical question answering model to obtain a fine-tuned model; S5. Performing inference optimization on the fine-tuned model to obtain a trained medical question-answering model; S6. Input the medical question to be processed into the medical question-answering model and output the answer prediction result; The language model includes an input layer, a Transformer encoding layer and an output layer. The input layer is connected to the input end of the Transformer encoding layer, and the output end of the Transformer encoding layer is connected to the output layer. The input layer converts the input sequence into The embedding matrix is ​​input to the Transformer encoding layer, which includes multiple layers of Transformer encoders. Each layer of Transformer encoders encodes the embedding matrix using a multi-head self-attention mechanism and a feedforward neural network to obtain an encoded output result. Using the first training total loss function The language model is trained, and the calculation expression of the first training total loss function is: in, is a hyperparameter used to control the weight of the masked language model task, is a hyperparameter used to control the weight of the next sentence prediction task; is the loss function for training the masked language model task, The loss function for training the next sentence prediction task; Using the second training total loss function Fine-tune the initial medical question answering model, and the second training total loss function The calculation expression is as follows: in, The optimization objective loss function generated for the answer, is the semantic loss function; The optimization objective loss function The calculation expression is as follows: in, (.) is the probability distribution function, is the total number of true answer words generated, For the In the sample The generated true answer words, For the The historical generation sequence in samples, For the An input sequence of medical questions, are model parameters; The semantic loss function The calculation expression is as follows: in, is the answer semantic embedding function, is the total number of samples, is the true answer to the question, The answers generated for the fine-tuned model, . Represents the norm of a vector.

2. The medical question answering method based on knowledge enhancement and reasoning optimization according to claim 1 is characterized in that: The preprocessing of the medical corpus comprises: S101. Define the medical corpus as ,in For the documents, is the total number of documents; S102. The medical corpus is Clean and get the cleaned corpus The calculation expression is as follows: in, It is the data validation function; S103. Cleaned corpus Perform word segmentation to generate a set of subword units The calculation expression is as follows: in, is the encoding function; S104. Group the subword units The Lth subword unit in Split the input into fixed-length sequences as follows: in, is the sequence length; S105. Input the sequence As a preprocessed medical corpus.

3. The medical question answering method based on knowledge enhancement and reasoning optimization according to claim 1 is characterized in that: Each layer of the Transformer encoder encodes the embedding matrix using a multi-head self-attention mechanism and a feedforward neural network to obtain an encoded output result, including: S201. No. The Transformer encoder of the first layer is added to the output of the Transformer encoder of the previous layer through the multi-head self-attention mechanism, and then normalized to obtain the initial output result. The calculation expression is as follows: in, For the -1 layer Transformer encoder output, is the layer normalization operation, It is a multi-head self-attention mechanism. For the The query matrix of the layer Transformer encoder, For the The key matrix of the layer Transformer encoder, For the The value matrix of the layer Transformer encoder; S202. The initial output result Enter The feedforward neural network in the layer Transformer encoder obtains the output of the current layer feedforward neural network and compares the output of the current layer feedforward neural network with the initial output result. Add them together and go through layer normalization to get The encoded output of the layer Transformer encoder as follows: in, is a feedforward neural network, For the The initial output of the layer Transformer encoder.

4. The medical question answering method based on knowledge enhancement and reasoning optimization according to claim 1, characterized in that: The knowledge graph is represented as a directed graph ,in is a set of nodes containing medical entities; is a set of edges used to describe the relationship between medical entities, each edge is represented by ,in , is the relationship type; The method of generating a knowledge embedding vector using a preset knowledge graph includes: S301. Based on the nodes in the knowledge graph, a multi-layer graph neural network is used to calculate the node embedding vectors in the knowledge graph as follows: in, For the first l Node embeddings computed by layer graph neural networks, is the activation function, is a set of relationship types, For nodes The neighbor set of is the attention parameter, For relationship type The projection matrix, For the first l -1 layer of graph neural network to calculate node embedding; S302. Based on node embedding , calculate the final node embedding vector The expression is as follows: in, is the total number of layers of the graph neural network; S303. Based on the node embedding vector , calculate the knowledge embedding vector The expression is as follows: Where n is the total number of nodes in the knowledge graph.

5. The medical question answering method based on knowledge enhancement and reasoning optimization according to claim 1, characterized in that: The knowledge embedding vector is fused with the semantic vector to obtain a fusion vector The calculation expression is as follows: in, is the knowledge embedding vector, is the semantic vector, is the vector concatenation operation, is the fusion mapping matrix in the fine-tuning training phase.

6. The medical question answering method based on knowledge enhancement and reasoning optimization according to claim 1, characterized in that: The performing inference optimization on the fine-tuning model includes: S501. Initialize the reasoning path using the fusion vector and medical question after fine-tuning the fine-tuning model; S502. Decompose the medical problem into multiple sub-problems under the reasoning path, and generate an answer corresponding to each sub-problem; S503. Calculate the total path loss function of the reasoning path based on the answer corresponding to each sub-question and the weight adjustment item of the reasoning path; S504. Perform path optimization on the path loss function to obtain a medical question-answering model that outputs a final answer.

7. A medical question-answering system based on knowledge enhancement and reasoning optimization, characterized in that: include: A preprocessing module is used to obtain a medical corpus and preprocess the medical corpus to obtain a preprocessed medical corpus; A model training module is used to train a preset language model using the preprocessed medical corpus to obtain an initial medical question-answering model for outputting semantic vectors; A knowledge embedding module is used to generate a knowledge embedding vector using a preset knowledge graph, and fuse the knowledge embedding vector with the semantic vector to obtain a fusion vector; a fine-tuning module, configured to use the fusion vector as input to the initial medical question-answering model, and fine-tune the initial medical question-answering model to obtain a fine-tuned model; An inference optimization module, configured to perform inference optimization on the fine-tuning model to obtain a trained medical question-answering model; A prediction module, configured to input the medical question to be processed into the medical question-answering model and output an answer prediction result; The language model includes an input layer, a Transformer encoding layer and an output layer. The input layer is connected to the input end of the Transformer encoding layer, and the output end of the Transformer encoding layer is connected to the output layer. The input layer converts the input sequence into The embedding matrix is ​​input to the Transformer encoding layer, which includes multiple layers of Transformer encoders. Each layer of Transformer encoders encodes the embedding matrix using a multi-head self-attention mechanism and a feedforward neural network to obtain an encoded output result. Using the first training total loss function The language model is trained, and the calculation expression of the first training total loss function is: in, is a hyperparameter used to control the weight of the masked language model task, is a hyperparameter used to control the weight of the next sentence prediction task; is the loss function for training the masked language model task, The loss function for training the next sentence prediction task; Using the second training total loss function Fine-tune the initial medical question answering model, and the second training total loss function The calculation expression is as follows: in, The optimization objective loss function generated for the answer, is the semantic loss function; The optimization objective loss function The calculation expression is as follows: in, (.) is the probability distribution function, is the total number of true answer words generated, For the In the sample The generated true answer words, For the The historical generation sequence in samples, For the An input sequence of medical questions, are model parameters; The semantic loss function The calculation expression is as follows: in, is the answer semantic embedding function, is the total number of samples, is the true answer to the question, The answers generated for the fine-tuned model, . Represents the norm of a vector.

Images