Metaquestion-answer-pair-based knowledge enhancement method for large language model in medical field

Abstract

The invention relates to the technical field of artificial intelligence, in particular to a medical field large language model knowledge enhancement method based on meta-question-answer pairs. Preprocessing the data; generating meta-question-answer pairs; performing adaptive fine tuning on the meta-question-answer pairs; constructing and updating a dynamic knowledge base, wherein the dynamic knowledge base comprises a medical knowledge base which is constructed after encoding the generated domain knowledge meta-question-answer pairs; according to the method, an enhanced fine tuning data set is formed on the basis of a large amount of generated high-quality meta-question-answer pair data, so that a large model can learn field core knowledge more deeply, a high-quality medical field knowledge base is constructed, meanwhile, the field knowledge base supplements additional information through a retrieval enhanced generation method, and the medical field knowledge base is improved. The generation effect is further optimized, and it is ensured that the model has the understanding ability of core domain knowledge and can dynamically adapt to newest knowledge through retrieval.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and specifically to a method for enhancing knowledge of a large language model in the medical field based on meta-question-answer pairs. Background Art

[0002] Large-scale language models are profoundly changing the model and efficiency of medical services, with their impact permeating every level of the healthcare system. In clinical decision support, large-scale language models can rapidly analyze massive amounts of medical data, providing doctors with precise diagnostic recommendations and treatment plans, thereby improving diagnostic accuracy and treatment effectiveness. In medical education, large-scale language models help medical students and professionals more efficiently master complex knowledge and improve their professional skills by providing personalized learning resources and simulated clinical scenarios. In health management, large-scale language models integrate patients' health data to provide personalized health advice and risk warnings, helping patients better manage themselves and prevent disease.

[0003] Large language models have shown great potential in the medical field. Retrieval-augmented generation methods enable the dynamic updating of medical knowledge, while model fine-tuning strategies can optimize generation quality.

[0004] However, due to the frequent knowledge updates in the medical field, extremely high professional requirements and complex decision-making chains, existing methods separate the optimization of retrieval enhancement and model fine-tuning, failing to coordinate the dual goals of knowledge updating and generation accuracy improvement. As a result, the accuracy dilemma of retrieval enhancement generation methods and the efficiency bottleneck of fine-tuning strategies are simultaneously amplified, resulting in limited performance of large language models and inability to better adapt to the medical field. Therefore, to address the above problems, a knowledge enhancement method for large language models in the medical field based on meta-question-answer pairs is proposed. Summary of the Invention

[0005] The purpose of the present invention is to provide a knowledge enhancement method for large language models in the medical field based on meta-question-answer pairs, so as to solve the problem that due to the frequent updates of knowledge in the medical field, extremely high professional requirements and complex decision-making chains, the existing methods separate the optimization of retrieval enhancement and model fine-tuning, and fail to coordinate the dual goals of knowledge updating and generation accuracy improvement, resulting in the simultaneous amplification of the two defects of the accuracy dilemma of the retrieval enhancement generation method and the efficiency bottleneck of the fine-tuning strategy, resulting in limited performance of large language models and inability to better adapt to the medical field.

[0006] To achieve the above object, the present invention provides the following technical solutions:

[0007] A method for knowledge enhancement of large language models in the medical field based on meta-question-answer pairs, comprising:

[0008] Dataset construction: The dataset includes a medical guide dataset, a real patient consultation dataset, and a general conversation dataset;

[0009] Data preprocessing: Preprocess the data in the dataset to extract and organize medical related information;

[0010] Generation of meta-question-answer pairs: The question-answering model uses the Qwen2.5-72B general language model to generate corresponding answers based on the content of medical textbooks and medical documents;

[0011] Adaptive fine-tuning of meta-question-answer pairs: This fine-tuning technique uses the Qwen2-7B universal language model as a base model and performs adaptive fine-tuning on the model through a small amount of parameter adjustments;

[0012] Dynamic knowledge base construction and update: The dynamic knowledge base includes a medical knowledge base constructed by encoding the generated domain knowledge meta-question-answer pairs;

[0013] Performance testing and comparative experiments: The performance testing includes test design and test process, and different evaluation indicators are used in the comparative experiments.

[0014] As further optimized content of the present invention, the medical guide data set includes diagnostic criteria, detection methods, treatment plans and patient education for diseases, and the real patient consultation data set includes clinical data, clinical notes and consultation dialogue data.

[0015] As a further optimized content of the present invention, wherein: using OCR technology to perform text recognition on the PDF file of the medical document to extract the text content in the document;

[0016] Use OCR technology to convert unstructured image data into actionable text data;

[0017] A segmentation strategy is used to segment the text data after OCR recognition in the dataset into paragraphs.

[0018] As a further optimization of the present invention, the text content after OCR recognition is preliminarily divided into paragraphs based on 500 characters;

[0019] Verify the integrity of each segmented paragraph;

[0020] Adjust the boundaries of paragraphs that fail integrity verification;

[0021] Output the paragraphs that pass the integrity verification and the breaks after adjusting the boundaries as independent data units.

[0022] As further optimized content of the present invention, among others: medical knowledge structured input: pre-processed data is input into the Qwen2.5-72B model through block embedding technology;

[0023] Based on the extracted medical knowledge points, question-answer pairs related to the medical field are generated: the model automatically identifies core medical entities in the text and generates core conclusions, diagnostic basis, and clinical considerations based on pathological concepts, diagnostic and treatment operations, and clinical reasoning.

[0024] As a further optimization of the present invention, a low-rank matrix is ​​introduced to achieve efficient parameter fine-tuning: the calculation formula is:

[0025] W=W+ΔW=W+BC T

[0026] Where W represents the original weight matrix of the pre-trained model, ΔW represents the adaptive weight matrix, and B and C represent low-rank matrices whose rank dimensions are much smaller than the original matrix W.

[0027] As a further optimization of the present invention, a knowledge base is constructed using question-answer pairs in the medical field, Q is set as the search tag, A is set as the search content, and all search tags Q are integrated into a set D;

[0028] Given a user query q, find the most relevant Q in the set D using the relevance calculation formula. The formula is as follows:

[0029]

[0030] Where: sim(q, Q) represents the similarity between user query q and Q, Q′ is the generalized representation of Q in set D excluding those similar to q;

[0031] Find the most relevant Q to the user query q through the relevance calculation formula and return the corresponding search content A;

[0032] Based on the user query q and the retrieved content A, a final answer is generated, and the conditional probability of generating the answer in time period t is calculated. The calculation formula is as follows:

[0033]

[0034] Where, time period t∈T, w<t represents all words generated before time period t;

[0035] Set a similarity threshold τ, the value range of the threshold τ is 0.7≤τ≤0.9;

[0036] Let Q and A correspond to (Q i ,A i ) is an existing question-answer pair, (Q new , Anew ) is a newly generated question-answer pair, and the similarity function is introduced to evaluate Q new With Q i The similarity function formula is as follows:

[0037]

[0038] Where: v new and v i Represents Q new With Q i vectorization;

[0039] Update existing question-answer pairs: When sim(Q new ,Q i )≥τ, the corresponding question-answer pair in D will be updated to (Q new , A new );

[0040] New question and answer pair: When sim(Q new ,Q i )<τ, (Q new , A new ) is added to D.

[0041] As further optimized content of the present invention, among others: adopting a medical test question examination model: compiling medical test questions based on the examination questions of the medical attending physician qualification examination;

[0042] Model testing: Each model is tested independently, using different test questions for each test;

[0043] Selection of evaluation indicators: The average accuracy and standard deviation are calculated by averaging the accuracy of several tests.

[0044] As a further optimization of the present invention, a large language model in the medical field based on meta-question-answer pairs includes the following modules:

[0045] Hybrid Data Integration Module: This module integrates medical guideline datasets, real patient consultation datasets, and general conversation datasets. The medical guideline datasets cover disease diagnostic criteria, treatment plans, and patient education content, while the real patient consultation datasets include clinical conversations and notes.

[0046] Multimodal preprocessing module: Based on OCR technology, it extracts text and segments paragraphs from unstructured medical documents, and uses character unit division and integrity verification strategies to generate independent data units;

[0047] Meta-Q&A generation engine: Based on the Qwen2.5-72B model, it implements structured input of medical knowledge and automatically generates core Q&A pairs that include diagnostic evidence and clinical reasoning.

[0048] Efficient parameter fine-tuning module: Using Qwen2-7B as the base model, it dynamically adjusts weight parameters through low-rank matrix decomposition to achieve domain-adaptive training;

[0049] Dynamic knowledge base management unit: This unit stores meta-question-answer pairs in vector encoding and supports a retrieval content update mechanism based on similarity thresholds, including the replacement and addition of question-answer pairs.

[0050] Multi-dimensional evaluation system: Built-in medical attending physician examination question bank, calculates the average accuracy and standard deviation of the model through independent test questions, and combines the performance comparison with the diagnosis and treatment logic consistency index.

[0051] As further optimized content of the present invention, there is provided: a large language model in the medical field based on meta-question-answer pairs, comprising a storage device, a processing and computing device, and a computing program stored in the storage device and executable on the processing and computing device.

[0052] The present invention has the following beneficial effects:

[0053] Compared with the existing technology, the present invention forms an enhanced fine-tuning dataset based on a large amount of high-quality meta-question-answer pair data, so that the large model can learn the core knowledge of the domain more deeply, thereby building a high-quality medical domain knowledge base. At the same time, the domain knowledge base is supplemented with additional information through the retrieval enhancement generation method to further optimize the generation effect, ensuring that the model not only has the ability to understand the core domain knowledge, but also can dynamically adapt to the latest knowledge through retrieval. Then, by matching user queries with questions in the knowledge base, the most relevant questions to the query are identified to obtain more accurate retrieval results, thereby generating more accurate and professional answers. This method overcomes the respective limitations of the fine-tuning strategy and the retrieval enhancement generation method, and enables the two to be synergistically optimized and improved. BRIEF DESCRIPTION OF THE DRAWINGS

[0054] Figure 1 This is a flow chart of a method for knowledge enhancement of a large language model in the medical field based on meta-question-answer pairs according to the present invention;

[0055] Figure 2 This is a structural diagram of a large language model in the medical field based on meta-question and answer pairs in the present invention. DETAILED DESCRIPTION

[0056] The present invention will be further described below in conjunction with specific embodiments. The accompanying drawings are for illustrative purposes only and represent only schematic diagrams rather than actual drawings. They should not be understood as limiting the present invention. In order to better illustrate the specific embodiments of the present invention, some parts of the drawings may be omitted, enlarged or reduced, and do not represent the size of the actual product.

[0057] The specific implementation process of the technical solution of the present invention includes the following steps:

[0058] Step 1: Dataset Construction: The dataset includes a medical guideline dataset, a real patient consultation dataset, and a general dialogue dataset. The medical guideline dataset includes disease diagnostic criteria, detection methods, treatment plans, and patient education. The selected guidelines are all the latest versions published in recent years, reflecting the latest research progress in the medical field; the real patient consultation dataset includes clinical data, clinical notes, and consultation dialogue data, covering multiple medical fields such as medicine and general medicine; the general dialogue dataset covers a large number of high-quality Chinese question-answer pairs, which can be easily understood by people outside the medical field.

[0059] Step 2: Data preprocessing: Preprocess the data in the dataset to extract and organize medical related information;

[0060] Step 21: Use OCR technology to perform text recognition on the PDF file of the medical document and extract the text content in the document;

[0061] Step 22: Use OCR technology to convert unstructured image data into actionable text data;

[0062] Step 23: Use the segmentation strategy to segment the text data after OCR recognition in the dataset into paragraphs.

[0063] Step 231: performing preliminary paragraph division on the text content recognized by OCR based on 500 characters;

[0064] Step 232: Verify the integrity of each segmented paragraph;

[0065] Step 233: Adjusting the boundaries of the paragraphs that fail the integrity verification;

[0066] Step 234: Output the paragraphs that have passed the integrity verification and the fractures after adjusting the boundaries as independent data units;

[0067] After data preprocessing, medical information can be efficiently extracted and organized, laying a good data foundation for subsequent knowledge retrieval and model training.

[0068] Step 3: Generation of meta-question-answer pairs: The question-answering model uses the Qwen2.5-72B general language model to generate corresponding answers based on the content of medical textbooks and medical documents;

[0069] Step 31: Structured input of medical knowledge: Input the pre-processed data into the Qwen2.5-72B model using block embedding technology;

[0070] Step 32: Generate medical-related question-answer pairs based on the extracted medical knowledge points: The model automatically identifies core medical entities in the text (such as disease names, test indicators, and drug mechanisms) and generates core conclusions, diagnostic evidence, and clinical considerations based on pathological concepts, diagnostic procedures, and clinical reasoning.

[0071] This can first generate high-quality questions and then generate corresponding answers, providing basic data support for the model's knowledge learning and dynamic retrieval.

[0072] Step 4: Adaptive fine-tuning of the meta-question-answer pair: The fine-tuning technique uses the Qwen2-7B universal language model as the base model and performs adaptive fine-tuning on the model through a small amount of parameter adjustments;

[0073] Introduce low-rank matrix to achieve efficient parameter fine-tuning: the calculation formula is:

[0074] W=W+ΔW=W+BC T

[0075] Where W represents the original weight matrix of the pre-trained model, ΔW represents the adaptive weight matrix, and B and C represent low-rank matrices whose rank dimensions are much smaller than the original matrix W.

[0076] This enables it to better internalize core domain knowledge, thereby improving the model's understanding and generation capabilities in specific medical fields.

[0077] Step 5: Dynamic knowledge base construction and update: The dynamic knowledge base includes a medical knowledge base constructed by encoding the generated domain knowledge meta-question-answer pairs;

[0078] Step 51: Use question-answer pairs in the medical field to build a knowledge base, set Q as the search tag, A as the search content, and integrate all search tags Q into a set D;

[0079] Step 52: Given a user query q, find the most relevant Q in the set D using the correlation calculation formula. The formula is as follows:

[0080]

[0081] Where: sim(q, Q) represents the similarity between user query q and Q, Q′ is the generalized representation of Q in set D excluding those similar to q;

[0082] Step 53: Find the most relevant Q to the user query q through the correlation calculation formula, and return the corresponding search content A;

[0083] Step 54: Generate a final answer based on the user query q and the search content A, and calculate the conditional probability of generating the answer in time period t. The calculation formula is as follows:

[0084]

[0085] Where, time period t∈T, w ＜t represents all words generated before time period t;

[0086] Step 55: Set a similarity threshold τ, where the value range of the threshold τ is 0.7≤τ≤0.9;

[0087] Step 56: Let Q be the corresponding (Q i ,A i ) is an existing question-answer pair, (Q new , A new ) is a newly generated question-answer pair, and the similarity function is introduced to evaluate Q new With Q i The similarity function formula is as follows:

[0088]

[0089] Where: v new and v i Represents Q new With Q i vectorization;

[0090] Step 57: Update the existing question-answer pair: When sim(Q new ,Q i )≥τ, the corresponding question-answer pair in D will be updated to (Q new , A new );

[0091] Step 58: Add a new question and answer pair: When sim(Q new ,Q i )<τ, (Q new , A new ) is added to D;

[0092] The fine-tuned question-answer pairs serve as a knowledge supplement for the large model, thereby significantly improving the quality and accuracy of the model's answers.

[0093] Step 6: Performance testing and comparative experiments: The performance testing includes test design and test process, and different evaluation indicators are used in the comparative experiments;

[0094] Step 61: using a medical test question assessment model: compiling medical test questions based on the medical attending physician qualification examination questions;

[0095] Step 62: Model testing: Test each model independently, using different test questions for each test;

[0096] Step 63: Selection of evaluation indicators: Calculate the average accuracy and standard deviation by averaging the accuracy of several tests;

[0097] This study evaluated the performance of multiple large language models in medical diagnosis and treatment tasks, including:

[0098] GPT-4o: As one of the most advanced large-scale language models, GPT-4o has a vast amount of pre-training data and a complex model architecture. It demonstrates excellent performance in a variety of natural language processing tasks and is widely used in scenarios such as conversation, question answering, and text generation.

[0099] Llama 3.1-8B: This model is an iterative evolution of the LLaMa series of models. The LLaMa model uses an efficient Transformer architecture and is pre-trained on a large cross-lingual dataset, resulting in strong multilingual understanding capabilities. Llama 3.1-8B optimizes this model to improve its inference efficiency and generation quality.

[0100] Qwen2-7B: The Qwen2 series models use the Transformer decoder architecture and implement context-aware language understanding and generation through a self-attention mechanism.

[0101] Qwen2.5-7B RAG: This model introduces the RAG mechanism based on Qwen2-7B. By building a vectorized knowledge base containing medical guidelines, Qwen2.5-7B RAG can dynamically retrieve relevant knowledge when generating answers, enhancing the model's performance in the medical diagnosis and treatment field.

[0102] Qwen2-7B: A medical large language model with a knowledge enhancement method based on meta-question-answer pairs and a fine-tuning-retrieval strategy based on domain question-answer pair enhancement.

[0103] Table 1 Evaluation results of different large language models in medical diagnosis and treatment tasks

[0104]

[0105] By adopting a model optimization and dynamic update strategy based on domain core and knowledge meta-question-answer pairs, high-quality answers can be effectively generated. This enhanced knowledge fusion strategy can significantly improve the performance of large models in the diagnosis and consultation process, and provide patients with more accurate answers.

[0106] The principles and implementation methods of the present invention are described in this article using specific examples. The above examples are only used to help understand the method and core ideas of the present invention. The above is only a preferred implementation method of the present invention. It should be pointed out that due to the limitations of textual expression, there are objectively infinite specific structures. For ordinary technicians in this technical field, without departing from the principles of the present invention, they can make several improvements, modifications or changes, and can also combine the above technical features in an appropriate manner; these improvements, modifications, changes or combinations, or the direct application of the inventive concept and technical solution to other occasions without improvement, should be regarded as the scope of protection of the present invention.

Claims

1. A method for knowledge enhancement of large language models in the medical field based on meta-question-answer pairs, characterized by: The following steps are involved: S1: Dataset construction: The dataset includes a medical guide dataset, a real patient consultation dataset, and a general conversation dataset; S2: Data preprocessing: Preprocess the data in the dataset to extract and organize medical related information; S3: Generation of meta-question-answer pairs: The question-answering model uses the Qwen2.5-72B general language model to generate corresponding answers based on the content of medical textbooks and medical documents; S4: Adaptive fine-tuning of meta-question-answer pairs: The fine-tuning technique uses the Qwen2-7B universal language model as the base model and performs adaptive fine-tuning on the model through a small amount of parameter adjustments; S5: Dynamic knowledge base construction and update: The dynamic knowledge base includes a medical knowledge base constructed by encoding the generated domain knowledge meta-question-answer pairs; S6: Performance testing and comparative experiments: The performance testing includes test design and test process, and different evaluation indicators are used in the comparative experiments.

2. The method for enhancing knowledge of a large language model in the medical field based on meta-question-answer pairs according to claim 1, characterized in that: The medical guideline dataset includes disease diagnosis criteria, detection methods, treatment plans, and patient education, and the real patient consultation dataset includes clinical data, clinical notes, and consultation dialogue data.

3. The method for enhancing knowledge of a large language model in the medical field based on meta-question-answer pairs according to claim 1, characterized in that: The step S2 comprises the following steps: S21: Use OCR technology to perform text recognition on PDF files of medical documents and extract the text content in the documents; S22: Use OCR technology to convert unstructured image data into actionable text data; S23: Use the segmentation strategy to segment the text data after OCR recognition in the dataset into paragraphs.

4. The method for enhancing knowledge of a large language model in the medical field based on meta-question-answer pairs according to claim 3, characterized in that: The step S23 includes the following steps: S231: Perform preliminary paragraph division on the text content recognized by OCR based on 500 characters; S232: Verify the integrity of each segmented paragraph; S233: Adjust the boundaries of paragraphs that fail integrity verification; S234: Output the paragraphs that have passed the integrity verification and the breaks after adjusting the boundaries as independent data units.

5. The method for enhancing knowledge of a large language model in the medical field based on meta-question-answer pairs according to claim 1, characterized in that: The step S3 comprises the following steps: S31: Structured input of medical knowledge: pre-processed data is input into the Qwen2.5-72B model through block embedding technology; S32: Generate question-answer pairs related to the medical field based on extracted medical knowledge points: The model automatically identifies core medical entities in the text (such as disease names, test indicators, and drug mechanisms), and generates core conclusions, diagnostic basis, and clinical considerations based on pathological concepts, diagnostic and treatment operations, and clinical reasoning.

6. The method for enhancing knowledge of a large language model in the medical field based on meta-question-answer pairs according to claim 1, characterized in that: Introduce low-rank matrix to achieve efficient parameter fine-tuning: the calculation formula is: W=W+ΔW=W+BC T Where W represents the original weight matrix of the pre-trained model, ΔW represents the adaptive weight matrix, and B and C represent low-rank matrices whose rank dimensions are much smaller than the original matrix W.

7. The method for enhancing knowledge of a large language model in the medical field based on meta-question-answer pairs according to claim 1, characterized in that: The step S5 comprises the following steps: S51: Use question-answer pairs in the medical field to build a knowledge base. Let Q be the search tag and A be the search content. All search tags Q are integrated into a set D. S52: Given a user query q, find the most relevant Q in the set D using the correlation calculation formula. The formula is as follows: Where: sim(q, Q) represents the similarity between user query q and Q, Q′ is the generalized representation of Q in set D excluding those similar to q; S53: Find the most relevant Q to the user query q through the correlation calculation formula, and return the corresponding search content A; S54: Generate a final answer based on the user query q and the search content A, and calculate the conditional probability of generating the answer in time period t. The calculation formula is as follows: Where, time period t∈T, w ＜t represents all words generated before time period t; S55: setting a similarity threshold τ, where the threshold τ ranges from 0.7≤τ≤0.9; S56: Let Q and A correspond to (Q i ,A i ) is an existing question-answer pair, (Q new , A new ) is a newly generated question-answer pair, and the similarity function is introduced to evaluate Q new With Q i The similarity function formula is as follows: Where: v new and v i Represents Q new With Q i vectorization; S57: Update the existing question-answer pair: When sim(Q new ,Q i )≥τ, the corresponding question-answer pair in D will be updated to (Q new , A new ); S58: New question and answer pair: When sim(Q new ,Q i )<τ, (Q new , A new ) is added to D.

8. The method for enhancing knowledge of a large language model in the medical field based on meta-question-answer pairs according to claim 1, characterized in that: The step S6 comprises the following steps: S61: Use the medical test question assessment model: compile medical test questions based on the medical attending physician qualification examination questions; S62: Model testing: Each model is tested independently, using different test questions for each test; S63: Selection of evaluation indicators: Calculate the average accuracy and standard deviation by averaging the accuracy of several tests.

9. A large medical language model based on meta-question-answer pairs according to claims 1-8, characterized in that: Includes the following modules: Hybrid Data Integration Module: This module integrates medical guideline datasets, real patient consultation datasets, and general conversation datasets. The medical guideline datasets cover disease diagnostic criteria, treatment plans, and patient education content, while the real patient consultation datasets include clinical conversations and notes. Multimodal preprocessing module: Based on OCR technology, it extracts text and segments paragraphs from unstructured medical documents, and uses character unit division and integrity verification strategies to generate independent data units; Meta-Q&A generation engine: Based on the Qwen2.5-72B model, it implements structured input of medical knowledge and automatically generates core Q&A pairs that include diagnostic evidence and clinical reasoning. Efficient parameter fine-tuning module: Using Qwen2-7B as the base model, it dynamically adjusts weight parameters through low-rank matrix decomposition to achieve domain-adaptive training; Dynamic knowledge base management unit: This unit stores meta-question-answer pairs in vector encoding and supports a retrieval content update mechanism based on similarity thresholds, including the replacement and addition of question-answer pairs. Multi-dimensional evaluation system: Built-in medical attending physician examination question bank, calculates the average accuracy and standard deviation of the model through independent test questions, and combines the performance comparison with the diagnosis and treatment logic consistency index.

10. The large language model in the medical field based on meta-question-answer pairs according to claim 9, characterized in that: The system comprises a storage device, a processing and computing device, and a computing program stored in the storage device and executable on the processing and computing device, wherein the storage device is used to store the encoded content of a medical guide data set, a real patient consultation data set, a general dialogue data set, and a dynamic knowledge base; the processing and computing device is used to execute the program and run functional modules including a hybrid data integration module, a multimodal preprocessing module, a meta-question-answer pair generation engine, a parameter efficient fine-tuning module, a dynamic knowledge base management unit, and a multi-dimensional evaluation system to implement the steps of any one of the methods of claims 1-8.

Images