Traditional Chinese medicine knowledge question-answering system based on fine-tuning large model and dual retrieval enhancement

Abstract

The invention provides a traditional Chinese medicine knowledge question-answering system based on a fine-tuning large model and dual retrieval enhancement. The traditional Chinese medicine knowledge question-answering system comprises a large model fine-tuning module, a dual retrieval enhancement module, a prompt template module and an answer generation module. The large model fine tuning module constructs a high-quality corpus by using traditional Chinese medicine ancient books, clinical cases and the like, performs incremental pre-training and supervised fine tuning on a ChatGLM3-6B pre-training model, and adopts technologies such as low-rank adaptation (LoRA) and direct preference optimization (DPO) to improve the adaptability of the model to traditional Chinese medicine professional knowledge. The dual retrieval module enables the model to map user questions to related contents in traditional Chinese medicine classical literatures, guidelines and modern literatures through text retrieval based on a LangChain framework on one hand, and provides structured background knowledge through map retrieval on the other hand. The system effectively makes up for the knowledge blind area of a large-scale general model in the field of traditional Chinese medicine, realizes the improvement of the accuracy, continuity and speciality of question and answer results, and has wide application prospects and relatively high innovativeness.

Description

Technical Field

[0001] This invention relates to the fields of artificial intelligence and medical information technology, and in particular to an intelligent question-answering system for Traditional Chinese Medicine (TCM) knowledge. This system utilizes a combination of large-scale pre-trained language model (LLM) fine-tuning technology, retrieval-augmented generation (RAG) methods, and knowledge graph technology to achieve specialized optimization for TCM Q&A. Background Art

[0002] Traditional Chinese Medicine (TCM) boasts a long and rich history, centered on a holistic approach and the principle of syndrome differentiation and treatment. It encompasses a wealth of theoretical, diagnostic, and therapeutic knowledge, as well as prescriptions. However, traditional TCM texts are vast, and many classic theories and prescriptions remain in the oral or handwritten stages. These fragmented and obscure information complicates the systematic organization and digitization of TCM knowledge. TCM emphasizes the principles of "observation, listening, questioning, and palpation" and "preventive treatment," emphasizing syndrome differentiation, a holistic approach, and personalized diagnosis and treatment. Advances in artificial intelligence (AI) have made it possible to integrate TCM diagnostic and treatment expertise with modern intelligent algorithms. However, intelligent question-answering systems that can simultaneously integrate classic TCM theories with the latest research trends are currently lacking. In recent years, scholars have attempted to introduce AI technologies into the field of TCM, digitizing TCM literature through methods such as knowledge graph construction and text mining to support the dissemination and application of TCM knowledge. However, due to the complexity and variability of TCM terminology and the lack of structured annotation, existing TCM intelligent question-answering systems often rely on static knowledge bases or single models, failing to fully leverage the deep structure of TCM knowledge and resulting in frequently missing or inaccurate responses.

[0003] On the other hand, while large-scale pre-trained language models perform well in general question-answering tasks, their direct application to the field of Traditional Chinese Medicine (TCM) faces challenges. For one thing, TCM terminology, classical theories, and clinical experience are not adequately covered in general training corpora. Furthermore, even when trained with existing TCM data, models can still suffer from knowledge gaps and "hallucinations," impacting the accuracy and professionalism of the Q&A process. Retrieval-augmented generation (RAG) has been proposed as a solution. This technology retrieves relevant information from an external knowledge base before generating an answer. The results, along with the user's question, are then fed into a large model for generation, allowing domain knowledge to be incorporated without retraining the model. Furthermore, knowledge graphs, which structure knowledge in an "entity-relationship-entity" format, can organically integrate dispersed TCM entities and relationships, providing an intuitive knowledge context for the model. Existing research has shown that combining text retrieval with knowledge graphs can effectively expand the model's knowledge coverage and improve the reliability and accuracy of question-answering systems. However, in the field of TCM Q&A, a comprehensive solution combining large-scale model fine-tuning with dual retrieval enhancement is still lacking, a technical problem addressed by this present invention. Summary of the Invention

[0004] The present invention discloses a TCM question-answering optimization system based on large-scale model fine-tuning and dual retrieval enhancement to address the problems of insufficient utilization of domain knowledge and low answer accuracy in existing TCM question-answering systems. The system is named "Qijing Question-Answering System". The main purpose of the present invention is to build an intelligent, high-precision TCM question-answering platform by combining incremental pre-training and fine-tuning of TCM domain knowledge, as well as retrieval enhancement generation and knowledge graph retrieval technologies, so that the system can call on professional TCM knowledge in real time in answering, thereby significantly improving the accuracy and professionalism of TCM questions and answers.

[0005] In order to achieve the above objectives, the following technical solutions are adopted:

[0006] A TCM knowledge question-answering system based on fine-tuning a large model and dual retrieval enhancement. It includes a data preprocessing module, a large model fine-tuning module, a dual retrieval enhancement module, a prompt template module, and an answer generation module:

[0007] Large Model Fine-tuning Module: This module leverages data from ancient TCM texts, modern literature, and clinical cases to incrementally pre-train and supervised fine-tune pre-trained language models, enhancing their adaptability to the TCM field. Specifically, the system uses open-source large models such as ChatGLM3-6B as a foundation, and refines the model through self-questioning and self-answering to generate question-answer pairs and entity self-instruction methods. Combined with low-rank adaptation (LoRA) and direct preference optimization (DPO) strategies, this system improves the model's ability to retain TCM knowledge while avoiding the catastrophic forgetting associated with traditional fine-tuning.

[0008] Dual Search Enhancement Module: This module constructs a search enhancement framework, including a text search submodule based on vector retrieval and a graph search submodule based on a knowledge graph. The text search submodule vectorizes and stores TCM literature based on the LangChain framework, and performs semantic search on user questions to retrieve relevant text. The knowledge graph submodule constructs a TCM knowledge graph (entities include symptoms, syndromes, prescriptions, medicinal materials, meridians, etc., and relationships include various types such as treatment methods, meridians, and combinations). This module identifies corresponding entities in user questions and retrieves their relationship information, providing structured knowledge support for question and answering.

[0009] The large model fine-tuning module includes incremental pre-training, low-rank adaptation (LoRA) fine-tuning and direct preference optimization (DPO). Incremental pre-training is used to allow the large model to continue learning domain knowledge on traditional Chinese medicine texts. LoRA fine-tuning adapts to traditional Chinese medicine question-answering tasks under small-scale parameter adjustments, and DPO optimization further fine-tunes the output quality according to human preferences.

[0010] The large model fine-tuning module includes:

[0011] Incremental pre-training: By continuously training large models on new domains or specific data, supplement and optimize their existing knowledge systems;

[0012] Low-Rank Adaptation (LoRA) fine-tuning: Use a very small number of new parameters to quickly adapt to new tasks or new data, reducing the fine-tuning overhead;

[0013] Preference Optimization (DPO): Directly optimize the model output according to human or system preferences to make it more in line with the expected performance;

[0014] Dual Retrieval Enhancement Module: Includes a text retrieval sub-module and a knowledge graph retrieval sub-module. The text retrieval sub-module vectorizes the pre-processed traditional Chinese medicine literature based on the LangChain framework and stores it in a vector database, and retrieves and recalls user questions through a retrieval algorithm; The knowledge graph retrieval sub-module maps the question to the constructed traditional Chinese medicine knowledge graph through named entity recognition and intent analysis, and retrieves relevant entity and relationship information;

[0015] Prompt Template Module: Generate a dynamic prompt word template according to the user question and the text information and knowledge graph information returned by the dual retrieval module, and use the retrieval result and the question as context to input into the large model;

[0016] Answer Generation Module: Based on the fine-tuned large model, receive the context prompt generated by the prompt template module, and call external knowledge in real time during dialogue generation to achieve the generation of high-quality traditional Chinese medicine question and answer responses.

[0017] For the described traditional Chinese medicine knowledge Q&A system based on fine-tuning a large model and dual retrieval enhancement, the incremental pre-training unit of the large model fine-tuning module: Adopt the ChatGLM3-6B-Base model and conduct a large amount of pre-training on the constructed traditional Chinese medicine corpus to enhance the model's ability to understand the traditional Chinese medicine context. During the pre-training process, the model optimizes the parameters by minimizing the language modeling loss function L :

[0018]

[0019] where \(x_t\) represents the \(t\)-th word and \(x_{<t}\) represents the sequence of words before it. Through training on the traditional Chinese medicine domain corpus, the model can more accurately understand and generate text content related to traditional Chinese medicine.

[0020] For the described traditional Chinese medicine knowledge Q&A system based on fine-tuning a large model and dual retrieval enhancement, the Low-Rank Adaptation (LoRA) unit of the large model fine-tuning module: Adopt freezing some parameters of the large model and injecting trainable matrices to significantly reduce the number of trainable parameters.

[0021] In conventional model fine-tuning, for the fully connected weight matrix of a certain layer Perform gradient updates, which requires training The core idea of ​​LoRA is: First, freeze the original weights: Remain unchanged and do not participate in the gradient update; then, inject low-rank increments: In the forward calculation, use a rank of The trainable matrix To approximate the update amount:

[0022]

[0023] Finally, the new weights are synthesized

[0024]

[0025] in (scalar scaling factor) Optional.

[0026] The TCM knowledge question-answering system based on a fine-tuned large model and dual retrieval enhancement, wherein the direct preference optimization (DPO) unit of the large model fine-tuning module utilizes a direct preference optimization (DPO) strategy to optimize answers according to clinical expert preferences.

[0027] DPO is used to optimize the model scoring function using "human preferences" during the fine-tuning phase, directly maximizing the relative scores of the preferred samples. , where for the same input , human preferences Better than . Let the model be any The scoring function is:

[0028]

[0029] The goal of DPO is to The corresponding pairwise loss can be written as:

[0030]

[0031] in is the Sigmoid function.

[0032] The TCM knowledge question-answering system based on a fine-tuned large model and dual retrieval enhancement is described. The text retrieval submodule of the dual retrieval enhancement module converts question queries into embedded vectors through the vector retrieval method introduced by LangChain, quickly retrieving relevant fragments in TCM classics, guidelines, and medical papers. The multi-round retrieval upper limit of the recall results can be dynamically adjusted to ensure information coverage.

[0033] The text retrieval submodule is based on LangChain vector retrieval: First, the user query Mapped to Dimensional Embedding , for each fragment in the classic literature, guidelines and papers of traditional Chinese medicine Precomputed embeddings Retrieval is done by cosine similarity:

[0034]

[0035] For all Sort in descending order and select the first fragments To ensure information coverage in multiple rounds of searches, the search limit can be dynamically adjusted:

[0036]

[0037] in Indicates the The incremental recall coverage rate of the new round, is the adjustment factor; this mechanism effectively controls the retrieval cost while ensuring high coverage.

[0038] The described TCM knowledge question-answering system based on a fine-tuned large model and dual retrieval enhancement, the knowledge graph retrieval submodule of the dual retrieval enhancement module assists in retrieval through a pre-constructed TCM knowledge graph, and the knowledge graph is composed of numerous TCM entities (such as symptoms, syndromes, prescriptions, medicinal materials, meridians, etc.) and their relationships (such as treatment method relationships, meridian relationships, compatibility relationships, etc.); when the user's question involves a specific entity, the knowledge graph retrieval submodule can associate the corresponding knowledge structure and output relevant knowledge facts.

[0039] Assume that the knowledge graph is a directed graph ,in is a collection of entities, For user questions , first use the named entity recognition function

[0040]

[0041] Get the problem entity set . Then extract the subgraph

[0042]

[0043] in represents the shortest path length on the graph, is the preset hop count threshold, for For each candidate triple , by embedding the function Mapping to vector space:

[0044]

[0045] in Encoding function for pre-trained language model, encoding the question into a vector . Calculate similarity:

[0046]

[0047] The top-K relevant facts are selected and output in descending order of similarity. This method takes into account both structured retrieval and semantic matching, ensuring that the returned knowledge facts are both consistent with the graph relationship and highly relevant to the question semantics.

[0048] The described TCM knowledge question-and-answer system based on a fine-tuned large model and dual retrieval enhancement, wherein the prompt template module is designed with a customized Prompt format for TCM questions and answers, which is used to integrate the retrieved text information and knowledge graph information, such as organizing user questions, retrieval result summaries and related entity information into specific templates to enhance the contextual prompts generated by the model.

[0049] Let the user ask the question , the retrieval result summary set is , the knowledge graph entity information is .definition

[0050] Prompt Generation Function

[0051]

[0052] Among them, “||” represents text concatenation, weight Based on query-document similarity distribution, According to the connectivity of the entity in the graph, As context input to the model, by maximizing To improve the accuracy and consistency of generating TCM questions and answers.

[0053] The TCM knowledge question-answering system based on a fine-tuned large model and dual retrieval enhancement, wherein the answer generation module uses the fine-tuned ChatGLM3-6B model to output the final question-answering result through a generative dialogue after receiving the context provided by the prompt template module, so as to ensure the coherence of the answer content and the professionalism of TCM.

[0054] Suppose the context sequence output by the prompt template is , the model parameters after fine-tuning are The answer generation module is As a condition, generate question-answer responses through autoregression , and its joint probability is

[0055]

[0056] In order to control the generation diversity, a temperature coefficient can be introduced

[0057]

[0058] Or use Top-k truncation to ensure the accuracy of TCM terminology. Finally, use Greedy or Beam Search

[0059]

[0060] Get the optimal sequence. At the same time, through the dialogue state vector (like Enc ) Dynamically updates context to maintain conversational coherence and TCM expertise.

[0061] The beneficial effects of the present invention are:

[0062] This paper proposes a TCM knowledge question-answering system based on a fine-tuned large model and dual-search enhancement. By organically integrating traditional TCM concepts with modern artificial intelligence technology, it not only achieves intelligent and efficient TCM knowledge question-answering, but also provides new insights for the digital inheritance of TCM knowledge. The system has a clear overall architecture and interoperable functional modules, making it widely applicable in TCM education, online medical consultation, and diagnostic assistance, possessing significant practical value and industrial prospects. BRIEF DESCRIPTION OF THE DRAWINGS

[0063] Figure 1 Build flow charts for large models.

[0064] Figure 2 Constructing a flow chart for dual search enhancement generation.

[0065] Figure 3 Query graph for knowledge graph.

[0066] Figure 4 This is an example of a prompt template.

[0067] Figure 5 This is the automatic evaluation result diagram.

[0068] Figure 6 This is the result of manual evaluation. DETAILED DESCRIPTION

[0069] The present invention is described in detail below with reference to specific embodiments.

[0070] A TCM knowledge question-answering system based on fine-tuning a large model and dual retrieval enhancement includes a data preprocessing module, a large model fine-tuning module, a dual retrieval enhancement module, a prompt template module, and an answer generation module. The large model construction flow chart is as follows: Figure 1 Shown, including:

[0071] A1. Data preprocessing module: collects texts of ancient Chinese medicine classics such as "Huangdi Neijing" and "Shanghan Lun", as well as modern Chinese medicine guidelines, journal articles and clinical case records. Through data cleaning, remove noise and duplicate content, standardize Chinese medicine terms (such as standardized disease names, drug names), and use word segmentation tools to segment the text. Finally, the sorted text is constructed into a special corpus in the field of Chinese medicine for subsequent model training and retrieval. The corpus can use database or vector storage to store the information required for text and knowledge graphs. In addition, during the data preprocessing process, the names of medicinal materials, prescriptions, etc. are also standardized and encoded, and a special indexing mechanism is constructed to support fast matching during retrieval. The preprocessing module can regularly update and merge new Chinese medicine literature, such as adding materials such as "Compendium of Materia Medica" and "Shennong's Herbal Classic" to continuously enrich the corpus;

[0072] A2. Large Model Training (Fine-tuning) Module: This module uses the constructed TCM corpus as a foundation to perform domain adaptation and fine-tuning on the pre-trained language model. Specifically, it includes the following steps:

[0073] The specific steps of step A2 are as follows:

[0074] A21. Select large, open-source conversational models such as ChatGLM3-6B as the base model for incremental pre-training. Retrain on the Traditional Chinese Medicine (TCM) corpus (incremental pre-training) to enable the model to better capture TCM contextual information. For example, the original model can be trained for several rounds to learn TCM terminology, etiology, and pathogenesis. Model training can be performed using a distributed strategy, leveraging a multi-GPU cluster to accelerate incremental pre-training and fine-tuning. Common optimizers (such as AdamW) and learning rate scheduling strategies are also used during training. When applying LoRa technology, different low-rank matrices can be injected into different layers to balance training efficiency and model performance.

[0075] A large amount of pre-training is performed on the constructed TCM corpus to enhance the model's ability to understand the TCM context. During the pre-training process, the model optimizes the parameters by minimizing the language modeling loss function L. :

[0076]

[0077] Among them, \(x_t\) represents the \(t\)-th word, and \(x_{<t}\) represents the sequence of words before it. Through training on the corpus in the field of traditional Chinese medicine, the model can more accurately understand and generate text content related to traditional Chinese medicine.

[0078] A22. During the fine-tuning process, the Low-Rank Adaptation (LoRA) technique is introduced to reduce the tuning cost. The specific approach is to freeze most of the parameters of the original model and only insert trainable low-rank matrices in each layer of the Transformer, thereby significantly reducing the number of trainable parameters.

[0079] In the conventional model fine-tuning, for the fully connected weight matrix of a certain layer to perform gradient update, parameters need to be trained. The core idea of LoRA is: freeze the original weights: let remain unchanged and not participate in gradient update; inject low-rank increments: in the forward calculation, use a trainable matrix with a rank of to approximate the update amount,

[0080]

[0081] synthesize new weights:

[0082]

[0083] where (scalar scaling factor) is optional.

[0084] A23. The Direct Preference Optimization (DPO) strategy is adopted to optimize the training according to the preferences of traditional Chinese medicine experts or common cases for the output answers. DPO makes the model's generated results more in line with the expectations of domain experts by maximizing the probability of preference samples without the need to train a reward model. Through the above strategies, a "QiZhi" model specifically adapted to the traditional Chinese medicine knowledge问答 task is finally obtained, and this model has significant advantages in understanding the traditional Chinese medicine context, identifying traditional Chinese medicine terms, and generating diagnosis and treatment suggestions.

[0085] DPO is used to optimize the model scoring function using "human preferences" in the fine-tuning stage, directly maximizing the relative scores of preferred samples. Suppose there is a preference dataset , where for the same input , the human preference is better than . Let the scoring function of the model for any be:

[0086]

[0087] Then the goal of DPO is to make The corresponding pairwise loss can be written as:

[0088]

[0089] in is the Sigmoid function.

[0090] A3. To address the issues of missing knowledge and hallucination in purely generative question answering, this system has designed a dual retrieval enhancement module to obtain knowledge related to the question to enhance the answering ability of the large model.

[0091] The specific steps are as follows:

[0092] A31. Extract ancient texts related to the problem, and then enhance the reference basis of the large model based on ancient documents.

[0093] Considering the current maturity of the LangChain technical framework, a vector retrieval system based on the LangChain framework was developed to more accurately match similar text segments. This system segments the text in the Traditional Chinese Medicine (TCM) corpus (e.g., by paragraph or sentence). Using text embedding techniques (such as Text2Vec and BERT), each segment is encoded as a vector and stored in a vector database (such as Milvus or Faiss). When a user enters a question, the question is vectorized. A vector retrieval algorithm then searches the corpus for the text segments with the highest semantic relevance to the question, returning the most relevant information. This module's efficient search of the text vector database enables rapid recall of external knowledge, supplementing domain knowledge beyond the larger model's own corpus. During the retrieval process, multiple search strategies can be configured, such as expanding the query vector or dynamically adjusting the search threshold, to ensure richness and coverage of the retrieved content.

[0094] The text retrieval submodule is based on LangChain vector retrieval: First, the user query Mapped to Dimensional Embedding For each fragment in the classic literature, guidelines and papers of traditional Chinese medicine Precomputed embeddings Cosine similarity is used for retrieval

[0095]

[0096] For all Sort in descending order and select the first fragments To ensure information coverage in multiple rounds of searches, the search limit can be dynamically adjusted:

[0097]

[0098] in Indicates the The incremental recall coverage rate of the new round, is the adjustment factor; this mechanism effectively controls the retrieval cost while ensuring high coverage.

[0099] A32. Traditional Chinese Medicine knowledge graph retrieval. Since another major aspect of traditional Chinese medicine knowledge content is the traditional Chinese medicine knowledge graph, the entities of the user's questions are obtained to retrieve relevant traditional Chinese medicine entities from the knowledge graph to assist large model reasoning.

[0100] In addition to text retrieval, the system also constructs a Traditional Chinese Medicine (TCM) knowledge graph for structured search. This graph is constructed using a semi-automatic knowledge extraction process: First, core entities (such as disease names, syndromes, prescriptions, medicinal materials, meridians, and internal organs) and inter-entity relationships (such as "drug-treatment-disease," "prescription-meridian-internal organs," or "prescription-drug compatibility") are identified from collected TCM literature. These identified entities and relationships are then organized into a graph database (e.g., Neo4j). When a user asks a question, the system uses named entity recognition and semantic analysis to identify the core entities or intent involved in the question. The corresponding nodes and their associated information are then retrieved from the knowledge graph. For example, if a question involves "lung deficiency cough," the knowledge graph retrieval submodule can directly locate the relationship between "cough" and related causes, syndromes, and prescriptions, providing accurate medical knowledge support for the answer. The knowledge graph incorporates a multi-level knowledge structure, supporting multi-level search and reasoning, from macro-level "internal organs" and meridians to micro-level "symptoms" and prescriptions. During graph retrieval, graph query languages ​​(such as Cypher) or path search algorithms can be used to quickly retrieve relevant knowledge paths, assisting generative models in mining implicit associations from the graph structure. The TCM knowledge graph is composed of numerous entities and relationships, and can be supplemented with new knowledge through a regular update mechanism to ensure that the graph continues to reflect the latest achievements in the field.

[0101] Assume that the knowledge graph is a directed graph ,in is a collection of entities, For user questions , first use the named entity recognition function

[0102]

[0103] Get the problem entity set . Then extract the subgraph

[0104]

[0105] in represents the shortest path length on the graph, is the preset hop count threshold, for For each candidate triple , by embedding the function Mapping to vector space:

[0106]

[0107] in Encoding function for pre-trained language model, encoding the question into a vector . Calculate similarity:

[0108]

[0109] The top-K relevant facts are selected and output in descending order of similarity. This method takes into account both structured retrieval and semantic matching, ensuring that the returned knowledge facts are both consistent with the graph relationship and highly relevant to the question semantics.

[0110] A4 prompt template module, based on the user question and the dual search module returns text information and knowledge graph information, generates a dynamic prompt word template, the search results and questions as context input to the large model; such as Figure 4 As shown in the figure, by giving the large model a bunch of relevant information directly, it is difficult for the large model to understand what the user wants to do. Therefore, a set of prompt templates is needed.

[0111] The prompt template module is responsible for integrating the retrieved information with the user's question to generate prompts suitable for the large model. Specifically, a specialized template is designed based on the characteristics of TCM question-answering. For example, the prompt contains sections such as "question," "reference knowledge," and "answer requirements" in a fixed format. The system then populates the "reference knowledge" section of the prompt template with relevant literature excerpts returned by the text retrieval submodule and relevant entity relationship information returned by the knowledge graph retrieval submodule. The user's original question is then added to the "question" section to complete the prompt input. The prompt template can be pre-configured with various formats to accommodate different question types, such as example context and answer structure requirements, to constrain model generation. This prompt design allows the large model to reference retrieved knowledge content when generating answers, significantly reducing knowledge gaps and hallucinations. Prompt templates can also incorporate stylistic and formatting cues, such as requiring rigorous and formal answer language and accurate terminology, to further improve answer quality. The system dynamically selects the most suitable prompt template and populates it with the corresponding content, reducing reliance on manual template design and increasing flexibility.

[0112] Let the user ask the question , the retrieval result summary set is , the knowledge graph entity information is .definition

[0113] Prompt Generation Function

[0114]

[0115] Among them, “||” represents text concatenation, weight Based on query-document similarity distribution, According to the connectivity of the entity in the graph, As context input to the model, by maximizing

[0116]

[0117] To improve the accuracy and consistency of generating TCM questions and answers.

[0118] A5. The answer generation module, based on the fine-tuned large model, receives contextual prompts generated by the prompt template module and uses external knowledge in real time during conversation generation to generate high-quality TCM Q&A answers.

[0119] After obtaining the prompt word input, the system calls the pre-fine-tuned model to generate the final answer. Specifically, the user question contained in the prompt word is input into the ChatGLM3-6B model together with the retrieved external knowledge, and the model generates the corresponding text answer. The answer generation process is autoregressive dialogue generation. The model will gradually output professional answer content related to the question based on the input prompt. In this process, the model not only utilizes the language ability accumulated during pre-training, but also combines the latest traditional Chinese medicine knowledge information contained in the prompt, so that it can answer professional questions in the fields of traditional Chinese medicine diagnosis, treatment recommendations, and prescription usage. The generated answers usually include elements such as dialectical analysis, prescription recommendations, and conditioning suggestions to meet the characteristics of traditional Chinese medicine questions and answers.

[0120] The answer generation module also performs necessary post-processing to ensure output compliance. For example, the generated text is verified for medical terminology and filtered for sensitive terms, and the output format (such as paragraphs and sequence numbers) is fine-tuned. The system incorporates a confidence assessment mechanism: when the uncertainty of the model output is high, the system can automatically initiate additional searches or prompt the user to provide more information to enhance the reliability of the answer. The final generated answer is presented to the user in its entirety, and can include references and knowledge graph information to enhance persuasiveness.

[0121] Figure 5 and Figure 6The paper comprehensively validates the performance of the large-scale model in various TCM question-answering scenarios through a combination of automated evaluation and expert scoring. The results demonstrate that the model outperforms existing general-purpose large-scale models and other TCM-specific models in both accurately answering TCM theories and reasoning on complex syndrome differentiation and treatment problems, demonstrating strong application potential and broad prospects for widespread adoption.

[0122] It should be understood that those skilled in the art can make improvements or changes based on the above description, and all such improvements and changes should fall within the scope of protection of the appended claims of the present invention.

Claims

1. A TCM knowledge question-answering system based on large model fine-tuning and dual retrieval enhancement, characterized by: It includes a data preprocessing module, a large model fine-tuning module, a dual search enhancement module, a prompt template module, and an answer generation module: the data preprocessing module is used to collect and clean ancient Chinese medical books, modern literature, and clinical case data to construct Chinese medicine corpus; Large model fine-tuning module: used to perform domain adaptation on large pre-trained language models, including incremental pre-training units, low-rank adaptation (LoRA) fine-tuning, direct preference optimization (DPO), and supervised fine-tuning units. The incremental pre-training unit continues to pre-train pre-trained models such as ChatGLM3-6B using the corpus; the supervised fine-tuning unit fine-tunes the pre-trained model by generating dialogue data through self-questioning and self-answering, entity self-instruction, etc., and combines low-rank adaptation (LoRA) and direct preference optimization (DPO) strategies to improve the model's ability to answer TCM professional terminology and syndrome differentiation and treatment logic; LoRA fine-tuning adapts to TCM question-answering tasks with small-scale parameter adjustments, and DPO optimization further fine-tunes the output quality according to human preferences; Dual retrieval enhancement module: includes a text retrieval submodule and a knowledge graph retrieval submodule. The text retrieval submodule vectorizes the pre-processed TCM literature based on the LangChain framework and stores it in a vector database, and retrieves and recalls user questions through a retrieval algorithm. The knowledge graph retrieval submodule maps questions to the constructed TCM knowledge graph through named entity recognition and intent analysis, and retrieves relevant entity and relationship information. Prompt template module: Generates dynamic prompt word templates based on user questions and the text information and knowledge graph information returned by the dual search module, and inputs the search results and questions as context into the big model; Answer generation module: Based on the fine-tuned large model, it receives contextual prompts generated by the prompt template module, calls external knowledge in real time during dialogue generation, and realizes high-quality TCM question and answer generation.

2. The TCM knowledge question-answering system according to claim 1, characterized in that: The incremental pre-training unit uses the ChatGLM3-6B-Base model and performs a large amount of pre-training on the constructed TCM corpus to enhance the model's ability to understand the TCM context. During the pre-training process, the model optimizes the parameters by minimizing the language modeling loss function L. : ; where \(x_t\) represents the \(t\)-th word, and \(x_{<t}\) represents the sequence of words before it; through training on the corpus in the field of traditional Chinese medicine, the model can more accurately understand and generate text content related to traditional Chinese medicine.

3. The TCM knowledge question-answering system according to claim 1, characterized in that: The low-rank adaptation (LoRA) freezes some parameters of the large model and injects them into the trainable matrix to significantly reduce the number of trainable parameters; In conventional model fine-tuning, the fully connected weight matrix of a certain layer Perform gradient updates, which requires training parameters; the core idea of ​​LoRA is to freeze the original weight: Remain unchanged and do not participate in gradient update; inject low-rank increment: in the forward calculation, use a rank of The trainable matrix To approximate the update amount, ; Synthesize new weights: ;in (scalar scaling factor) Optional.

4. The TCM knowledge question-answering system according to claim 1, characterized in that: The direct preference optimization (DPO) unit optimizes the answers according to the clinical expert's preferences using a direct preference optimization (DPO) strategy; DPO is used to optimize the model scoring function using "human preferences" during the fine-tuning phase, directly maximizing the relative scores of the preferred samples; there is a preference data set , where for the same input , human preferences Better than ; Let the model be any The scoring function is: ; then the goal of DPO is to make The corresponding pairwise loss can be written as: ;in is the Sigmoid function.

5. The TCM knowledge question-answering system according to claim 1, characterized in that: The text retrieval submodule uses the vector retrieval method introduced by LangChain to convert question queries into embedded vectors, quickly searching for relevant fragments in traditional Chinese medicine classics, guidelines, and medical papers. The upper limit of multiple rounds of retrieval can be dynamically adjusted to ensure information coverage. The text retrieval submodule is based on LangChain vector retrieval: First, the user query Mapped to Dimensional Embedding , for each fragment in the classic literature, guidelines and papers of traditional Chinese medicine Precomputed embeddings Retrieval by cosine similarity ; for all Sort in descending order and select the first fragments To ensure information coverage in multiple rounds of searches, the search limit can be adjusted dynamically: ;in Indicates the The incremental recall coverage rate of the new round, is the adjustment factor; this mechanism effectively controls the retrieval cost while ensuring high coverage.

6. The TCM knowledge question-answering system according to claim 1, characterized in that: The knowledge graph retrieval submodule assists retrieval through a pre-built TCM knowledge graph, which is composed of numerous TCM entities and their relationships. When a user's question involves a specific entity, the knowledge graph retrieval submodule can associate the corresponding knowledge structure and output relevant knowledge facts. Assume that the knowledge graph is a directed graph ,in is a collection of entities, For a set of relationships; for user questions , first use the named entity recognition function ; Get the problem entity set ; Then extract the subgraph ;in represents the shortest path length on the graph, is the preset hop count threshold, for For each candidate triple , by embedding the function Mapping to vector space: ;in Encoding function for pre-trained language model, encoding the question into a vector ; Calculate similarity: ; and select the Top-K related facts for output in descending order of similarity; this method takes into account both structured retrieval and semantic matching, ensuring that the returned knowledge facts are both consistent with the graph relationship and highly relevant to the question semantics.

7. The TCM knowledge question-answering system according to claim 1, characterized in that: The prompt template module is designed with a customized prompt format for TCM Q&A, which is used to integrate retrieved text information and knowledge graph information. For example, user questions, retrieval result summaries, and related entity information are organized into a specific template to enhance the contextual prompts generated by the model. Let the user ask the question , the retrieval result summary set is , the knowledge graph entity information is ;definition Prompt Generation Function ; "||" indicates text concatenation, weight Based on query-document similarity distribution, According to the connectivity of the entity in the graph, As context input to the model, by maximizing: ; to improve the accuracy and consistency of generating TCM questions and answers.

8. The TCM knowledge question-answering system according to claim 1, characterized in that: After receiving the context provided by the prompt template module, the answer generation module uses the fine-tuned ChatGLM3-6B model to output the final question-answering result through a generative dialogue method to ensure the coherence of the answer content and the professionalism of traditional Chinese medicine; Suppose the context sequence output by the prompt template is , the model parameters after fine-tuning are ; The answer generation module is As a condition, generate question-answer responses through autoregression , and its joint probability is ; To control the generation diversity, a temperature coefficient can be introduced Or use Top-k truncation to ensure the accuracy of TCM terminology; finally use Greedy or Beam Search ; Get the optimal sequence; At the same time, through the dialogue state vector (like Enc ) Dynamically updates context to maintain conversational coherence and TCM expertise.

Images