Two-stage knowledge reasoning distillation method and system for medical question and answer tasks and medium

Abstract

The invention discloses a two-stage knowledge reasoning distillation method and system for a medical question and answer task and a medium, and relates to the field of medical question and answer, and the method comprises the steps: employing a teacher model to carry out the data analysis and structural processing of an original medical question and answer data set, and obtaining a plurality of medical knowledge triples; performing one-stage knowledge distillation on the student model to obtain a student model after one-time training, and then performing verification to obtain a misjudgment sample; performing data analysis and reasoning on the misjudgment sample set by adopting a teacher model to obtain a plurality of medical reasoning tetrads, performing two-stage knowledge distillation on the student model, and performing reasoning migration training on the student model to obtain a student model subjected to secondary training; and after the student model after secondary training is deployed to the edge computing platform, generating a medical answer and a corresponding medical reasoning path according to the received natural language inquiry information. According to the method, the model operation efficiency can be improved, and the understanding ability and reasoning transparency of the model for complex input are enhanced.

Description

Technical Field

[0001] The present application relates to the field of medical question answering, and in particular to a two-stage knowledge reasoning distillation method, system, and medium for medical question answering tasks. Background Art

[0002] In healthcare, medical question-answering systems based on large language models (LLMs) are gaining popularity, gradually penetrating key areas such as clinical decision support, online consultations, intelligent medical guidance, and medical information retrieval. While current medical question-answering systems based on LLMs hold great potential, they suffer from key limitations in terms of model compression, deployment feasibility, interpretability, and robustness.

[0003] (1) The model is large in size and has high deployment costs. The parameter scale of current mainstream large models such as GPT-4, LLaMA2-70B and Qwen-72B is often in the billions or even tens of billions, which places extremely high demands on computing resources, memory and energy consumption. Most of these models rely on cloud deployment and need to run on high-end GPU server clusters. They cannot be directly applied to edge scenarios with limited computing power, such as community health service centers, mobile health terminals or smart wearable devices. Especially in the environment with limited primary medical conditions, lack of high computing power equipment or stable network connection, such models cannot be deployed at all, which seriously restricts their application and technology promotion.

[0004] (2) There are serious deficiencies in interpretability. Most language model outputs are limited to short text responses, lacking clear reasoning paths and causal chains, and cannot answer questions such as "where does the answer come from" or "how to draw conclusions based on symptoms." This "black box" decision-making process makes it difficult to meet the doctors' need for abductive judgment, and also hinders the acceptance and adoption of model results in real scenarios. Some current attempts have provided explanations through attention visualization, prompt word intervention, etc., but these methods are limited to formal salience prompts and have not yet formed a structured and traceable medical reasoning mechanism. Therefore, due to the lack of causal chain mapping and reconstruction capabilities, when faced with complex decisions, ambiguous descriptions, or new sample distributions, the model is prone to "knowing the answer but not the reason," and thus cannot form a closed-loop system with enhanced interpretability.

[0005] (3) Due to the significant decrease in model capacity after parameter compression, the robustness of the small model in the face of actual user input is also significantly reduced. Medical questions in reality have typical characteristics such as colloquialism, incomplete information, and logical jumps. Patients often do not have standard medical expression skills, which makes the input text likely to contain multiple parallel symptoms, fuzzy time descriptions, and redundant information. The large model can rely on its huge context modeling capabilities to achieve fault tolerance, but the small model after distillation lacks the perception and error correction mechanism of complex structures, and is prone to problems such as symptom misunderstanding, disease confusion, or logical breaks. Multiple studies have pointed out that when faced with the integration of multiple diseases and cross-symptom input, the prediction accuracy of the distillation model drops significantly, and even creates the risk of misdiagnosis and misjudgment.

[0006] Current distillation frameworks generally lack specialized training mechanisms for misjudgment. Once a model misjudgments during validation or actual application, the faulty paths and logic cannot be identified, recorded, or fixed. As distillation cycles progress, these logical blind spots accumulate, even solidifying incorrect habits during training, ultimately leading to model behavior where the output appears reasonable but the reasoning is completely incorrect. Summary of the Invention

[0007] The purpose of this application is to provide a two-stage knowledge reasoning distillation method, system and medium for medical question-answering tasks, which can improve the model's operating efficiency and enhance the model's ability to understand complex inputs and reasoning transparency.

[0008] To achieve the above objectives, this application provides the following solutions.

[0009] In the first aspect, the present application provides a two-stage knowledge reasoning distillation method for medical question answering tasks, comprising the following steps.

[0010] Get the pre-trained teacher model.

[0011] The teacher model is used to perform data analysis and structured processing on the original medical question-and-answer dataset to obtain multiple medical knowledge triples and divide them into training sets and validation sets.

[0012] The training set is input into the student model for a first-stage knowledge distillation to guide the student model to learn the knowledge expression method and answering ability of the teacher model in the medical question-answering task, thereby obtaining a student model that has been trained once; based on the verification set, the student model that has been trained once is verified to obtain misjudgment samples; all the misjudgment samples constitute a misjudgment sample set.

[0013] The teacher model is used to perform data analysis and reasoning on the misjudged sample set to obtain a plurality of medical reasoning quadruplets; the medical reasoning quadruplets include a medical reasoning path from symptoms to diagnosis.

[0014] Multiple medical reasoning quadruples are used as training data and input into the student model for two-stage knowledge distillation to perform reasoning transfer training on the student model to obtain a student model that has completed secondary training. After the student model has completed secondary training, it is deployed on the edge computing platform and generates medical answers and corresponding medical reasoning paths based on the received natural language consultation information.

[0015] Secondly, this application provides a two-stage knowledge reasoning distillation system for medical question-answering tasks, including the following modules.

[0016] The model loading module is used to deploy the student model that has completed secondary training based on the two-stage knowledge reasoning distillation method for medical question-answering tasks to the edge computing platform.

[0017] The question-answering processing module is used to: after receiving the natural language question information input by the user, call the student model that has completed secondary training to generate medical answers and corresponding medical reasoning paths.

[0018] The reasoning path display module is used to: display the medical reasoning path in a structured or visual form and receive feedback error samples.

[0019] The error correction and update module is used to: regularly collect the error samples, upload the error samples to the server, and re-execute the steps of the two-stage knowledge reasoning and distillation method for medical question-answering tasks to achieve incremental model learning or remote update and redeployment.

[0020] In a third aspect, the present application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the steps of a two-stage knowledge reasoning distillation method for medical question-answering tasks.

[0021] According to the specific embodiments provided in this application, the present application achieves the following technical effects: A two-stage distillation and transfer architecture is designed. Based on a teacher model, semantic parsing is performed on a large-scale raw medical question-and-answer dataset to construct medical knowledge triples. This knowledge triples are then fed into a student model for a first-stage knowledge distillation, guiding the student model to learn the teacher model's knowledge representation and answering capabilities in medical question-and-answer tasks, resulting in a fully trained student model. This mechanism substantially enhances the student model's ability to understand why an answer is valid. This mechanism, unlike traditional soft-label distillation that relies solely on "form transfer" of distributional imitation, achieves distillation improvements at the "knowledge level." The trained student model is then evaluated, automatically identifying samples that deviate significantly from the standard answer or the teacher's answer as misclassified samples. In the second stage, the student model is trained based on these misclassified samples, while introducing medical reasoning pathways to simulate the reasoning process used by real doctors in diagnosis. This significantly strengthens the model's logical generation and self-correction capabilities, enhancing its ability to understand complex inputs and its reasoning transparency.

[0022] By introducing structured triple knowledge distillation and the reasoning chain migration mechanism driven by misjudgment samples, the logical capability migration from large models to small models is achieved, solving the key bottleneck problem of the existing medical question-answering system, which is "compressible but not interpretable, generative but not reasonable". It has four major technical advantages of high performance, low resource occupation, high robustness and strong deployment adaptability, which improves the model operation efficiency and can be widely used in primary diagnosis and treatment, remote assistance, mobile health and other scenarios. It has important engineering value and technology promotion prospects in promoting the popularization of intelligent medical systems to terminals. BRIEF DESCRIPTION OF THE DRAWINGS

[0023] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0024] Figure 1 A flowchart of a two-stage knowledge reasoning distillation method for medical question-answering tasks provided in one embodiment of the present application.

[0025] Figure 2 A simplified flowchart of a two-stage knowledge reasoning distillation method for medical question-answering tasks provided in one embodiment of the present application.

[0026] Figure 3 This is the overall framework diagram of the medical question-answering task in the Helicobacter pylori diagnosis task.

[0027] Figure 4Functional flowchart for the medical question answering task in the Helicobacter pylori diagnosis task.

[0028] Figure 5 Schematic diagram of the test results for the Helicobacter pylori diagnosis task. DETAILED DESCRIPTION

[0029] The following will be combined with the drawings in the embodiments of this application to clearly and completely describe the technical solutions in the embodiments of this application. Obviously, the embodiments described are only part of the embodiments of this application, not all of the embodiments. Based on the embodiments in this application, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of this application.

[0030] This application aims to solve key technical problems existing in existing medical question-answering systems, such as large models, lack of logical explainability, high misjudgment rate, and inability to transfer reasoning capabilities through distillation. By constructing a structured knowledge distillation mechanism and an inference chain migration enhancement mechanism, it not only achieves efficient compression and migration of large-model medical knowledge, but also significantly improves the logical consistency, self-correction ability and deployment adaptability of the student model in medical question-answering scenarios, providing a practical technical path for the application of intelligent medical question-answering systems in edge terminals.

[0031] In this application, with the goal of building and deploying student models, it demonstrates how to start from medical question-and-answer data and gradually complete small model training and reasoning capability compression through multiple stages such as structured extraction, knowledge transfer, logical reasoning training, and deployment optimization.

[0032] In order to make the above-mentioned purposes, features and advantages of the present application more obvious and easy to understand, the present application is further described in detail below with reference to the accompanying drawings and specific implementation methods.

[0033] In an exemplary embodiment, Figure 1 As shown, a two-stage knowledge reasoning distillation method for medical question-answering tasks is provided. The method is executed by a computer device, and can be executed solely by a computer device such as a terminal or a server, or can be executed jointly by a terminal and a server. In an embodiment of the present application, the method includes the following steps 101 to 105.

[0034] Step 101: Obtain a pre-trained teacher model.

[0035] In step 102, the teacher model is used to parse and structure the original medical question-and-answer dataset to obtain multiple medical knowledge triples, which are then divided into a training set and a validation set. Each original medical question-and-answer data set includes a patient's natural language inquiry information. The original medical question-and-answer dataset is large-scale data. The teacher model is then used to extract and construct semantically well-structured triples from the natural language information contained therein.

[0036] The medical knowledge triplet is a triplet data structure of question-medical answer-background knowledge; wherein the question (Question) is determined according to the natural language medical inquiry information, and usually includes descriptions such as symptoms, physical sensations, and onset time, for example, "I have been suffering from acid reflux and bloating recently", which represents the clinical chief complaint; the medical answer (Answer) is the medical advice generated by the teacher model after the natural language medical inquiry information is input into the teacher model. It is a standardized medical response with clinical rationality and terminology standardization, such as "may be related to Helicobacter pylori infection"; background knowledge (Knowledge) is the medical basis information related to the natural language medical inquiry information and the medical advice, such as "gastroscopy results indicate Hp positive", and the background knowledge includes but is not limited to disease definition, pathogenesis, diagnostic guidelines, examination recommendations, treatment methods, etc., and can be mapped to predefined medical knowledge graph entities and their relationships.

[0037] Specifically, background knowledge not only includes supplementary explanations for medical answers but also covers medical terminology and sentence structures related to diseases, examination methods, clinical guidelines, drug treatments, and pathogenesis. This knowledge can be understood as fragments of specialized knowledge supporting medical answers. This knowledge can be mapped into existing medical knowledge graphs, such as UMLS, SNOMED CT, or self-built disease-symptom-examination-treatment entity relationship graphs, to form a structured medical semantic network that can be parsed by machines.

[0038] Background knowledge can be obtained through medical knowledge graph queries, literature review, or large language model completion. This structured medical knowledge triple not only enables semantic reconstruction of natural language question-answering samples, but also provides a foundation for semantic connections for subsequent training.

[0039] Step 103: Input the training set into the student model for a stage of knowledge distillation to guide the student model to learn the knowledge expression method and answering ability of the teacher model in the medical question-answering task, and obtain a student model that has been trained once; based on the verification set, the student model that has been trained once is verified to obtain misjudgment samples; all the misjudgment samples constitute a misjudgment sample set.

[0040] During the first-stage knowledge distillation process, the medical knowledge triples output by the teacher model are transferred to the student model to achieve knowledge compression and ability retention. Specifically, the medical knowledge triples are fed into the student model as input, guiding it to learn to identify question elements, match answer patterns, and establish semantic associations of medical knowledge. The goal of the student model is to simulate the teacher model's ability to understand and generate triples, and learn its language expression and knowledge mapping capabilities under limited parameter resources. Based on this, a dual loss mechanism is introduced for the training of the student model.

[0041] The dual loss mechanism comprises a language modeling loss and a distillation loss, thereby reducing parameter size while preserving critical reasoning capabilities. The language modeling loss is used to encourage the student model to learn to generate medical answers from the question and background knowledge, thereby improving generation quality. The distillation loss is used to encourage the student model's output distribution to approximate the generation distribution of the teacher model. In specific applications, the distillation loss can employ the Kullback-Leibler divergence.

[0042] Through multiple rounds of training, the student model not only learns how to generate reasonable answers based on medical knowledge, but also potentially grasps the semantic dependencies between questions, answers, and background knowledge, thereby acquiring a certain level of semantic interpretation capabilities. This stage focuses on building the student model's basic expression and cognitive capabilities, making it a vehicle for subsequent logic training.

[0043] In a specific application, based on the validation set, the student model that has been trained once is verified to obtain misjudgment samples. This step quantifies the model output error by evaluating the prediction performance of the student model on the validation set and comparing its output with the similarity of the teacher model or the standard answer. It specifically includes the following steps (1)-(3).

[0044] (1) The medical knowledge triples in the validation set are input into the student model that has been trained once to obtain the model's predicted medical answers.

[0045] (2) Calculate the similarity between the medical answer predicted by the model and the medical answer in the corresponding medical knowledge triple.

[0046] (3) When the similarity meets the preset conditions, the medical knowledge triple is marked as a misjudgment sample. The preset conditions are: the BLEU score is lower than 0.6, or the answer similarity is lower than 75% (the specific value here can be adjusted by relevant technical personnel as needed), that is: samples with large deviations from the standard answer are defined as misjudgment samples. These misjudgment samples often reflect the model's inadaptability to implicit logic, rare combinations or spoken expressions, and its lack of ability in understanding ambiguous questions, identifying implicit causes, and conducting cross-sentence reasoning. They are representative and suitable as the core sample source for subsequent reasoning enhancement. The above process does not require manual labeling, is scalable and efficient, and can be applied to iterative optimization of large-scale question-answering data.

[0047] Step 104: Use the teacher model to perform data analysis and reasoning on the misjudged sample set to obtain a plurality of medical reasoning quadruplets; the medical reasoning quadruplets include a medical reasoning path from symptoms to diagnosis.

[0048] Specifically, the medical reasoning quaternary is a quaternary data structure of question-student answer-medical answer-medical reasoning path; wherein, the student answer is the medical answer generated by the student model after one training; the medical reasoning path is a set of medical statements with sequential logical relationships, presenting a causal reasoning link from symptoms to diagnosis.

[0049] A medical reasoning chain generally includes no fewer than three logical chains, such as "symptoms (indicating the patient's chief complaint, such as "upper abdominal distension" or "acid reflux") → examination (such as "gastroscopy recommended" or "Helicobacter pylori testing recommended") → diagnosis (such as "possible gastritis" or "omeprazole recommended")" as a standard template for a medical reasoning chain. This can also be expanded as needed to a complex structure such as "symptom representation → examination recommendation → examination results → disease diagnosis." For example, in the case of Helicobacter pylori infection, a medical reasoning chain will be generated: "The patient has long-term gastric distension → gastroscopy recommended → examination results indicate HP positive → infection considered." This not only restores the reasoning process of the teacher model, but also provides a clear logical training goal for the student model. This causal path can significantly improve the student model's understanding of the medical reasoning mechanism, which is different from the traditional black-box distillation method.

[0050] In step 105, multiple medical reasoning quadruples are used as training data and input into the student model for two-stage knowledge distillation to perform reasoning transfer training on the student model to obtain a student model that has completed secondary training. This processing enables the student model to have explainable reasoning capabilities, while improving the robustness and correction capabilities of misjudged samples.

[0051] During the two-stage knowledge distillation process, an inference supervision objective is constructed for the training of the student model, and a multi-task loss function is introduced; wherein, the inference supervision objective is used to guide the student model to simultaneously generate the corresponding medical reasoning path when generating medical answers, so that the final output form is "answer + reasoning"; the multi-task loss function includes language modeling loss, reasoning path generation loss and error correction supervision loss, thereby collaboratively optimizing the student model so that its generation results are significantly improved in the three dimensions of language, logic and robustness, and establishing a balanced optimization between logical consistency and answer quality.

[0052] The language modeling loss measures the coherence and accuracy of the student model's text generation; the inference path generation loss measures the consistency between the medical inference paths generated by the student model and the output of the teacher model; and the error correction supervision loss penalizes or rewards the student model for improving misclassified samples during the first stage of knowledge distillation, guiding the correction of misclassified patterns. Correspondingly, the multi-task loss function is: Total loss = α × language modeling loss + β × inference path generation loss + γ × error correction supervision loss; where α, β, and γ are hyperparameters that control the weights of different tasks, satisfying α + β + γ = 1.

[0053] In practical applications, the student model needs to generate medical answers consistent with the previous stage. Secondly, it must also generate corresponding reasoning chains, requiring semantic coherence, structural integrity, and logical rationality. Finally, after identifying previously misjudged questions, the student model must be able to correctly correct its output and provide reasoning support. This multi-objective optimization not only improves the student model's question-answering accuracy, but also enhances its reasoning consistency and interpretability, significantly improving the "post-compression distortion" common problem of small models.

[0054] In a specific application, the teacher model and the student model are both large language models; the ratio of the number of parameters of the student model to the number of parameters of the teacher model is lower than a preset percentage; the teacher model is a large language model with billions of parameters, and the models used include but are not limited to Qwen-72B, LLaMA2-70B, ChatGLM3-130B, etc., and has rich medical text reasoning capabilities; the student model is a compressed lightweight language model with a parameter number not exceeding 20% ​​of the teacher model, and the models used include but are not limited to LLaMA2-7B, Qwen-7B, and have edge deployment adaptability.

[0055] Due to the significant reduction in model parameters, its reasoning efficiency is significantly improved, and it can be run on mobile devices, consultation terminals, or clinic servers. Furthermore, the student model has a structured dual-output capability, which means that it outputs the answer and provides the reasoning chain at the same time. Doctors or users can trace back the model's reasoning process, enhancing their confidence in the results.

[0056] The student model that has completed secondary training is deployed to an edge computing platform with limited computing power. The edge computing platform includes but is not limited to edge servers, embedded diagnosis and treatment tools, or mobile medical devices. The student model that has completed secondary training has the following advantages: efficient question and answer response (inference delay less than 250ms), visualization of the reasoning chain (supporting logical tracing of each diagnostic output), fast loading and automatic update (facilitating low-cost and long-term operation in primary healthcare scenarios). After the student model that has completed secondary training is deployed to the edge computing platform, it generates medical answers and corresponding medical reasoning paths based on the received natural language inquiry information, which can ensure the provision of reliable, real-time, and explainable intelligent medical question and answer services in the edge environment.

[0057] like Figure 2 The figure further illustrates the specific strategies, methods, and logical relationships of each step in the two-stage knowledge reasoning distillation method, as follows: S1: Generate structured medical knowledge triples; S2: Perform structured knowledge distillation on the student model; S3: After completing initial distillation training, identify misclassified samples of the student model; S4: Invoke the teacher model for re-inference on these misclassified samples, generating medical reasoning quadruplets consisting of the question, student answer, correct answer (i.e., medical answer), and medical reasoning path; S5: Based on these medical reasoning quadruplets, perform reasoning transfer training on the student model; S6: The trained student model is deployed to the edge computing platform, completing model delivery. Experiments demonstrate that the model can complete question-answering tasks in real-world application scenarios with a response time of less than 250ms, while maintaining high stability and robustness.

[0058] In summary, this application introduces reasoning transfer and multi-task training mechanisms based on knowledge distillation through a systematic two-stage design, effectively realizing the compressible, interpretable and deployable transfer of knowledge of large-scale medical language models, and providing a technical solution with practical value for building a medical question-answering system for real-world scenarios.

[0059] To further illustrate the technical solution of this application, multiple typical implementation methods are described in detail below in combination with specific task scenarios.

[0060] In a specific application example, for the Helicobacter Pylori diagnosis task, focusing on the question-answering diagnosis scenario of Helicobacter pylori infection, the open medical question-answering dataset is used to extract user questions related to stomach discomfort, acid reflux, gastroscopy, etc., and the large language model Qwen-72B (Instruct) is used as the teacher model for semantic extraction and reasoning generation. Figure 3The figure shows the entire process from raw medical Q&A data preprocessing, structured triple extraction, first-stage knowledge distillation, second-stage inference transfer training, to model deployment on edge computing devices. The specific steps are as follows: (1) Construct a structured triple containing questions, medical answers, and background knowledge, for example: Question: "I have been having upper abdominal pain recently. Do I need to be checked for Helicobacter pylori?"

[0061] Medical answer: "A C13 breath test is recommended."

[0062] Background knowledge: "Helicobacter pylori infection can cause symptoms such as stomach pain and indigestion. The C13 breath test is a commonly used detection method."

[0063] These medical knowledge triples are fed into the teacher model for text vectorization and structural semantic modeling, and then fed into the student model as training data. The student model is a medium-sized model with approximately 7B parameters.

[0064] (2) In the first stage of knowledge distillation, the joint training strategy of language modeling and output distribution fitting is used to enable the student model to learn the knowledge expression and answering ability of the teacher model in the medical question answering task. After the training is completed, Figure 5 As shown in the figure, the Top-1 accuracy of the model in the Helicobacter pylori diagnosis subtask increased from 22.4% of the original model to 26.2%, indicating that the accuracy of the student model trained using the method of this application continued to improve in multiple evaluation rounds, outperforming the traditional model.

[0065] (3) Complete the reasoning transfer training in the two-stage knowledge distillation. Figure 4 As shown in the figure, the internal logical relationships of key modules such as triple construction, misjudgment sample screening, reasoning path generation, and multi-task training have been further refined. For misjudgment samples, the teacher model is called to generate a complete reasoning path as a supervisory signal for the next round of training. For example, for incorrectly predicted samples, the reasoning path given by the teacher model may be: "upper abdominal pain → common symptoms of Helicobacter pylori infection → breath test → positive test → recommended combination treatment with amoxicillin and clarithromycin", and the student model needs to generate this logical chain simultaneously with the answer output. After training, the Top-1 accuracy rate was further improved to 27.3%, and the output results showed a clear causal structure and logical consistency, significantly enhancing the explanatory power.

[0066] The above examples demonstrate that this application demonstrates significant practical benefits and technological advancements in Helicobacter pylori diagnosis tasks. By constructing structured medical triplet inputs and introducing inference paths as supervisory signals, the student model effectively transfers the teacher model's answering capabilities after the first stage of distillation, improving the Top-1 accuracy by 3.8 percentage points. Further enhancing the model's causal inference capabilities through inference transfer training further increases the accuracy to 27.3%, demonstrating the enhanced role of inference chain supervision in model logical consistency.

[0067] In a specific application embodiment, for the comprehensive diagnosis of gastrointestinal diseases, it is expanded to the comprehensive consultation task of multiple diseases, and a Chinese-English parallel question-answering dataset covering twelve common digestive system diseases is constructed, including gastritis, gastric ulcer, indigestion, irritable bowel syndrome and other diseases. The two-stage knowledge reasoning distillation method of this application is used for model training, in which the teacher model is LLaMA2-70B and the student model is the INT8 quantized 6B model. After the first stage of knowledge distillation, the model has a strong cross-disease answering ability and can associate key symptoms with diagnostic suggestions. After introducing reasoning chain training in the second stage of knowledge distillation, the model can output a clearly structured medical logic expression chain, for example: Question: "I feel bloated, have a decreased appetite, and often have acid reflux."

[0068] The student model answered: "It may be gastritis, and a gastroscopy is recommended."

[0069] The medical reasoning path output by the student model: "Stomach bloating + acid reflux + poor appetite → gastric mucosal irritation → common manifestations of gastritis."

[0070] After evaluation, the student model's average accuracy on twelve disease seed tasks increased by 4.6%. It also has the ability to output "symptom combination-examination suggestion-diagnosis conclusion", demonstrating strong generalization and explanation capabilities.

[0071] It can be seen from the above examples that in the scenario of multi-disease consultation for gastrointestinal diseases, this application demonstrates good task generalization capabilities. On a comprehensive question-and-answer dataset covering twelve digestive system diseases, the student model has an average accuracy improvement of 4.6% compared with the traditional distillation method without adding additional parameters, and can output the logical path of "symptom combination-examination suggestion-diagnosis conclusion", which significantly enhances the clinical interpretability of the output. This shows that reasoning chain migration is not only applicable to single disease tasks, but also can improve the stability and comprehension ability of the student model in multi-disease and multi-language mixed input scenarios.

[0072] In a specific application embodiment, for the edge deployment and operation performance of the model, this application selects the edge computing platform as the deployment target, and deploys and tests the trained student model. A lightweight reasoning framework and a local cache mechanism are used, combined with a Web API interface, to achieve offline medical question-answering capabilities. The deployment results show that the initialization loading time of the student model does not exceed 6 seconds, and the average reasoning time for a single round of question-answering is less than 250 milliseconds, which can support real-time interactive scenarios. Through the embedded front-end interface, the model can not only give standard answers, but also synchronously display the reasoning path, such as: "Symptom input → examination suggestions → disease judgment → treatment suggestions", and doctors can manually verify and edit the model reasoning process to further enhance security and credibility. In addition, this application supports operation without Internet access and is suitable for low-resource environments such as remote areas, community clinics, and mobile medical terminals. It has significant advantages such as flexible deployment, efficient reasoning, fast response, and transparent structure.

[0073] Through the above examples, it can be seen that in terms of edge deployment, the number of student model parameters obtained by training in this application is about one-fifth of that of the teacher model, with high compressibility and execution efficiency. After deployment on an 8GB video memory terminal, the model loading time is less than 6 seconds, and the inference response time is less than 250 milliseconds, meeting the dual requirements of mobile medical care and remote consultation for low latency and offline operation. At the same time, by displaying the reasoning path of the model in real time through the front-end system, doctors can review and intervene in the model's diagnostic logic, further improving the credibility and practical usability of the medical question-and-answer system.

[0074] In summary, this application is suitable for medical question-answering model compression and reasoning enhancement in edge deployment scenarios. The proposed two-stage reasoning distillation method achieves the organic unity of knowledge compression, logic transfer and explainable output while maintaining high generation quality. It is significantly better than the existing simple output distillation method or lightweight question-answering model, and has important engineering practical value and promotion prospects.

[0075] The two-stage structure of this application not only forms a progressive enhancement mechanism in the technical path, but also builds a three-dimensional output capability of "answer + reasoning + error correction" at the model behavior level, which is significantly better than the traditional question-answering system that only outputs answers. This application significantly improves the accuracy, interpretability score and misjudgment correction ability of small models in multiple medical question-answering tasks (such as disease diagnosis, symptom judgment, and drug recommendation). In particular, when deployed on an 8GB video memory edge device, the model loading speed is more than 30% faster than the traditional BERT baseline, and the average response time of a single round of reasoning is less than 250 milliseconds, which fully meets the timeliness requirements of primary diagnosis and treatment and mobile medical scenarios.

[0076] Especially in the context of pursuing lightweight and end-side deployment, there is an urgent need for a technical path that can take into account both performance compression and logical reasoning migration. This application is based on this and solves the key technical problems of existing medical question-answering systems, such as large models, lack of logical explainability, high misjudgment rate, and inability to transfer reasoning capabilities through distillation. By constructing a structured knowledge distillation mechanism and an enhanced mechanism for reasoning chain migration, it not only achieves efficient compression and migration of large-model medical knowledge, but also significantly improves the logical consistency, self-correction ability, and deployment adaptability of student models in medical question-answering scenarios, providing a practical and feasible technical path for the application of intelligent medical question-answering systems in edge terminals.

[0077] Based on the same inventive concept, the present application embodiment also provides a two-stage knowledge reasoning distillation system for medical question-answering tasks for implementing the above method. The implementation scheme for solving the problem provided by this system is similar to the implementation scheme described in the above method, so the specific limitations of one or more system embodiments provided below can be found in the above method limitations and will not be repeated here. The system of this application includes: The model loading module is used to deploy the student model that has completed secondary training based on the two-stage knowledge reasoning distillation method for medical question-answering tasks to the edge computing platform. Specifically, the student model that has completed secondary training for medical question-answering can output standardized medical answers and corresponding medical reasoning paths simultaneously when receiving natural language question inputs, and has the following capabilities: automatically determine whether the input needs to be supplemented by a reasoning chain; for inputs that are ambiguous, redundant, or semantically missing, it can generate a clearly structured and logically complete reasoning chain; the output results include a combination of "answer + reasoning process" to facilitate doctor review and verification. Specifically, the loaded trained student model can be initialized on a GPU, NPU, or embedded device.

[0078] The question-answering processing module is used to: after receiving the natural language question information input by the user, call the student model that has completed secondary training to generate medical answers and corresponding medical reasoning paths.

[0079] The reasoning path display module is used to: display the medical reasoning path in a structured or visual form, support doctors to make quick judgments, and receive feedback on error samples.

[0080] The error correction and update module is used to: regularly collect the error samples and misjudgment samples, upload the error samples and misjudgment samples to the server, and re-execute the steps of the above-mentioned two-stage knowledge reasoning and distillation method for medical question-answering tasks to achieve incremental model learning or remote update and redeployment.

[0081] Among them, the misjudgment samples are obtained by verifying the student model that has completed one training based on the verification set after the first stage of knowledge distillation. All misjudgment samples constitute the misjudgment sample set, which reflects the low-confidence samples that appear during operation.

[0082] In a specific application embodiment, the present application also provides a computer-readable storage medium on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-mentioned two-stage knowledge reasoning distillation method for medical question-answering tasks.

[0083] In summary, this application provides a method for compressing and enhancing medical question-answering models in resource-constrained environments. This method involves two phases: structured knowledge distillation and inference chain migration. This strategy compresses model parameters by over 80% while ensuring a diagnostic accuracy of ≥90% for multiple diseases. It also explicitly outputs a traceable medical inference chain, significantly improving interpretability and robustness. This application can run in real time on edge devices with 8GB of video memory (single inference latency <250ms), meeting the application needs of primary care and mobile terminals.

[0084] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with relevant regulations.

[0085] Those skilled in the art will appreciate that all or part of the processes in the above-mentioned embodiments can be implemented by instructing the relevant hardware through a computer program. The computer program can be stored in a non-volatile computer-readable storage medium. When the computer program is executed, it can include the processes of the above-mentioned embodiments. In particular, any reference to memory, database, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM may be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM).

[0086] The databases involved in the various embodiments provided herein may include at least one of a relational database and a non-relational database. Non-relational databases may include, but are not limited to, distributed databases based on blockchains. The processors involved in the various embodiments provided herein may include, but are not limited to, general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic units, data processing logic units based on quantum computing, and the like.

[0087] The technical features of the above embodiments can be combined arbitrarily. To make the description concise, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0088] This document uses specific examples to illustrate the principles and implementation methods of this application. The description of the above examples is only intended to help understand the method and core concept of this application. At the same time, for those skilled in the art, based on the concept of this application, there may be changes in the specific implementation methods and application scope. In summary, the content of this specification should not be understood as limiting this application.

Claims

1. A two-stage knowledge reasoning distillation method for medical question answering tasks, characterized by: The method comprises: Get the pre-trained teacher model; The teacher model is used to perform data parsing and structural processing on the original medical question-answering dataset to obtain multiple medical knowledge triples and divide them into a training set and a validation set; The training set is input into the student model for a first-stage knowledge distillation to guide the student model to learn the knowledge expression method and answering ability of the teacher model in the medical question-answering task, thereby obtaining a student model that has been trained once; based on the validation set, the student model that has been trained once is validated to obtain misclassified samples; all the misclassified samples constitute a misclassified sample set; Using the teacher model to perform data analysis and reasoning on the misjudged sample set to obtain a plurality of medical reasoning quadruplets; the medical reasoning quadruplets include a medical reasoning path from symptoms to diagnosis; Multiple medical reasoning quadruples are used as training data and input into the student model for two-stage knowledge distillation to perform reasoning transfer training on the student model to obtain a student model that has completed secondary training. After the student model has completed secondary training, it is deployed on the edge computing platform and generates medical answers and corresponding medical reasoning paths based on the received natural language consultation information.

2. The two-stage knowledge reasoning distillation method for medical question answering tasks according to claim 1 is characterized in that: One original medical question-and-answer data in the original medical question-and-answer data set includes a patient's natural language question information; The medical knowledge triple is a triplet of question-medical answer-background knowledge; wherein the question is determined based on the natural language medical inquiry information; the medical answer is the medical advice generated by the teacher model after the natural language medical inquiry information is input into the teacher model; and the background knowledge is the medical evidence information related to the natural language medical inquiry information and the medical advice. The medical reasoning quaternary is a quaternary data structure of question-student answer-medical answer-medical reasoning path; wherein, the student answer is a medical answer generated by a student model after one training; the medical reasoning path is a set of medical statements with sequential logical relationships, presenting a causal reasoning link from symptoms to diagnosis.

3. The two-stage knowledge reasoning distillation method for medical question answering tasks according to claim 2 is characterized in that: The method further comprises: During the one-stage knowledge distillation process, a dual loss mechanism is introduced for the training of the student model; wherein, the dual loss mechanism includes language modeling loss and distillation loss; the language modeling loss is used to encourage the student model to learn the ability to generate medical answers from questions and background knowledge to improve the generation quality; the distillation loss is used to encourage the student model to approach the generation distribution of the teacher model in terms of output distribution.

4. The two-stage knowledge reasoning distillation method for medical question answering tasks according to claim 2 is characterized in that: The method further comprises: In the two-stage knowledge distillation process, an inference supervision objective is constructed for the training of the student model, and a multi-task loss function is introduced; The reasoning supervision objective is used to guide the student model to simultaneously generate a corresponding medical reasoning path when generating a medical answer; the multi-task loss function includes language modeling loss, reasoning path generation loss, and error correction supervision loss; The language modeling loss is used to measure the coherence and accuracy of the text generated by the student model; the reasoning path generation loss is used to measure the consistency between the medical reasoning path generated by the student model and the output of the teacher model; the error correction supervision loss is used to impose penalties or rewards on the improvement of the misjudgment samples of the student model during the first stage of knowledge distillation, guiding the repair of misjudgment patterns.

5. The two-stage knowledge reasoning distillation method for medical question answering tasks according to claim 4 is characterized in that: The multi-task loss function is: total loss = α × language modeling loss + β × reasoning path generation loss + γ × error correction supervision loss; Among them, α, β, and γ are hyperparameters that control the weights of different tasks, satisfying α+β+γ=1.

6. The two-stage knowledge reasoning distillation method for medical question answering tasks according to claim 1 is characterized in that: Based on the validation set, the student model that has completed training is validated to obtain misclassified samples, including: Input the medical knowledge triples in the validation set into the trained student model to obtain the model-predicted medical answers; Calculating similarity between the medical answer predicted by the model and the medical answer in the corresponding medical knowledge triplet; When the similarity meets a preset condition, the medical knowledge triplet is marked as a misjudgment sample.

7. The two-stage knowledge reasoning distillation method for medical question answering tasks according to claim 2, characterized in that: The background knowledge includes disease definition, pathogenesis, diagnostic guidelines, examination recommendations, and treatment methods.

8. The two-stage knowledge reasoning distillation method for medical question answering tasks according to claim 1 is characterized in that: The teacher model and the student model are both large language models; the ratio of the number of parameters of the student model to the number of parameters of the teacher model is lower than a preset percentage; The model used by the teacher model is one of Qwen-72B, LLaMA2-70B, and ChatGLM3-130B; the model used by the student model is one of LLaMA2-7B and Qwen-7B; The edge computing platform is an edge server, an embedded diagnosis and treatment tool or a mobile medical device.

9. A two-stage knowledge reasoning distillation system for medical question answering tasks, characterized by: The system comprises: A model loading module, configured to deploy a student model obtained through secondary training using the two-stage knowledge reasoning and distillation method for medical question-answering tasks according to any one of claims 1 to 8 to an edge computing platform; The question-answering processing module is used to: after receiving the natural language question information input by the user, call the student model that has completed the secondary training to generate the medical answer and the corresponding medical reasoning path; A reasoning path display module is used to: display the medical reasoning path in a structured or visual form and receive feedback error samples; The error correction update module is used to: regularly collect the error samples and misjudgment samples, upload the error samples and the misjudgment samples to the server, so as to re-execute the steps of the two-stage knowledge reasoning distillation method for medical question-answering tasks described in any one of claims 1-8, and realize model incremental learning or remote update and redeployment.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, it implements the steps of the two-stage knowledge reasoning distillation method for medical question answering tasks as described in any one of claims 1 to 8.

Images