Heart failure auxiliary diagnosis and treatment knowledge distillation method and system

Abstract

The invention relates to the technical field of heart failure auxiliary diagnosis and treatment, in particular to a heart failure auxiliary diagnosis and treatment knowledge distillation method and system.The knowledge distillation method comprises the steps that an age-crossing fusion data set is built by combining ancient medical case text data related to traditional Chinese medicine heart failure; performing field fine tuning on the pre-training model to generate a basic teacher model; performing semantic alignment, and outputting a cross-time contrast database; constructing a curative effect scoring model by using the time sequence model, and outputting a quantitative association database; using a graph neural network to output an intelligent heart failure knowledge graph, and generating a comprehensive teacher model; performing knowledge distillation on the comprehensive teacher model to generate a clinical student model; the knowledge distillation system is applied to the knowledge distillation method, has a dynamic learning ability, can fuse ancient and modern trans-time knowledge and combine traditional Chinese medicine ancient book wisdom to carry out heart failure auxiliary diagnosis and treatment, and can give consideration to model performance and deployment efficiency through knowledge distillation.

Description

Technical Field

[0001] This invention relates to the field of auxiliary diagnosis and treatment technology for heart failure, specifically to a knowledge distillation method and system for auxiliary diagnosis and treatment of heart failure. Background Technology

[0002] Ancient Chinese medical texts contain a wealth of treatment experiences related to heart failure, such as palpitations, edema, and wheezing. However, their language is obscure and their forms are varied, making them difficult to analyze and utilize in a structured manner by existing auxiliary diagnostic and treatment systems. At the same time, modern electronic medical record systems have accumulated a large amount of real clinical case data, reflecting the actual efficacy of Chinese medicine treatments in modern clinical practice. Existing auxiliary diagnostic and treatment systems have failed to achieve a cross-generational integration of the wisdom of ancient Chinese medical texts and modern electronic medical record data.

[0003] Furthermore, in order to improve the accuracy of diagnosis and treatment recommendations, existing auxiliary diagnosis and treatment systems tend to use large language models with a huge number of parameters for training. Such models consume a lot of computing resources and have high inference latency, making them difficult to deploy directly on hospital workstations with limited resources and unable to meet the needs of clinical scenarios for real-time auxiliary decision-making. Moreover, the knowledge sources of existing auxiliary diagnosis and treatment systems mostly rely on structured clinical guidelines, textbooks or limited expert experience bases. Once the system is built, it tends to become rigid and it is difficult to evolve itself based on continuously generated new medical record data, resulting in a lack of dynamic learning ability.

[0004] Therefore, there is an urgent need for a knowledge distillation method and system for heart failure auxiliary diagnosis and treatment that can integrate knowledge from ancient and modern times, balance model performance and deployment efficiency, and have dynamic learning capabilities. Summary of the Invention

[0005] To address the aforementioned problems, this invention provides a knowledge distillation method and system for assisting in the diagnosis and treatment of heart failure, thereby resolving the technical issues raised in the background section.

[0006] The technical solution adopted by this invention to solve its technical problem is as follows:

[0007] A method for distilling knowledge for the auxiliary diagnosis and treatment of heart failure includes the following steps:

[0008] S1: Process and combine ancient medical case texts and modern electronic medical record data related to heart failure in traditional Chinese medicine to construct a cross-era fusion dataset;

[0009] S2: Using a general large language model as a pre-trained model, introducing a semantic compression algorithm, and fine-tuning the pre-trained model based on a cross-generational fusion dataset to generate a basic teacher model;

[0010] S3: Using a basic teacher model, extract diagnostic and treatment knowledge units centered on "syndrome type-pathogenesis-treatment method-prescription" from a cross-era fusion dataset, and semantically align ancient medical case text data with modern electronic medical record data to output a cross-era comparative database;

[0011] S4: Extract follow-up data from modern electronic medical record data, construct a efficacy scoring model using a time series model, and output a quantitative correlation database between treatment plans and clinical efficacy.

[0012] S5: Based on diagnostic and treatment knowledge units, cross-generational comparative databases, and quantitative correlation databases, construct triples, use graph neural networks to output intelligent heart failure knowledge graphs, and generate comprehensive teacher models;

[0013] S6: Perform knowledge distillation on the integrated teacher model to generate a clinical student model, and use the clinical student model for auxiliary diagnosis and treatment of heart failure in clinical practice.

[0014] Furthermore, the specific steps of S1 include:

[0015] S11: Using optical character recognition technology, professional proofreading, and simplified / traditional Chinese character conversion to process ancient medical case texts and construct a traditional Chinese medicine corpus;

[0016] S12: Extract structured medical record text related to the diagnosis of heart failure from modern electronic medical record data;

[0017] S13: Standardize the terminology in the TCM corpus and structured medical record texts;

[0018] S14: A medical-specific named entity recognition model is used to annotate the standardized TCM corpus and structured medical record text with corresponding associated tags for different terms.

[0019] S15: Merge the annotated TCM corpus and structured medical record texts, and based on the association tags, transform them into a readable cross-generational fusion dataset.

[0020] Furthermore, the specific steps for generating the basic teacher model described in S2 are as follows: using the LoRA algorithm as a semantic compression algorithm to freeze the parameters in the pre-trained model that are not related to heart failure knowledge; using the cross-generational fusion dataset as a training set to train the parameters in the pre-trained model that are related to heart failure knowledge to generate the basic teacher model.

[0021] Furthermore, the specific steps of semantic alignment described in S3 are as follows:

[0022] S31: Using the basic teacher model, Transformer encoding is performed on different texts in ancient medical record data and modern electronic medical record data to construct semantic vectors;

[0023] S32: Calculate the similarity between semantic vectors and initially screen out semantic vectors with high similarity as candidate matching pairs;

[0024] S33: Use distributional consistency statistical rules to perform auxiliary verification on candidate matching pairs;

[0025] S34: The candidate matching pairs after auxiliary verification are used to form a cross-generational comparison database and output.

[0026] Furthermore, the specific steps of S4 are as follows:

[0027] S41: Extract follow-up data from modern electronic medical record data and extract multidimensional time-series data of efficacy related to efficacy evaluation;

[0028] S42: Use an LSTM model to track the multidimensional time-series data of efficacy for each medical record and generate efficacy feature vectors;

[0029] S43: Establish a efficacy scoring standard that includes subjective and objective indicators, and determine the efficacy level corresponding to the efficacy feature vector;

[0030] S44: Output therapeutic effect classification training samples containing treatment plan, therapeutic effect feature vector and therapeutic effect level, and use the therapeutic effect level as the label of the therapeutic effect classification training samples.

[0031] S45: A clinical efficacy classifier is constructed using a machine learning classification model. The classifier is trained using efficacy classification training samples and outputs a quantitative correlation database between treatment plans and clinical efficacy.

[0032] Furthermore, the specific steps of S5 are as follows:

[0033] S51: Obtain the correlation between syndrome types and treatment plans from diagnostic and treatment knowledge units and cross-generational comparative databases, obtain the correlation between treatment plans and clinical efficacy from quantitative correlation databases, and construct a "syndrome type-treatment plan-clinical efficacy" ternary set;

[0034] S52: Use a graph database to store triples and generate an initial heart failure knowledge graph;

[0035] S53: Input new heart failure patient medical records with actual clinical efficacy results, and verify the efficacy deviation between the actual clinical efficacy results and the clinical efficacy predicted by the initial heart failure knowledge graph;

[0036] S54: Using a graph neural network, based on the efficacy bias, the edge weights of the triples are iteratively adjusted to generate an intelligent heart failure knowledge graph.

[0037] S55: Integrate the intelligent heart failure knowledge graph with the basic teacher model to generate a comprehensive teacher model.

[0038] Furthermore, the specific steps of S6 are as follows:

[0039] S61: Design and build a lightweight neural network model as a base student model;

[0040] S62: A knowledge distillation training set is hierarchically sampled from a cross-generational fusion dataset, and a comprehensive teacher model is used to generate guidance information for each training sample in the knowledge distillation training set;

[0041] S63: Using guidance information as the training objective, a composite loss function is used to train the basic student model and generate a clinical student model;

[0042] S64: Deploy clinical student models in real-world clinical environments.

[0043] Furthermore, the guidance information described in S62 includes soft labels, hard labels, intermediate layer semantic features, and graph edge weight information in the intelligent heart failure knowledge graph; for each training sample in the knowledge distillation training set, the soft label contains the probability distribution of all possible treatment options for that training sample, the hard label is the final treatment option output by the integrated teacher model, the intermediate layer semantic features include the feature vectors generated by the integrated teacher model at different levels of the internal neural network, and the graph edge weight information includes the edge weights of strongly correlated triplets as perceived by the integrated teacher model.

[0044] Furthermore, the composite loss function described in S63 includes distillation loss, supervision loss, and internal regression loss. The training objective of distillation loss is to make the probability distribution output by the base student model close to the soft label. The training objective of supervision loss is to make the final answer output by the base student model close to the hard label. The internal regression loss is to make the internal output of the base student model close to the semantic features of the intermediate layer and the graph edge weight information.

[0045] A knowledge distillation system for heart failure auxiliary diagnosis and treatment, applied to the aforementioned knowledge distillation method for heart failure auxiliary diagnosis and treatment, includes a teacher model construction backend and a student model clinical frontend. The teacher model construction backend includes a dataset construction layer, a domain-wide large model training layer, a cross-generational knowledge fusion layer, and a knowledge graph construction layer. The student model clinical frontend includes a knowledge distillation layer, a reasoning and decision-making layer, and a human-computer interaction layer.

[0046] Compared with the prior art, the beneficial effects of the present invention are:

[0047] 1. The present invention provides a knowledge distillation method for the auxiliary diagnosis and treatment of heart failure. By fine-tuning the large language model and semantic alignment in the domain, it solves the problem of aligning the obscure syndrome descriptions in ancient Chinese medicine books with modern electronic medical record data, and realizes the cross-era deep integration and utilization of medical knowledge from ancient and modern times.

[0048] 2. The present invention provides a knowledge distillation method for heart failure auxiliary diagnosis and treatment, which generates an intelligent heart failure knowledge graph that can continuously self-iterate by continuously feeding in new clinical cases and dynamically adjusting the edge weights of triples using a graph neural network, thereby enabling the heart failure auxiliary diagnosis and treatment system to have dynamic learning capabilities.

[0049] 3. The knowledge distillation system for heart failure auxiliary diagnosis and treatment provided by the present invention first builds a powerful comprehensive teacher model in the background, and then transfers its core capabilities to a lightweight clinical student model deployed in the front end through knowledge distillation, which reduces the computing and storage costs of clinical terminal devices and enables rapid response to clinical requests and real-time auxiliary decision-making. Attached Figure Description

[0050] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, for those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0051] Figure 1 This is a schematic diagram of the knowledge distillation method for auxiliary diagnosis and treatment of heart failure as described in this invention;

[0052] Figure 2 for Figure 1 A flowchart of S1 in the middle;

[0053] Figure 3 for Figure 1 A flowchart of the S3 process;

[0054] Figure 4 for Figure 1 A flowchart of the S4 process;

[0055] Figure 5 for Figure 1 A flowchart of the S5 process;

[0056] Figure 6 for Figure 1 A flowchart of the S6 process;

[0057] Figure 7 This is a schematic diagram of the knowledge distillation system for auxiliary diagnosis and treatment of heart failure as described in this invention. Detailed Implementation

[0058] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0059] This invention provides a method for distilling knowledge for the auxiliary diagnosis and treatment of heart failure, such as... Figures 1-6 As shown, the method includes the following steps:

[0060] S1: Process and combine ancient medical case texts and modern electronic medical record data related to heart failure in traditional Chinese medicine to construct a cross-era fusion dataset;

[0061] Furthermore, the specific steps of S1 include:

[0062] S11: Using optical character recognition technology, professional proofreading, and simplified / traditional Chinese character conversion to process ancient medical case texts and construct a traditional Chinese medicine corpus;

[0063] Among them, the ancient medical case text data includes authoritative ancient books such as "Huangdi Neijing", "Jinkui Yaolue", "Wenbing Tiaobian", and "Yizong Jinjian", as well as modern medical case collections.

[0064] S12: Extract structured medical record text related to the diagnosis of heart failure from modern electronic medical record data;

[0065] The structured medical record text includes diagnosis, syndrome type, prescription, follow-up, efficacy indicators, etc.

[0066] S13: Standardize the terminology in the TCM corpus and structured medical record texts;

[0067] The standardization of terminology is based on a dictionary specifically for Traditional Chinese Medicine (TCM). This dictionary is constructed by integrating authoritative classics, clinical guidelines, and expert knowledge, and includes standard terms, entity categories, and synonym relationships.

[0068] S14: A medical-specific named entity recognition model is used to annotate the standardized TCM corpus and structured medical record text with corresponding associated tags for different terms.

[0069] The medical proprietary named entity recognition model is a pre-trained model based on Transformer or recurrent neural network architecture.

[0070] S15: Merge the annotated TCM corpus and structured medical record texts, and based on the association tags, transform them into a readable cross-generational fusion dataset.

[0071] S2: Using a general large language model as a pre-trained model, introducing a semantic compression algorithm, and fine-tuning the pre-trained model based on a cross-generational fusion dataset to generate a basic teacher model;

[0072] Furthermore, the specific steps for generating the basic teacher model are as follows: using the LoRA algorithm as a semantic compression algorithm to freeze most of the parameters in the pre-trained model that are unrelated to heart failure knowledge; using a cross-generational fusion dataset as a training set to train a small portion of the parameters in the pre-trained model that are related to heart failure knowledge, thereby generating the basic teacher model.

[0073] Through domain fine-tuning, the pre-trained model can access and learn proprietary knowledge in the field of traditional Chinese medicine, thereby improving its understanding of TCM terminology; through semantic compression algorithms, the number of parameters that need to be trained is greatly reduced, thereby reducing memory usage and training time.

[0074] S3: Using a basic teacher model, extract diagnostic and treatment knowledge units centered on "syndrome type-pathogenesis-treatment method-prescription" from a cross-era fusion dataset, and semantically align ancient medical case text data with modern electronic medical record data to output a cross-era comparative database;

[0075] Among them, by semantic alignment, a cross-era comparative database is constructed, combining ancient Chinese medicine books with modern real-world evidence to form a more comprehensive, three-dimensional, and inherited knowledge system of heart failure in Chinese medicine.

[0076] Furthermore, the basic teacher model automatically identifies named entities from the cross-generational fusion dataset and determines the relationships between named entities, ultimately extracting the diagnostic and treatment knowledge units. The diagnostic and treatment knowledge units are centered on "syndrome type - pathogenesis - treatment method - prescription". For example, for heart failure complicated with edema and shortness of breath, the syndrome type in its diagnostic and treatment knowledge unit is heart and kidney yang deficiency syndrome, the pathogenesis is insufficient heart yang, weak peristalsis, kidney yang deficiency, water retention, upward overflow as shortness of breath, and downward overflow as edema, the treatment method is to warm and tonify the heart and kidney, replenish qi and promote diuresis, and the prescription is Zhenwu Decoction combined with Ginseng and Aconite Decoction with modifications.

[0077] Furthermore, the specific steps of semantic alignment described in S3 are as follows:

[0078] S31: Using the basic teacher model, Transformer encoding is performed on different texts in ancient medical record data and modern electronic medical record data to construct semantic vectors;

[0079] The Transformer encoding work is completed using the existing TCM BERT / RoBERTa architecture. The semantic vectors constructed by Transformer encoding can transform the messy and ambiguous ancient medical case texts and modern electronic medical record data into mathematical vectors that the system can understand and that are rich in contextual information.

[0080] S32: Calculate the similarity between semantic vectors and initially screen out semantic vectors with high similarity as candidate matching pairs;

[0081] In this embodiment, the similarity calculation method adopts the commonly used cosine similarity. The semantic vector of an ancient medical record text and the semantic vector of a modern electronic medical record are extracted, and the similarity score of the two semantic vectors is calculated. The similarity score between the semantic vectors of different ancient medical record texts and modern electronic medical record data is calculated, and candidate matching pairs with high similarity are selected from them.

[0082] S33: Use distributional consistency statistical rules to perform auxiliary verification on candidate matching pairs;

[0083] The statistical rules for distribution consistency include clinical manifestation co-occurrence rules and treatment regimen relevance rules. The clinical manifestation co-occurrence rules specifically involve statistically analyzing the co-occurrence frequency distribution of clinical manifestations associated with ancient and modern terms in the candidate matching pairs. Clinical manifestations include symptoms and signs. When the similarity between the two frequency distributions is higher than a preset first threshold, the verification passes. The treatment regimen relevance rules specifically involve statistically analyzing the usage frequency distribution of treatment regimens associated with ancient and modern terms in the candidate matching pairs. Treatment regimens include prescriptions and drugs. When the overlap between two treatment regimen sets in terms of name, core ingredients, or pharmacological classification is higher than a preset second threshold, the verification passes.

[0084] S34: The candidate matching pairs after auxiliary verification are used to form a cross-generational comparison database and output.

[0085] The cross-era comparative database contains alignment relationships between ancient medical record texts and modern electronic medical record data in terms of semantics such as disease names, syndrome types, treatment plans, and efficacy. Some alignment relationships are shown in Table 1.

[0086] Table 1. Alignment of ancient and modern semantic parts in cross-era comparative databases

[0087]

[0088] S4: Extract follow-up data from modern electronic medical record data, construct a efficacy scoring model using a time series model, and output a quantitative correlation database between treatment plans and clinical efficacy.

[0089] Furthermore, the specific steps of S4 are as follows:

[0090] S41: Extract follow-up data from modern electronic medical record data and extract multidimensional time-series data of efficacy related to efficacy evaluation;

[0091] The efficacy multidimensional time-series data is extracted from multiple follow-up records of patients in the follow-up data. It includes various status indicators that can reflect the patient's status. The status indicators include symptom changes, laboratory indicators, tongue and pulse observation, subjective scores, etc. The time point of each follow-up record is used as the timestamp of the efficacy multidimensional time-series data.

[0092] S42: Use an LSTM model to track the multidimensional time-series data of efficacy for each medical record and generate efficacy feature vectors;

[0093] Each efficacy feature vector reflects the evolution of treatment effect during the treatment process of the patient in that case.

[0094] S43: Establish a efficacy scoring standard that includes subjective and objective indicators, and determine the efficacy level corresponding to the efficacy feature vector;

[0095] The subjective indicators are obtained from patient self-report and physician observation, including the degree of symptom improvement, quality of life score, tongue and pulse changes, and subjective comfort score. The degree of symptom improvement includes relief of shortness of breath, palpitations, fatigue, and chest tightness. The quality of life score is obtained through scales such as SF-36 and MLHFQ. The tongue and pulse changes include changes in tongue color, coating texture, and pulse pattern in the four diagnostic methods of traditional Chinese medicine. The subjective comfort score is obtained by patient self-assessment. The objective indicators include laboratory indicators, cardiac function classification indicators, exercise tolerance quantitative indicators, and the number of hospitalizations / readmissions during the follow-up period. The laboratory indicators include changes in blood routine and electrolyte levels. The cardiac function classification indicators include changes in ventricular wall thickness. The exercise tolerance quantitative indicators include the 6-minute walking distance. In this embodiment, subjective indicators account for 30%, and objective indicators account for 70%. The efficacy level includes four levels: "significantly effective," "effective," "ineffective," and "worsened."

[0096] S44: Output therapeutic effect classification training samples containing treatment plan, therapeutic effect feature vector and therapeutic effect level, and use the therapeutic effect level as the label of the therapeutic effect classification training samples.

[0097] S45: A clinical efficacy classifier is constructed using a machine learning classification model. The classifier is trained using efficacy classification training samples and outputs a quantitative correlation database between treatment plans and clinical efficacy.

[0098] In this embodiment, the machine learning classification model is selected from random forest model or simple neural network model.

[0099] S5: Based on diagnostic and treatment knowledge units, cross-generational comparative databases, and quantitative correlation databases, construct triples, use graph neural networks to output intelligent heart failure knowledge graphs, and generate comprehensive teacher models;

[0100] Furthermore, the specific steps of S5 are as follows:

[0101] S51: Obtain the correlation between syndrome types and treatment plans from diagnostic and treatment knowledge units and cross-generational comparative databases, obtain the correlation between treatment plans and clinical efficacy from quantitative correlation databases, and construct a "syndrome type-treatment plan-clinical efficacy" ternary set;

[0102] The “syndrome type-treatment plan-clinical efficacy” ternary set is the basic building block of the heart failure knowledge graph. Syndrome type, treatment plan, and clinical efficacy are the subject node, relation edge, and object node in the ternary set, respectively.

[0103] S52: Use a graph database to store triples and generate an initial heart failure knowledge graph;

[0104] The graph database uses an existing general graph database. All triples are imported into the graph database in batches. The database automatically generates a queryable and manageable initial heart failure knowledge graph. The initial heart failure knowledge graph is a static knowledge graph.

[0105] S53: Input new heart failure patient medical records with actual clinical efficacy results, and verify the efficacy deviation between the actual clinical efficacy results and the clinical efficacy predicted by the initial heart failure knowledge graph;

[0106] S54: Using a graph neural network, based on the efficacy bias, the edge weights of the triples are iteratively adjusted to generate an intelligent heart failure knowledge graph.

[0107] Specifically, for large efficacy deviations, the graph neural network reduces the edge weights of the triples, while for small efficacy deviations, it increases the edge weights of the triples. By continuously adjusting the edge weights of the triples, the accuracy and timeliness of the heart failure knowledge graph will continue to improve as more new medical records are added, thereby transforming the static initial heart failure knowledge graph into an intelligent heart failure knowledge graph that can self-evolve.

[0108] S55: Integrate the intelligent heart failure knowledge graph with the basic teacher model to generate a comprehensive teacher model.

[0109] The generated integrated teacher model possesses both the language comprehension ability of the basic teacher model in the field of traditional Chinese medicine and the reasoning and dynamic learning ability of the intelligent heart failure map, enabling it to complete the auxiliary diagnosis and treatment of heart failure based on cross-generational integration.

[0110] S6: Perform knowledge distillation on the integrated teacher model to generate a clinical student model, and use the clinical student model for auxiliary diagnosis and treatment of heart failure in clinical practice.

[0111] Furthermore, the specific steps of S6 are as follows:

[0112] S61: Design and build a lightweight neural network model as a base student model;

[0113] The basic student model has fewer parameters, shallower network layers, and a simpler structure than the integrated teacher model, which is beneficial for deployment in hospital terminals.

[0114] S62: A knowledge distillation training set is hierarchically sampled from a cross-generational fusion dataset, and a comprehensive teacher model is used to generate guidance information for each training sample in the knowledge distillation training set;

[0115] The guidance information includes soft labels, hard labels, intermediate layer semantic features, and graph edge weight information in the intelligent heart failure knowledge graph. For each training sample in the knowledge distillation training set, the soft label contains the probability distribution of all possible treatment options for that training sample, the hard label is the final treatment option output by the integrated teacher model, the intermediate layer semantic features include the feature vectors generated by the integrated teacher model at different levels of the internal neural network, and the graph edge weight information includes the edge weights of strongly correlated triplets as perceived by the integrated teacher model.

[0116] S63: Using guidance information as the training objective, a composite loss function is used to train the basic student model and generate a clinical student model;

[0117] The composite loss function includes distillation loss, supervision loss, and internal regression loss. The training objective of distillation loss is to make the probability distribution output by the basic student model close to the soft label; in this embodiment, KL divergence is used to calculate the distillation loss. The training objective of supervision loss is to make the final answer output by the basic student model close to the hard label; in this embodiment, cross-entropy is used to calculate the distillation loss. The internal regression loss is to make the internal output of the basic student model close to the semantic features of the intermediate layer and the edge weight information of the graph. During the training phase, the smoothness of the probability distribution is adjusted by the temperature parameter to ensure that the student model inherits the diagnostic and treatment knowledge of the teacher model while significantly reducing the computation and storage costs, thereby achieving efficient and dynamically updatable heart failure auxiliary diagnosis and treatment decision support.

[0118] S64: Deploy clinical student models in real-world clinical environments.

[0119] The deployed clinical student model combines the accuracy and rapid reasoning capabilities of the integrated teacher model, has lower requirements for hardware facilities, and is more suitable for actual clinical environments.

[0120] This invention also provides a knowledge distillation system for assisting in the diagnosis and treatment of heart failure, and applies the aforementioned knowledge distillation method for assisting in the diagnosis and treatment of heart failure, such as... Figure 7 As shown, it includes the teacher model building backend and the student model clinical frontend.

[0121] Furthermore, the teacher model construction backend includes a dataset construction layer, a domain-wide large model training layer, a cross-era knowledge fusion layer, and a knowledge graph construction layer. The dataset construction layer includes a traditional Chinese medicine corpus module, an electronic medical record access module, and a text annotation module, used to combine ancient medical case text data and modern electronic medical record data to construct a cross-era fused dataset. The domain-wide large model training layer includes a large model loading module and a domain fine-tuning module, used to generate a basic teacher model using a general large language model. The cross-era knowledge fusion layer includes a knowledge unit extraction module, a semantic alignment module, and a efficacy scoring module, used to extract diagnostic and treatment knowledge units, output a cross-era comparative database through semantic alignment, and output a quantitative correlation database through clinical efficacy scoring. The knowledge graph construction layer includes a triplet construction module, a graph database management module, and a graph optimization module, used to output an intelligent heart failure knowledge graph and generate a comprehensive teacher model.

[0122] Furthermore, the clinical front-end of the student model includes a knowledge distillation layer, a reasoning and decision-making layer, and a human-computer interaction layer. The knowledge distillation layer is connected to the teacher model construction back-end through an interface and includes a basic student model construction module and a distillation training module for generating clinical student models. The reasoning and decision-making layer is used to run the clinical student model in a clinical environment and output heart failure auxiliary diagnosis and treatment suggestions. The human-computer interaction layer is used for doctors to interact with the system.

[0123] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention, the scope of which is defined by the appended claims and their equivalents.

Claims

1. A method for distilling knowledge for the auxiliary diagnosis and treatment of heart failure, characterized in that, Includes the following steps: S1: Process and combine ancient medical case texts and modern electronic medical record data related to heart failure in traditional Chinese medicine to construct a cross-era fusion dataset; S2: Using a general large language model as a pre-trained model, introducing a semantic compression algorithm, and fine-tuning the pre-trained model based on a cross-generational fusion dataset to generate a basic teacher model; S3: Using a basic teacher model, extract diagnostic and treatment knowledge units centered on "syndrome type-pathogenesis-treatment method-prescription" from a cross-era fusion dataset, and semantically align ancient medical case text data with modern electronic medical record data to output a cross-era comparative database; S4: Extract follow-up data from modern electronic medical record data, construct a efficacy scoring model using a time series model, and output a quantitative correlation database between treatment plans and clinical efficacy. S5: Based on diagnostic and treatment knowledge units, cross-generational comparative databases, and quantitative correlation databases, construct triples, use graph neural networks to output intelligent heart failure knowledge graphs, and generate comprehensive teacher models; S6: Perform knowledge distillation on the integrated teacher model to generate a clinical student model, and use the clinical student model for auxiliary diagnosis and treatment of heart failure in clinical practice.

2. The method for knowledge distillation in the auxiliary diagnosis and treatment of heart failure according to claim 1, characterized in that, The specific steps of S1 include: S11: Using optical character recognition technology, professional proofreading, and simplified / traditional Chinese character conversion to process ancient medical case texts and construct a traditional Chinese medicine corpus; S12: Extract structured medical record text related to the diagnosis of heart failure from modern electronic medical record data; S13: Standardize the terminology in the TCM corpus and structured medical record texts; S14: A medical-specific named entity recognition model is used to annotate the standardized TCM corpus and structured medical record text with corresponding associated tags for different terms. S15: Merge the annotated TCM corpus and structured medical record texts, and based on the association tags, transform them into a readable cross-generational fusion dataset.

3. The method for knowledge distillation in the auxiliary diagnosis and treatment of heart failure according to claim 2, characterized in that, The specific steps for generating the basic teacher model described in S2 are as follows: using the LoRA algorithm as the semantic compression algorithm, freezing the parameters in the pre-trained model that are unrelated to heart failure knowledge; using the cross-generational fusion dataset as the training set, training the parameters in the pre-trained model that are related to heart failure knowledge, and generating the basic teacher model.

4. The knowledge distillation method for auxiliary diagnosis and treatment of heart failure according to claim 3, characterized in that, The specific steps for semantic alignment described in S3 are as follows: S31: Using the basic teacher model, perform Transformer encoding on different texts in ancient medical record data and modern electronic medical record data to construct semantic vectors; S32: Calculate the similarity between semantic vectors and initially screen out semantic vectors with high similarity as candidate matching pairs; S33: Use distributional consistency statistical rules to perform auxiliary verification on candidate matching pairs; S34: The candidate matching pairs after auxiliary verification are used to form a cross-generational comparison database and output.

5. The knowledge distillation method for auxiliary diagnosis and treatment of heart failure according to claim 4, characterized in that, The specific steps of S4 are as follows: S41: Extract follow-up data from modern electronic medical record data and extract multidimensional time-series data of efficacy related to efficacy evaluation; S42: Use an LSTM model to track the multidimensional time-series data of efficacy for each medical record and generate efficacy feature vectors; S43: Establish a efficacy scoring standard that includes subjective and objective indicators, and determine the efficacy level corresponding to the efficacy feature vector; S44: Output therapeutic effect classification training samples containing treatment plan, therapeutic effect feature vector and therapeutic effect level, and use the therapeutic effect level as the label of the therapeutic effect classification training samples. S45: A clinical efficacy classifier is constructed using a machine learning classification model. The classifier is trained using efficacy classification training samples and outputs a quantitative correlation database between treatment plans and clinical efficacy.

6. The knowledge distillation method for auxiliary diagnosis and treatment of heart failure according to claim 5, characterized in that, The specific steps of S5 are as follows: S51: Obtain the correlation between syndrome types and treatment plans from diagnostic and treatment knowledge units and cross-era comparative databases, obtain the correlation between treatment plans and clinical efficacy from quantitative correlation databases, and construct a "syndrome type-treatment plan-clinical efficacy" ternary set; S52: Use a graph database to store triples and generate an initial heart failure knowledge graph; S53: Input new heart failure patient medical records with actual clinical efficacy results, and verify the efficacy deviation between the actual clinical efficacy results and the clinical efficacy predicted by the initial heart failure knowledge graph; S54: Using a graph neural network, based on the efficacy bias, the edge weights of the triples are iteratively adjusted to generate an intelligent heart failure knowledge graph. S55: Integrate the intelligent heart failure knowledge graph with the basic teacher model to generate a comprehensive teacher model.

7. The knowledge distillation method for auxiliary diagnosis and treatment of heart failure according to claim 6, characterized in that, The specific steps of S6 are as follows: S61: Design and build a lightweight neural network model as a base student model; S62: A knowledge distillation training set is hierarchically sampled from a cross-generational fusion dataset, and a comprehensive teacher model is used to generate guidance information for each training sample in the knowledge distillation training set; S63: Using guidance information as the training objective, a composite loss function is used to train the basic student model and generate a clinical student model; S64: Deploy clinical student models in real-world clinical environments.

8. The method for distilling knowledge for the auxiliary diagnosis and treatment of heart failure according to claim 7, characterized in that, The guidance information described in S62 includes soft labels, hard labels, intermediate layer semantic features, and graph edge weight information in the intelligent heart failure knowledge graph. For each training sample in the knowledge distillation training set, the soft label contains the probability distribution of all possible treatment options for that training sample, the hard label is the final treatment option output by the integrated teacher model, the intermediate layer semantic features include the feature vectors generated by the integrated teacher model at different levels of the internal neural network, and the graph edge weight information includes the edge weights of strongly correlated triplets as perceived by the integrated teacher model.

9. The method for distilling knowledge for auxiliary diagnosis and treatment of heart failure according to claim 8, characterized in that, The composite loss function described in S63 includes distillation loss, supervised loss, and internal regression loss. The training objective of distillation loss is to make the probability distribution output by the base student model close to the soft label. The training objective of supervised loss is to make the final answer output by the base student model close to the hard label. The internal regression loss is to make the internal output of the base student model close to the semantic features of the intermediate layer and the edge weight information of the graph.

10. A knowledge distillation system for assisting in the diagnosis and treatment of heart failure, applied to the knowledge distillation method for assisting in the diagnosis and treatment of heart failure as described in any one of claims 1-9, characterized in that, It includes a teacher model construction backend and a student model clinical frontend. The teacher model construction backend includes a dataset construction layer, a domain-wide large model training layer, a cross-generational knowledge fusion layer, and a knowledge graph construction layer. The student model clinical frontend includes a knowledge distillation layer, a reasoning and decision-making layer, and a human-computer interaction layer.

Images