Electrocardiogram diagnosis model personalization method and system based on differentiated knowledge distillation

Abstract

The invention provides an electrocardiogram diagnosis model personalization method and system based on differentiated knowledge distillation, and the method comprises the steps: S1, constructing a differentiated data set according to a student model and a teacher model, and carrying out the random sampling of the data set and the replacement of individual electrocardiogram data; s2, performing individualized knowledge extraction according to the differentiated data set, wherein the knowledge extraction comprises a knowledge distillation process, a temperature coefficient and a soft label; and S3, carrying out classification loss calculation and total loss function design. According to the method, through a differentiation knowledge distillation method, the problems of model overfitting and performance reduction caused by sample scarcity in the individualization process of an existing ECG diagnosis model are solved.

Description

Technical Field

[0001] The present invention relates to the fields of deep learning and smart medical care, and specifically, to a method and system for personalizing an electrocardiogram diagnosis model based on differentiated knowledge distillation; more specifically, to a method and system for personalizing a single sample of an electrocardiogram diagnosis model based on differentiated knowledge distillation. Background Art

[0002] In recent years, the application of electrocardiogram (ECG) in the diagnosis of cardiovascular diseases has developed rapidly, but the existing ECG diagnosis models based on deep learning have limitations in dealing with individual differences. Most methods rely on a large amount of individual data for model fine-tuning, which is costly to collect and traditional data augmentation methods do not work well on ECG signals. In addition, when faced with a small amount of individual data, existing technologies are prone to overfitting and it is difficult to effectively extract individual-specific information. Therefore, there is an urgent need for an efficient method that can achieve personalized ECG diagnosis with a small amount of data.

[0003] Patent document CN116153503A discloses a personalized diagnosis method for fetal electrocardiogram based on deep learning, which mainly involves the fields of artificial intelligence and smart medical care. The main steps include: designing a multi-view encoder to fully extract sample features according to the physical morphology of the electrocardiogram; designing source tasks, auxiliary tasks, and target tasks respectively according to the actual situation of fetal electrocardiogram diagnosis; designing a dynamic self-learning weight fusion strategy in the meta-transfer learning framework to enable the model to learn useful transfer knowledge; simulating the actual diagnostic environment, designing two cases, cross-subject and cross-disease, to simulate the monitoring and diagnosis of real-time electrocardiograms. However, this invention is only for fetal electrocardiogram diagnosis. Summary of the invention

[0004] In view of the defects in the prior art, the purpose of the present invention is to provide a method and system for personalizing an electrocardiogram diagnosis model based on differentiated knowledge distillation.

[0005] A method for personalizing an electrocardiogram diagnosis model based on differentiated knowledge distillation provided by the present invention includes:

[0006] Step S1: construct a differentiated data set based on the student model and the teacher model, perform random sampling of the data set and replace individual ECG data;

[0007] Step S2: extracting individualized knowledge based on differentiated data sets, including knowledge distillation process and temperature coefficients and soft labels;

[0008] Step S3: Calculate the classification loss and design the total loss function.

[0009] Preferably, in step S1:

[0010] Differentiated dataset construction, including random sampling of datasets and replacement of individual ECG data; randomly extracting part of the data from the source domain dataset to form a small-scale dataset; using the patient's individual ECG records to replace the data in the small-scale dataset of the same category to construct a differentiated dataset;

[0011] The source domain dataset comes from the PTB-XL database with a sampling frequency of 500 Hz. The labels in the dataset include diagnostic labels, morphological labels, and rhythm labels, which are 5 diagnostic categories or 23 subcategories;

[0012] Five of the diagnostic categories included normal ECG, myocardial infarction, hypertrophy, ST-T changes, and conduction disturbances;

[0013] The 23 subcategories included normal ECG, inferior myocardial infarction, anterior myocardial infarction, ST-T changes, left ventricular hypertrophy, left anterior / posterior fascicular block, ischemic ST-T changes, incomplete right bundle branch block, ischemic ST-T changes in anterior leads, atrioventricular block, intraventricular conduction disorder, nonspecific ST-T changes, complete right bundle branch block, complete left bundle branch block, left atrial overload / left atrial enlargement, ischemic ST-T changes in inferior leads, lateral myocardial infarction, right ventricular hypertrophy, right atrial overload / right atrial enlargement, incomplete left bundle branch block, Wolff-Parkinson-White syndrome, ventricular septal hypertrophy, and posterior myocardial infarction.

[0014] Preferably, in step S2:

[0015] Individualized knowledge extraction unit, including knowledge distillation process and temperature coefficient and soft label; input the target patient sample into the student model, and input the source domain sample with the same label into the teacher model, measure the difference in the probability distribution of the two outputs, so that the student model can learn the individual knowledge of the patient; use temperature scaling and Softmax function to process the logits output by the model;

[0016] Knowledge distillation process: input the target patient samples into the student model, and input the source domain samples with the same labels into the teacher model, measure the difference in the probability distribution of the two outputs, and enable the student model to learn the individual knowledge of the patient; Temperature coefficient and soft label: use temperature scaling and Softmax function to process the logits output by the model;

[0017] The teacher model uses a neural network structure and is trained based on the complete PTB-XL dataset to generate soft labels for classification, i.e., the probability distribution of the model output. The initial structure of the student model is the same as that of the teacher model, and the parameter initialization is not trained. During the knowledge distillation process, the student model receives the source domain data processed by the teacher model and the individual data of the target patients. By comparing the output probability distributions of the two, the student model learns the classification knowledge in the teacher model and incorporates individual features.

[0018] Preferably, the difference in probability distribution of the two outputs is measured by calculating the Kullback-Leibler divergence. The specific calculation process is as follows: given an input sample X, their output logits are obtained through the teacher model and the student model respectively, and the output of the teacher model is obtained after Softmax and temperature scaling:

[0019]

[0020] Among them, σ1 is the Softmax function, is the classification logits probability output by the teacher model for the i-th ECG record, Model teacher (x i ) is the output of the teacher model for the i-th ECG record, x i is the i-th ECG record, x j is the jth ECG record;

[0021] Output of the student model:

[0022]

[0023] in, is the classification logits probability output by the academic model for the i-th ECG record, Model student (X i ) is the output of the student model for the i-th ECG record, X j is the jth ECG record; τ is the temperature coefficient, which is used to smooth the probability distribution of the output. The difference in the probability distribution of the teacher model and the student model output is calculated using the Kullback-Leibler divergence:

[0024]

[0025] Among them, L kl is the loss function calculated using KL divergence.

[0026] Preferably, in step S3:

[0027] The actual labels of the small-scale source domain dataset and the output probability of the student model are calculated through the binary cross entropy function to maintain the classification performance; the loss function is the weighted sum of the distillation loss and the classification loss;

[0028] The classification loss is used to measure the difference between the predicted output of the student model and the true label, and is calculated using the cross entropy loss function; the specific calculation process is as follows:

[0029]

[0030] Among them, L BCE is the loss function calculated using BCE, N is the number of samples; y i is the true label of the sample; is the prediction result of the model; σ2 is the Sigmoid activation function.

[0031] The total loss function l combines the classification loss and the distillation loss, which is a weighted sum of the two parts. The purpose is to guide the student model to learn the characteristics of the teacher model through knowledge distillation while maintaining the classification ability of the model.

[0032] l=α·L C +(1-α)·L D

[0033] Among them, α is the weight coefficient of the two types of losses.

[0034] According to the present invention, a personalized electrocardiogram diagnosis model system based on differentiated knowledge distillation is provided, comprising:

[0035] Module M1: Construct differentiated data sets based on the student model and the teacher model, perform random sampling of data sets, and replace individual ECG data;

[0036] Module M2: Individualized knowledge extraction based on differentiated datasets, including knowledge distillation process and temperature coefficients and soft labels;

[0037] Module M3: Perform classification loss calculation and total loss function design.

[0038] Preferably, in the module M1:

[0039] Differentiated dataset construction, including random sampling of datasets and replacement of individual ECG data; randomly extracting part of the data from the source domain dataset to form a small-scale dataset; using the patient's individual ECG records to replace the data in the small-scale dataset of the same category to construct a differentiated dataset;

[0040] The source domain dataset comes from the PTB-XL database with a sampling frequency of 500 Hz. The labels in the dataset include diagnostic labels, morphological labels, and rhythm labels, which are 5 diagnostic categories or 23 subcategories;

[0041] Five of the diagnostic categories included normal ECG, myocardial infarction, hypertrophy, ST-T changes, and conduction disturbances;

[0042] The 23 subcategories included normal ECG, inferior myocardial infarction, anterior myocardial infarction, ST-T changes, left ventricular hypertrophy, left anterior / posterior fascicular block, ischemic ST-T changes, incomplete right bundle branch block, ischemic ST-T changes in anterior leads, atrioventricular block, intraventricular conduction disorder, nonspecific ST-T changes, complete right bundle branch block, complete left bundle branch block, left atrial overload / left atrial enlargement, ischemic ST-T changes in inferior leads, lateral myocardial infarction, right ventricular hypertrophy, right atrial overload / right atrial enlargement, incomplete left bundle branch block, Wolff-Parkinson-White syndrome, ventricular septal hypertrophy, and posterior myocardial infarction.

[0043] Preferably, in the module M2:

[0044] Individualized knowledge extraction unit, including knowledge distillation process and temperature coefficient and soft label; input the target patient sample into the student model, and input the source domain sample with the same label into the teacher model, measure the difference in the probability distribution of the two outputs, so that the student model can learn the individual knowledge of the patient; use temperature scaling and Softmax function to process the logits output by the model;

[0045] Knowledge distillation process: input the target patient samples into the student model, and input the source domain samples with the same labels into the teacher model, measure the difference in the probability distribution of the two outputs, and enable the student model to learn the individual knowledge of the patient; Temperature coefficient and soft label: use temperature scaling and Softmax function to process the logits output by the model;

[0046] The teacher model uses a neural network structure and is trained based on the complete PTB-XL dataset to generate soft labels for classification, i.e., the probability distribution of the model output. The initial structure of the student model is the same as that of the teacher model, and the parameter initialization is not trained. During the knowledge distillation process, the student model receives the source domain data processed by the teacher model and the individual data of the target patients. By comparing the output probability distributions of the two, the student model learns the classification knowledge in the teacher model and incorporates individual features.

[0047] Preferably, the difference in probability distribution of the two outputs is measured by calculating the Kullback-Leibler divergence. The specific calculation process is as follows: given an input sample X, their output logits are obtained through the teacher model and the student model respectively, and the output of the teacher model is obtained after Softmax and temperature scaling:

[0048]

[0049] Among them, σ1 is the Softmax function, is the classification logits probability output by the teacher model for the i-th ECG record, Model teacher (x i ) is the output of the teacher model for the i-th ECG record, x i is the i-th ECG record, x j is the jth ECG record;

[0050] Output of the student model:

[0051]

[0052] in, is the classification logits probability output by the academic model for the i-th ECG record, Model student (X i ) is the output of the student model for the i-th ECG record, X j is the jth ECG record; τ is the temperature coefficient, which is used to smooth the probability distribution of the output. The difference in the probability distribution of the teacher model and the student model output is calculated using the Kullback-Leibler divergence:

[0053]

[0054] Among them, L kl is the loss function calculated using KL divergence.

[0055] Preferably, in the module M3:

[0056] The actual labels of the small-scale source domain dataset and the output probability of the student model are calculated through the binary cross entropy function to maintain the classification performance; the loss function is the weighted sum of the distillation loss and the classification loss;

[0057] The classification loss is used to measure the difference between the predicted output of the student model and the true label, and is calculated using the cross entropy loss function; the specific calculation process is as follows:

[0058]

[0059] Among them, L BCEis the loss function calculated using BCE, N is the number of samples; y i is the true label of the sample; is the prediction result of the model; σ2 is the Sigmoid activation function.

[0060] The total loss function L combines the classification loss and the distillation loss, which is a weighted sum of the two parts. The purpose is to guide the student model to learn the characteristics of the teacher model through knowledge distillation while maintaining the classification ability of the model.

[0061] L=α·L C +(1-α·)·L D

[0062] Among them, α is the weight coefficient of the two types of losses.

[0063] Compared with the prior art, the present invention has the following beneficial effects:

[0064] 1. The present invention solves the problem of model overfitting and performance degradation caused by sample scarcity in the individualization process of the existing ECG diagnosis model through a differentiated knowledge distillation method.

[0065] 2. The present invention introduces differentiated inputs between the teacher and student models, enabling the student model to effectively learn individual knowledge without affecting the general diagnostic ability of the original model, thereby significantly improving the model's diagnostic accuracy, micro-F1 score and micro-sensitivity for the target patient's ECG data. BRIEF DESCRIPTION OF THE DRAWINGS

[0066] Other features, objects and advantages of the present invention will become more apparent from the detailed description of non-limiting embodiments made with reference to the following drawings:

[0067] Figure 1 Schematic diagram of the system of the present invention. DETAILED DESCRIPTION

[0068] The present invention is described in detail below in conjunction with specific embodiments. The following embodiments will help those skilled in the art to further understand the present invention, but are not intended to limit the present invention in any form. It should be noted that, for those of ordinary skill in the art, several changes and improvements can also be made without departing from the concept of the present invention. These all belong to the protection scope of the present invention.

[0069] Embodiment 1:

[0070] This paper proposes a differentiated knowledge distillation method for achieving patient-specific personalization of the ECG diagnostic model, which can enhance the performance of the diagnostic model with only a single patient's ECG record. By inputting differentiated data into the teacher model and the student model during the knowledge distillation process, the student model can learn individual features while maintaining the diagnostic ability of the original teacher model.

[0071] According to the present invention, a personalized method for electrocardiogram diagnosis model based on differentiated knowledge distillation is provided. Figure 1 As shown, including:

[0072] Step S1: construct a differentiated data set based on the student model and the teacher model, perform random sampling of the data set and replace individual ECG data;

[0073] Specifically, in step S1:

[0074] Differentiated dataset construction, including random sampling of datasets and replacement of individual ECG data; randomly extracting part of the data from the source domain dataset to form a small-scale dataset; using the patient's individual ECG records to replace the data in the small-scale dataset of the same category to construct a differentiated dataset;

[0075] The source domain dataset comes from the PTB-XL database with a sampling frequency of 500 Hz. The labels in the dataset include diagnostic labels, morphological labels, and rhythm labels, which are 5 diagnostic categories or 23 subcategories;

[0076] Five of the diagnostic categories included normal ECG, myocardial infarction, hypertrophy, ST-T changes, and conduction disturbances;

[0077] The 23 subcategories included normal ECG, inferior myocardial infarction, anterior myocardial infarction, ST-T changes, left ventricular hypertrophy, left anterior / posterior fascicular block, ischemic ST-T changes, incomplete right bundle branch block, ischemic ST-T changes in anterior leads, atrioventricular block, intraventricular conduction disorder, nonspecific ST-T changes, complete right bundle branch block, complete left bundle branch block, left atrial overload / left atrial enlargement, ischemic ST-T changes in inferior leads, lateral myocardial infarction, right ventricular hypertrophy, right atrial overload / right atrial enlargement, incomplete left bundle branch block, Wolff-Parkinson-White syndrome, ventricular septal hypertrophy, and posterior myocardial infarction.

[0078] Step S2: extracting individualized knowledge based on differentiated data sets, including knowledge distillation process and temperature coefficients and soft labels;

[0079] Specifically, in step S2:

[0080] Individualized knowledge extraction unit, including knowledge distillation process and temperature coefficient and soft label; input the target patient sample into the student model, and input the source domain sample with the same label into the teacher model, measure the difference in the probability distribution of the two outputs, so that the student model can learn the individual knowledge of the patient; use temperature scaling and Softmax function to process the logits output by the model;

[0081] Knowledge distillation process: input the target patient samples into the student model, and input the source domain samples with the same labels into the teacher model, measure the difference in the probability distribution of the two outputs, and enable the student model to learn the individual knowledge of the patient; Temperature coefficient and soft label: use temperature scaling and Softmax function to process the logits output by the model;

[0082] The teacher model uses a neural network structure and is trained based on the complete PTB-XL dataset to generate soft labels for classification, i.e., the probability distribution of the model output. The initial structure of the student model is the same as that of the teacher model, and the parameter initialization is not trained. During the knowledge distillation process, the student model receives the source domain data processed by the teacher model and the individual data of the target patients. By comparing the output probability distributions of the two, the student model learns the classification knowledge in the teacher model and incorporates individual features.

[0083] Specifically, the difference in the probability distribution of the two outputs is measured by calculating the Kullback-Leibler divergence. The specific calculation process is as follows: given an input sample X, their output logits are obtained through the teacher model and the student model respectively, and the output of the teacher model is obtained after Softmax and temperature scaling:

[0084]

[0085] Among them, σ1 is the Softmax function, is the classification logits probability output by the teacher model for the i-th ECG record, Model teacher (x i ) is the output of the teacher model for the i-th ECG record, x i is the i-th ECG record, x j is the jth ECG record;

[0086] Output of the student model:

[0087]

[0088] in, is the classification logits probability output by the academic model for the i-th ECG record, Model student (X i ) is the output of the student model for the i-th ECG record, X jis the jth ECG record; τ is the temperature coefficient, which is used to smooth the probability distribution of the output. The difference in the probability distribution of the teacher model and the student model output is calculated using the Kullback-Leibler divergence:

[0089]

[0090] Among them, L kl is the loss function calculated using KL divergence.

[0091] Step S3: Calculate the classification loss and design the total loss function.

[0092] Specifically, in step S3:

[0093] The actual labels of the small-scale source domain dataset and the output probability of the student model are calculated through the binary cross entropy function to maintain the classification performance; the loss function is the weighted sum of the distillation loss and the classification loss;

[0094] The classification loss is used to measure the difference between the predicted output of the student model and the true label, and is calculated using the cross entropy loss function; the specific calculation process is as follows:

[0095]

[0096] Among them, L BCE is the loss function calculated using BCE, N is the number of samples; y i is the true label of the sample; is the prediction result of the model; σ2 is the Sigmoid activation function.

[0097] The total loss function L combines the classification loss and the distillation loss, which is a weighted sum of the two parts. The purpose is to guide the student model to learn the characteristics of the teacher model through knowledge distillation while maintaining the classification ability of the model.

[0098] L=α·L C +(1-α)·L D

[0099] Among them, α is the weight coefficient of the two types of losses.

[0100] Embodiment 2:

[0101] Embodiment 2 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0102] The present invention also provides an electrocardiogram diagnostic model personalization system based on differentiated knowledge distillation. The electrocardiogram diagnostic model personalization system based on differentiated knowledge distillation can be implemented by executing the process steps of the electrocardiogram diagnostic model personalization method based on differentiated knowledge distillation, that is, those skilled in the art can understand the electrocardiogram diagnostic model personalization method based on differentiated knowledge distillation as a preferred implementation of the electrocardiogram diagnostic model personalization system based on differentiated knowledge distillation.

[0103] According to the present invention, a personalized electrocardiogram diagnosis model system based on differentiated knowledge distillation is provided, comprising:

[0104] Module M1: Construct differentiated data sets based on the student model and the teacher model, perform random sampling of data sets, and replace individual ECG data;

[0105] Specifically, in the module M1:

[0106] Differentiated dataset construction, including random sampling of datasets and replacement of individual ECG data; randomly extracting part of the data from the source domain dataset to form a small-scale dataset; using the patient's individual ECG records to replace the data in the small-scale dataset of the same category to construct a differentiated dataset;

[0107] The source domain dataset comes from the PTB-XL database with a sampling frequency of 500 Hz. The labels in the dataset include diagnostic labels, morphological labels, and rhythm labels, which are 5 diagnostic categories or 23 subcategories;

[0108] Five of the diagnostic categories included normal ECG, myocardial infarction, hypertrophy, ST-T changes, and conduction disturbances;

[0109] The 23 subcategories included normal ECG, inferior myocardial infarction, anterior myocardial infarction, ST-T changes, left ventricular hypertrophy, left anterior / posterior fascicular block, ischemic ST-T changes, incomplete right bundle branch block, ischemic ST-T changes in anterior leads, atrioventricular block, intraventricular conduction disorder, nonspecific ST-T changes, complete right bundle branch block, complete left bundle branch block, left atrial overload / left atrial enlargement, ischemic ST-T changes in inferior leads, lateral myocardial infarction, right ventricular hypertrophy, right atrial overload / right atrial enlargement, incomplete left bundle branch block, Wolff-Parkinson-White syndrome, ventricular septal hypertrophy, and posterior myocardial infarction.

[0110] Module M2: Individualized knowledge extraction based on differentiated datasets, including knowledge distillation process and temperature coefficients and soft labels;

[0111] Specifically, in the module M2:

[0112] Individualized knowledge extraction unit, including knowledge distillation process and temperature coefficient and soft label; input the target patient sample into the student model, and input the source domain sample with the same label into the teacher model, measure the difference in the probability distribution of the two outputs, so that the student model can learn the individual knowledge of the patient; use temperature scaling and Softmax function to process the logits output by the model;

[0113] Knowledge distillation process: input the target patient samples into the student model, and input the source domain samples with the same labels into the teacher model, measure the difference in the probability distribution of the two outputs, and enable the student model to learn the individual knowledge of the patient; Temperature coefficient and soft label: use temperature scaling and Softmax function to process the logits output by the model;

[0114] The teacher model uses a neural network structure and is trained based on the complete PTB-XL dataset to generate soft labels for classification, i.e., the probability distribution of the model output. The initial structure of the student model is the same as that of the teacher model, and the parameter initialization is not trained. During the knowledge distillation process, the student model receives the source domain data processed by the teacher model and the individual data of the target patients. By comparing the output probability distributions of the two, the student model learns the classification knowledge in the teacher model and incorporates individual features.

[0115] Specifically, the difference in the probability distribution of the two outputs is measured by calculating the Kullback-Leibler divergence. The specific calculation process is as follows: given an input sample X, their output logits are obtained through the teacher model and the student model respectively, and the output of the teacher model is obtained after Softmax and temperature scaling:

[0116]

[0117] Among them, σ1 is the Softmax function, is the classification logits probability output by the teacher model for the i-th ECG record, Model teacher (x i ) is the output of the teacher model for the i-th ECG record, x i is the i-th ECG record, x j is the jth ECG record;

[0118] Output of the student model:

[0119]

[0120] in, is the classification logits probability output by the academic model for the i-th ECG record, Model student (X i ) is the output of the student model for the i-th ECG record, X jis the jth ECG record; τ is the temperature coefficient, which is used to smooth the probability distribution of the output. The difference in the probability distribution of the teacher model and the student model output is calculated using the Kullback-Leibler divergence:

[0121]

[0122] Among them, L kl is the loss function calculated using KL divergence.

[0123] Module M3: Perform classification loss calculation and total loss function design.

[0124] Specifically, in the module M3:

[0125] The actual labels of the small-scale source domain dataset and the output probability of the student model are calculated through the binary cross entropy function to maintain the classification performance; the loss function is the weighted sum of the distillation loss and the classification loss;

[0126] The classification loss is used to measure the difference between the predicted output of the student model and the true label, and is calculated using the cross entropy loss function; the specific calculation process is as follows:

[0127]

[0128] Among them, L BCE is the loss function calculated using BCE, N is the number of samples; y i is the true label of the sample; is the prediction result of the model; σ2 is the Sigmoid activation function.

[0129] The total loss function L combines the classification loss and the distillation loss, which is a weighted sum of the two parts. The purpose is to guide the student model to learn the characteristics of the teacher model through knowledge distillation while maintaining the classification ability of the model.

[0130] L=α·L C +(1-α)·L D

[0131] Among them, α is the weight coefficient of the two types of losses.

[0132] Embodiment 3:

[0133] Embodiment 3 is a preferred example of Embodiment 1, and is used to illustrate the present invention in more detail.

[0134] A personalized method for an electrocardiogram diagnosis model, comprising:

[0135] Differentiated datasets mean that in the process of personalized knowledge extraction, there are some differences in the input data of the student model and the teacher model, which is different from the classic knowledge distillation. Specifically, the student model inputs the patient's personal ECG data, and the teacher model inputs the ECG data in the dataset with the same disease classification as the patient's personal record. Therefore, the difference in the output of the two models can reflect the difference between the patient's ECG characteristics and the sample group, thereby realizing personalized knowledge extraction. This personalized knowledge extraction process is achieved through the calculation of KL divergence. In order to prevent personalized knowledge from affecting the student model's ability to classify diseases, we also calculated the classification loss based on the classification results output by the student model and the correct disease type corresponding to the ECG record, and guided the model learning together with the previous personalized knowledge.

[0136] Differentiated dataset construction, including random sampling of datasets and replacement of individual ECG data; first, 10% of the data is randomly extracted from the source domain dataset to form a small-scale dataset. The patient's individual ECG records are used to replace the data in the small-scale dataset of the same category to construct a differentiated dataset;

[0137] Source domain dataset: The source domain dataset comes from the PTB-XL database, which contains more than 21,000 clinical 12-lead ECG records from more than 18,000 patients with a sampling frequency of 500 Hz. The labels in the dataset include 44 diagnostic labels, 15 morphological labels, and 12 rhythm labels, which can be simplified into 5 main diagnostic categories (normal ECG, myocardial infarction, hypertrophy, ST-T changes, and conduction disorders) or 23 subcategories (normal ECG, inferior myocardial infarction, anterior myocardial infarction, ST-T changes, left ventricular hypertrophy, left anterior / posterior fascicular block, ischemic ST-T changes, incomplete right bundle branch block, ischemic ST-T changes in anterior leads, atrioventricular block, intraventricular conduction disorders, nonspecific ST-T changes, complete right bundle branch block, complete left bundle branch block, left atrial overload / left atrial enlargement, ischemic ST-T changes in inferior leads, lateral myocardial infarction, right ventricular hypertrophy, right atrial overload / right atrial enlargement, incomplete left bundle branch block, preexcitation syndrome, ventricular septal hypertrophy, and posterior myocardial infarction).

[0138] Individualized knowledge extraction unit, including knowledge distillation process and temperature coefficient and soft label; input the target patient sample into the student model, and input the source domain sample with the same label into the teacher model, measure the difference in the probability distribution of the two outputs, so that the student model can learn the individual knowledge of the patient. Use temperature scaling and Softmax function to process the logits output by the model to reduce the excessive bias towards the actual category during the classification process, thereby revealing the potential category information;

[0139] Knowledge distillation process: The target patient samples are input into the student model, and the source domain samples with the same labels are input into the teacher model, and the difference in the probability distribution of the two outputs is measured, so that the student model learns the individual knowledge of the patient; Temperature coefficient and soft label: Use temperature scaling and Softmax function to process the logits output by the model to reduce excessive bias towards the actual category during the classification process, thereby revealing potential category information.

[0140] The teacher model can use a neural network structure such as ResNet18 or ResNet50 and is trained based on the complete PTB-XL dataset to generate soft labels for classification, that is, the probability distribution of the model output; the initial structure of the student model is the same as the teacher model, but the parameter initialization has not been trained. During the knowledge distillation process, the student model receives the source domain data processed by the teacher model and the individual data of the target patient, and learns the classification knowledge in the teacher model by comparing the output probability distributions of the two, while incorporating individual features.

[0141] The difference in the probability distribution of the two outputs is measured by calculating the Kullback-Leibler (KL) divergence. The specific calculation process is as follows: Given an input sample X, the output (logits) of the teacher model and the student model are obtained respectively. After Softmax and temperature scaling, the output of the teacher model is obtained. and the output of the student model Among them, τ is the temperature coefficient, which is used to smooth the probability distribution of the output, so that the model will not be too concentrated when outputting category probabilities, and can better learn the relationship between different categories. Finally, the Kullback-Leibler (KL) divergence is used to calculate the difference in the probability distribution output by the teacher model and the student model.

[0142] Temperature scaling is used to smooth the probability distribution of the output so that the model is not too concentrated when outputting category probabilities and can better learn the relationship between different categories.

[0143] Classification ability preservation unit, classification loss calculation and total loss function design; In order to prevent the classification ability of the student model from deviating from the teacher model, the actual label of the small-scale source domain data set and the output probability of the student model are calculated through the binary cross entropy function to maintain the classification performance; The final loss function is the weighted sum of the distillation loss and the classification loss.

[0144] The classification loss is used to measure the difference between the predicted output of the student model and the true label, and is calculated using the cross entropy loss function. The specific calculation process is as follows: Where N is the number of samples; yi is the true label of the sample; yi is the prediction result of the model; σ2 is the Sigmoid activation function.

[0145] The total loss function combines the classification loss and the distillation loss, which is a weighted sum of the two parts. The purpose is to guide the student model to learn the characteristics of the teacher model through knowledge distillation while maintaining the classification ability of the model. L = α·L C +(1-α)·L D α is the weight coefficient of the two types of losses, which is generally 0.5.

[0146] Those skilled in the art know that, in addition to realizing the system and its various devices, modules, and units provided by the present invention in a purely computer-readable program code, it is entirely possible to realize the same functions in the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers by logically programming the method steps. Therefore, the system and its various devices, modules, and units provided by the present invention can be considered as a hardware component, and the devices, modules, and units included therein for realizing various functions can also be regarded as structures within the hardware component; the devices, modules, and units for realizing various functions can also be regarded as both software modules for realizing the method and structures within the hardware component.

[0147] The above describes the specific embodiments of the present invention. It should be understood that the present invention is not limited to the above specific embodiments, and those skilled in the art can make various changes or modifications within the scope of the claims, which does not affect the essence of the present invention. In the absence of conflict, the embodiments of the present application and the features in the embodiments can be combined with each other arbitrarily.

Claims

1. A personalized method for electrocardiogram diagnosis model based on differentiated knowledge distillation, characterized in that: include: Step S1: construct a differentiated data set based on the student model and the teacher model, perform random sampling of the data set and replace individual ECG data; Step S2: extracting individualized knowledge based on differentiated data sets, including knowledge distillation process and temperature coefficients and soft labels; Step S3: Calculate the classification loss and design the total loss function.

2. The electrocardiogram diagnosis model personalization method based on differentiated knowledge distillation according to claim 1, characterized in that: In step S1: Differentiated dataset construction, including random sampling of datasets and replacement of individual ECG data; randomly extracting part of the data from the source domain dataset to form a small-scale dataset; using the patient's individual ECG records to replace the data in the small-scale dataset of the same category to construct a differentiated dataset; The source domain dataset comes from the PTB-XL database with a sampling frequency of 500 Hz. The labels in the dataset include diagnostic labels, morphological labels, and rhythm labels, which are 5 diagnostic categories or 23 subcategories; Five of the diagnostic categories included normal ECG, myocardial infarction, hypertrophy, ST-T changes, and conduction disturbances; The 23 subcategories included normal ECG, inferior myocardial infarction, anterior myocardial infarction, ST-T changes, left ventricular hypertrophy, left anterior / posterior fascicular block, ischemic ST-T changes, incomplete right bundle branch block, ischemic ST-T changes in anterior leads, atrioventricular block, intraventricular conduction disorder, nonspecific ST-T changes, complete right bundle branch block, complete left bundle branch block, left atrial overload / left atrial enlargement, ischemic ST-T changes in inferior leads, lateral myocardial infarction, right ventricular hypertrophy, right atrial overload / right atrial enlargement, incomplete left bundle branch block, Wolff-Parkinson-White syndrome, ventricular septal hypertrophy, and posterior myocardial infarction.

3. The electrocardiogram diagnosis model personalization method based on differentiated knowledge distillation according to claim 1, characterized in that: In step S2: Individualized knowledge extraction unit, including knowledge distillation process and temperature coefficient and soft label; input the target patient sample into the student model, and input the source domain sample with the same label into the teacher model, measure the difference in the probability distribution of the two outputs, so that the student model can learn the individual knowledge of the patient; use temperature scaling and Softmax function to process the logits output by the model; Knowledge distillation process: input the target patient samples into the student model, and input the source domain samples with the same labels into the teacher model, measure the difference in the probability distribution of the two outputs, and enable the student model to learn the individual knowledge of the patient; Temperature coefficient and soft label: use temperature scaling and Softmax function to process the logits output by the model; The teacher model uses a neural network structure and is trained based on the complete PTB-XL dataset to generate soft labels for classification, i.e., the probability distribution of the model output. The initial structure of the student model is the same as that of the teacher model, and the parameter initialization is not trained. During the knowledge distillation process, the student model receives the source domain data processed by the teacher model and the individual data of the target patients. By comparing the output probability distributions of the two, the student model learns the classification knowledge in the teacher model and incorporates individual features.

4. The electrocardiogram diagnosis model personalization method based on differentiated knowledge distillation according to claim 3, characterized in that: The difference in the probability distribution of the two outputs is measured by calculating the Kullback-Leibler divergence. The specific calculation process is as follows: given an input sample X, their output logits are obtained through the teacher model and the student model respectively, and the output of the teacher model is obtained after Softmax and temperature scaling: Among them, σ1 is the Softmax function, is the classification logits probability output by the teacher model for the i-th ECG record, Model teacher (x i ) is the output of the teacher model for the i-th ECG record, x i is the i-th ECG record, x j is the jth ECG record; Output of the student model: in, is the classification logit probability output by the academic model for the i-th ECG record, Model student (X i ) is the output of the student model for the i-th ECG record, X j is the jth ECG record; τ is the temperature coefficient, which is used to smooth the probability distribution of the output. The difference in the probability distribution of the teacher model and the student model output is calculated using the Kullback-Leibler divergence: Among them, L kl is the loss function calculated using KL divergence.

5. The electrocardiogram diagnosis model personalization method based on differentiated knowledge distillation according to claim 1, characterized in that: In step S3: The actual labels of the small-scale source domain dataset and the output probability of the student model are calculated through the binary cross entropy function to maintain the classification performance; the loss function is the weighted sum of the distillation loss and the classification loss; The classification loss is used to measure the difference between the predicted output of the student model and the true label, and is calculated using the cross entropy loss function; the specific calculation process is as follows: Among them, L BCE is the loss function calculated using BCE, N is the number of samples; y i is the true label of the sample; is the prediction result of the model; σ2 is the Sigmoid activation function. The total loss function L combines the classification loss and the distillation loss, which is a weighted sum of the two parts. The purpose is to guide the student model to learn the characteristics of the teacher model through knowledge distillation while maintaining the classification ability of the model. L=ɑ·L c +(1-ɑ)·L D Among them, ɑ is the weight coefficient of the two types of losses.

6. A personalized system for electrocardiogram diagnosis model based on differentiated knowledge distillation, characterized in that: include: Module M1: Construct differentiated data sets based on the student model and the teacher model, perform random sampling of data sets, and replace individual ECG data; Module M2: Individualized knowledge extraction based on differentiated datasets, including knowledge distillation process and temperature coefficients and soft labels; Module M3: Perform classification loss calculation and total loss function design.

7. The electrocardiogram diagnosis model personalization system based on differentiated knowledge distillation according to claim 6, characterized in that: In the module M1: Differentiated dataset construction, including random sampling of datasets and replacement of individual ECG data; randomly extracting part of the data from the source domain dataset to form a small-scale dataset; using the patient's individual ECG records to replace the data in the small-scale dataset of the same category to construct a differentiated dataset; The source domain dataset comes from the PTB-XL database with a sampling frequency of 500 Hz. The labels in the dataset include diagnostic labels, morphological labels, and rhythm labels, which are 5 diagnostic categories or 23 subcategories; Five of the diagnostic categories included normal ECG, myocardial infarction, hypertrophy, ST-T changes, and conduction disturbances; The 23 subcategories included normal ECG, inferior myocardial infarction, anterior myocardial infarction, ST-T changes, left ventricular hypertrophy, left anterior / posterior fascicular block, ischemic ST-T changes, incomplete right bundle branch block, ischemic ST-T changes in anterior leads, atrioventricular block, intraventricular conduction disorder, nonspecific ST-T changes, complete right bundle branch block, complete left bundle branch block, left atrial overload / left atrial enlargement, ischemic ST-T changes in inferior leads, lateral myocardial infarction, right ventricular hypertrophy, right atrial overload / right atrial enlargement, incomplete left bundle branch block, Wolff-Parkinson-White syndrome, ventricular septal hypertrophy, and posterior myocardial infarction.

8. The electrocardiogram diagnosis model personalization system based on differentiated knowledge distillation according to claim 6, characterized in that: In the module M2: Individualized knowledge extraction unit, including knowledge distillation process and temperature coefficient and soft label; input the target patient sample into the student model, and input the source domain sample with the same label into the teacher model, measure the difference in the probability distribution of the two outputs, so that the student model can learn the individual knowledge of the patient; use temperature scaling and Softmax function to process the logits output by the model; Knowledge distillation process: input the target patient samples into the student model, and input the source domain samples with the same labels into the teacher model, measure the difference in the probability distribution of the two outputs, and enable the student model to learn the individual knowledge of the patient; Temperature coefficient and soft label: use temperature scaling and Softmax function to process the logits output by the model; The teacher model uses a neural network structure and is trained based on the complete PTB-XL dataset to generate soft labels for classification, i.e., the probability distribution of the model output. The initial structure of the student model is the same as that of the teacher model, and the parameter initialization is not trained. During the knowledge distillation process, the student model receives the source domain data processed by the teacher model and the individual data of the target patients. By comparing the output probability distributions of the two, the student model learns the classification knowledge in the teacher model and incorporates individual features.

9. The electrocardiogram diagnosis model personalization system based on differentiated knowledge distillation according to claim 8, characterized in that: The difference in the probability distribution of the two outputs is measured by calculating the Kullback-Leibler divergence. The specific calculation process is as follows: given an input sample X, their output logits are obtained through the teacher model and the student model respectively, and the output of the teacher model is obtained after Softmax and temperature scaling: Among them, σ1 is the Softmax function, is the classification logits probability output by the teacher model for the i-th ECG record, Model teacher (x i ) is the output of the teacher model for the i-th ECG record, x i is the i-th ECG record, x j is the jth ECG record; Output of the student model: in, is the classification logit probability output by the academic model for the i-th ECG record, Model student (X i ) is the output of the student model for the i-th ECG record, X j is the jth ECG record; τ is the temperature coefficient, which is used to smooth the probability distribution of the output. The difference in the probability distribution of the teacher model and the student model output is calculated using the Kullback-Leibler divergence: Among them, L kl is the loss function calculated using KL divergence.

10. The electrocardiogram diagnosis model personalization system based on differentiated knowledge distillation according to claim 6, characterized in that: In the module M3: The actual labels of the small-scale source domain dataset and the output probability of the student model are calculated through the binary cross entropy function to maintain the classification performance; the loss function is the weighted sum of the distillation loss and the classification loss; The classification loss is used to measure the difference between the predicted output of the student model and the true label, and is calculated using the cross entropy loss function; the specific calculation process is as follows: Among them, L BCE is the loss function calculated using BCE, N is the number of samples; y i is the true label of the sample; is the prediction result of the model; σ2 is the Sigmoid activation function. The total loss function L combines the classification loss and the distillation loss, which is a weighted sum of the two parts. The purpose is to guide the student model to learn the characteristics of the teacher model through knowledge distillation while maintaining the classification ability of the model. L=ɑ·L C +(1-ɑ)·L D Among them, ɑ is the weight coefficient of the two types of losses.

Images