Electrocardiogram classification and interpretability method based on unbalanced multi-label learning

Abstract

The invention relates to an improved multi-label electrocardiogram classification method and an interpretability method thereof, and aims to solve the problem that in the prior art, a multi-label electrocardiogram classification method based on a deep network mostly pays attention to label correlation or reconstructs a neural network, but neglects the essential problem in multi-label learning, namely naturally existing imbalance of positive and negative labels. In order to solve the problem, the invention provides a novel strategy, that is, only one pair of marks is selected for optimization each time, so that the positive and negative marks are kept balanced during training. The method comprises the following steps: firstly, resampling an original electrocardiosignal to 500Hz, carrying out normalization and cleaning work on the original signal, and dividing the original signal into time windows with fixed sizes as input of a neural network; secondly, performing model training by using a newly proposed loss function, namely, only selecting a pair of marks each time and maximizing the interval between the marks, so that the positive and negative marks are kept balanced during training; then, performing visual display on the diseases of the multi-label electrocardiogram by adopting a time domain significance scaling method so as to assist in positioning and explaining different diseases; and finally, in a test verification stage, according to a training set and a test set in a ratio of 8: 2, inputting multi-label unbalanced test data into the trained model to perform anomaly classification of the multi-label electrocardiogram.

Description

Technical Field

[0001] The embodiments of the present invention relate to the field of medical time series technology, and in particular to an electrocardiogram classification and interpretability method based on unbalanced multi-label learning. Background Art

[0002] Cardiovascular disease is one of the diseases with the highest mortality rate worldwide. According to statistics, its mortality rate continues to rank among the highest in the world. As an important diagnostic tool for screening and evaluating cardiac abnormalities, the 12-lead electrocardiogram (ECG) is widely used in clinical practice due to its cheap and non-invasive advantages. This detection method can provide a large amount of physiological information related to cardiac activity, providing important support for the analysis and diagnosis of cardiovascular diseases. However, recording and analyzing a large number of physiological signals is not only time-consuming and laborious, but also requires professional medical knowledge and rich clinical experience. In addition, manual analysis and diagnosis may be biased due to the doctor's experience, fatigue or subjective judgment. In response to the above problems, in recent years, researchers have proposed an electrocardiogram physiological signal analysis method based on automatic classification, aiming to improve the efficiency and accuracy of diagnosis and assist doctors in making decisions.

[0003] In the early research on automatic classification of ECG signals, traditional machine learning classification methods such as support vector machines and hidden Markov models were widely used in the processing and analysis of physiological signal data. However, these methods have two limitations: first, feature extraction mainly relies on manual design, which is difficult to fully adapt to complex and diverse signal features; second, the lack of processing capabilities for large-scale data sets limits its promotion and application in the era of big data. In recent years, with the rapid development of deep neural networks (DNNs), their application potential in automatic ECG classification has gradually emerged. With its powerful approximation ability, DNNs automatically fit data features, get rid of the dependence on manual feature extraction, and significantly improve analysis efficiency and accuracy. At the same time, thanks to its flexible model structure and powerful feature learning ability, DNNs outperform traditional machine learning classifiers in many scenarios.

[0004] At present, most ECG classification research works using DNN focus on the multi-classification problem, that is, segmenting the start and end points of the ECG wave group, extracting the timing data of each heartbeat, and then outputting its corresponding category, also known as single-label classification. Each patient's ECG data only corresponds to one cardiovascular disease. However, in clinical practice, each patient's ECG signal often contains multiple cardiovascular diseases at the same time, which requires multiple labels. At this time, single-label classification no longer meets the needs of clinical diagnosis, and multi-label ECG classification is more in line with reality. Compared with single-label classification, multi-label ECG classification faces greater challenges. Specifically, multi-label classification needs to deal with an exponentially growing label combination space. For example, for ECG data containing 100 categories, the possible label combinations are as many as 2 100 Such a huge label space significantly increases the difficulty of the learning task. In order to meet this challenge, a series of multi-label ECG classification algorithms have been proposed.

[0005] Existing DNN-based multi-label ECG classification research mainly focuses on multi-label correlation and network transformation. The rational use of label correlation has been proven to be beneficial to performance improvement in the field of multi-label learning. Inspired by this, researchers have also conducted a series of multi-label correlation research in the ECG field. For example, models or technologies such as graph convolutional networks (GCN), recurrent neural networks (RNN), manifold regularization, and label correlation embedding are used to characterize the correlation between labels to explore the relationship between different diseases in ECG. In addition, due to the particularity of ECG data, such as temporal sequence, periodicity, and multi-source, another popular work has targeted the transformation of DNN to better adapt to ECG data. For example, Luo et al. combined CNN with a bidirectional long short-term memory (LSTM) network, in which LSTM was used to process the temporal sequence and periodicity of ECG; Yang et al. designed a deep neural network based on multi-view and multi-scale, which regards different leads as different views and uses a multi-view network to fuse different lead features to cope with the multi-source of ECG. Although the above methods have made great progress in multi-label ECG classification tasks, they often ignore the essential problem in multi-label learning, that is, the natural imbalance of positive and negative labels. In addition, the existing multi-label ECG classification methods use Binary Cross Entropy (BCE) as the loss function by default. Since BCE treats each label equally, the heavily dominant negative labels will dominate the training process, resulting in the degradation of classification performance.

[0006] In addition to the above-mentioned imbalance of positive and negative labels, the multi-label ECG classification task faces another problem that needs to be solved urgently, namely the interpretability of the results. Given that ECG abnormal diagnosis is directly related to health and safety, deep learning models are also accompanied by high risks when applied to this task as a "black box". Therefore, how to explain the decision-making process of these black box models becomes a key issue. Most of the existing ECG classification interpretability studies use the gradient weighted class activation mapping (GradCAM) method to perform a heat map analysis of the feature importance of time domain signals, without fully considering the importance information of different spatial channels (leads) in multi-lead ECG, which just reflect the health status of different positions of the heart, and some diseases also mostly occur in specific lead areas, such as atrial fibrillation, which is usually most obvious in the V1 lead. Therefore, spatial channel (lead) information is also very critical for locating diseases and explaining decisions, and simply applying GradCAM to explain ECG classification is not completely reliable. To address this problem, the present invention introduces the Temporal Saliency Rescaling (TSR) method. Different from the GradCAM method that only considers time domain features, the TSR method considers the importance of features in both the time domain and the space domain (leads), and simultaneously performs a significant feature heat map analysis of the time and space (leads) of the 12-lead electrocardiogram by calculating the timestamps of important features and their corresponding channel correlation scores, thereby providing a more comprehensive and accurate explanation for the classification decision basis. In view of this, the present invention is proposed. Summary of the invention

[0007] Purpose of the invention: In view of the above problems, the purpose of the present invention is to propose improvements to the shortcomings of the prior art. Regarding the imbalance of positive and negative labels in multiple labels. The present invention proposes a novel strategy, which is to select only one pair of labels for optimization each time, so that the positive and negative labels maintain a balance during the training process. Specifically, the present invention selects the closest positive and negative labels each time and maximizes the interval between them, thereby deriving a novel loss function to alleviate the problem of imbalance between positive and negative labels. In addition, in response to the problem that the existing ECG methods are insufficiently interpretable and difficult to assist in diagnosis, the present invention introduces a time-domain significance rescaling method to visualize the experimental results of the proposed method to assist in locating and explaining different diseases.

[0008] Technical solution: In order to achieve the above objectives, the following technical solutions are provided:

[0009] The method of the present invention at least comprises:

[0010] Step S1: resample the original ECG signal to 500 Hz, normalize and clean the original signal, and divide it into fixed-size time windows as the input of the neural network;

[0011] Step S2: using the loss function newly proposed by the present invention to perform model training, that is, only one pair of markers is selected for optimization each time, so that the positive and negative markers are balanced during training;

[0012] Step S3: Visualizing the diseases of the multi-marker electrocardiogram by using a time-domain significance scaling method to assist in locating and explaining different diseases;

[0013] Step S4: In the test verification phase, the test data samples are input into the trained model for classification and performance evaluation.

[0014] Beneficial effects: Compared with the prior art, the technical solution of the present invention has at least the following beneficial effects:

[0015] 1) This paper proposes a novel loss function BPNL, which can solve the common problem of positive and negative label imbalance in ECG multi-label learning. This loss function does not contain any hyperparameters that need to be adjusted, and its effectiveness has been confirmed in multiple experimental verifications;

[0016] 2) This invention introduces the TSR method to assist in locating and interpreting the ECG features of different diseases, which is the first time in the field of multi-label ECG classification. This method provides a diagnostic basis for the model's decision-making, can better assist diagnosis, and also provides the possibility for deep learning methods to assist clinical medical diagnosis;

[0017] 3) The present invention was experimentally verified on the PhysioNet Challenge 2021 multi-label ECG standard dataset. The results showed that compared with the existing multi-label ECG classification method, the model trained using the loss function proposed in the present invention has superior performance. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Figure 1 is a flow chart of an electrocardiogram classification and interpretability method based on unbalanced multi-label learning proposed in the present invention;

[0019] Figure 2 The statistical results of positive and negative labels in the multi-label imbalanced data set used in the present invention;

[0020] Figure 3 The framework diagram of the overall model training of the BPNL loss function proposed in the present invention;

[0021] Figure 4 Clinical 12-lead electrocardiogram for patients diagnosed with atrial fibrillation and ST-segment depression;

[0022] Figure 5 Interpretability diagram of the 12-lead ECG model for diagnosis of atrial fibrillation;

[0023] Figure 6 This is an interpretability diagram of the 12-lead ECG model diagnosed with ST segment depression. DETAILED DESCRIPTION

[0024] In order to better understand the above technical solution, the above technical solution will be described in detail below in conjunction with the accompanying drawings and specific implementation methods.

[0025] The present invention provides an electrocardiogram classification and interpretability method based on unbalanced multi-label learning, such as Figure 1 As shown, it specifically includes steps S1 to S4.

[0026] Step S1: resample the original ECG signal to 500 Hz, normalize and clean the original signal, and divide it into fixed-size time windows as the input of the neural network;

[0027] Step S2: using the loss function newly proposed by the present invention to perform model training, that is, only one pair of markers is selected for optimization each time, so that the positive and negative markers are balanced during training;

[0028] Step S3: Visualizing the diseases of the multi-marker electrocardiogram by using a time-domain significance scaling method to assist in locating and explaining different diseases;

[0029] Step S4: In the test verification phase, the test data samples are input into the trained model for classification and performance evaluation.

[0030] The step S1 specifically includes: The PhysioNet Challenge 2021 standard ECG dataset is used, which contains 88,253 12-lead ECG data records from 8 different databases: CPSC2018 database, CPSC additional database (CPSC-EXTRA), Physical Technology Federal Database (PTB), PTB Extended Database (PTB-XL), Georgia 12-lead ECG Challenge Database (G12EC), Chapman University and Shaoxing People's Hospital Database (Chapman-Shaoxing), Ningbo First Hospital Database (Ningbo) and St. Petersburg Database (St Petersburg). In the experiment, the dataset was divided into training set and test set in a ratio of 8:2. During data preprocessing, all ECG signals were resampled to 500Hz and split into 10-second time windows, and finally Z-score normalization was applied. According to the setting of the PhysioNet / CinC 2020 / 2021 Challenge, the present invention uses the officially provided SNOMED CT code and uses 26 categories for model evaluation. Here we take three data sets as examples, and the results of other data sets are similar, such as Figure 2 As shown in Figure 2, all statistical data sets have serious imbalance problems between positive and negative labels. Existing multi-label ECG classification methods use Binary Cross Entropy (BCE) as the loss function by default. Since BCE treats each label equally, the heavily dominant negative labels will dominate the training process, resulting in the degradation of classification performance.

[0031] The step S2 includes using the loss function BPNL proposed in the present invention, that is, when training the model, only one pair of labels is pushed away each time so that the positive and negative labels are balanced during the training process. Specifically, it includes: Figure 3 First, the 12-lead ECG signal is cut into a fixed-size time series window and input into the backbone network for feature extraction. Then the network outputs a prediction vector To represent the classification result or confidence of the model for the ECG data, also recorded as The model will then predict the outcome With real mark The training is completed by calculating the BPNL loss function and optimizing the network parameters θ using the back propagation algorithm. The lower part of the framework diagram shows the training process of the loss function optimization mark, that is, only one pair of positive and negative marks is selected for optimization each time, and this process is iterated until the network training converges.

[0032] The derivation process is as follows: First, the interval between positive and negative markers is defined, that is, the distance between the positive marker with the smallest score and the negative marker with the largest score. Here, the score refers to the logits output of the neural network (without sigmoid activation). g The formal definition of is as follows:

[0033]

[0034] where s p and n are the scores of positive and negative labels, respectively, and Ω p and Ω n are the index sets corresponding to the positive and negative labels respectively.

[0035] The interval Δ defines the positive and negative markers. g Then, based on the large margin theory, maximize Δ g , we can get the following formula:

[0036]

[0037] Where θ is the parameter of the neural network.

[0038] Since min(a, b) = -max(-a, -b), the above formula can be converted into the following equivalent form:

[0039]

[0040] To facilitate the derivation of the loss function, replace the first max in the above formula with min, and you can get the following formula:

[0041]

[0042] Considering that the max function is not differentiable, its corresponding smooth approximate log sum exp function is used here instead. The definition of log sum exp is as follows:

[0043]

[0044] Among them, a i ∈(-∞, +∞), i = 1, 2, ..., k. log(·) is the logarithmic function, e (·) is an exponential function. Substituting (5) into (4), we can get the following objective function:

[0045]

[0046] Naturally, The loss function is unbounded, so minimize This will cause the loss to tend to negative infinity, making it unstable and difficult to optimize when used for neural network training. In order to solve this problem, the present invention adds two constants 1 to the loss function term, making the loss function non-negative, that is, bounded. The final balanced positive and negative labeled BPNL loss function is as follows:

[0047]

[0048] At this point, the derivation of the BPNL loss function is completed.

[0049] In the step S3, most of the existing ECG classification interpretability studies use the GradCAM method to perform a heat map analysis of the feature importance of the time domain signal, without fully considering the importance information of different spatial channels (leads) in the multi-lead ECG, and different leads just reflect the health status of different positions of the heart, and some diseases usually occur in specific lead areas, such as atrial fibrillation, which usually has the most obvious characteristics at the V1 lead. Therefore, spatial channel (lead) information is crucial for locating diseases and interpreting decisions, and simply applying GradCAM to interpret ECG classification is not completely reliable. In order to solve this problem, the present invention introduces the TSR method. Unlike the GradCAM method that only considers time domain features, the TSR method considers the feature importance of both time and space domains. Specifically, TSR calculates the timestamps considered to be important features and their corresponding channel correlation scores, and selects the first n timestamps as important features from the correlation scores from high to low, to simultaneously perform a significant feature heat map analysis on the time and space (leads) of the 12-lead electrocardiogram, and generate a more comprehensive model result interpretability map that can assist doctors in diagnosis. To the best of the authors’ knowledge, this is the first time in the ECG field. The process of TSR is summarized as follows as shown in Algorithm 1.

[0050]

[0051] In order to verify the rationality of the TSR method introduced in this paper in explaining ECG abnormalities, this section selects the ECG data of a patient from the PhysioNet2021 multi-label dataset as an example. The patient's 12-lead ECG is as follows Figure 4 As shown, medical experts noted the presence of both atrial fibrillation and ST segment depression.

[0052] Here, the 10-second ECG signal is divided into multiple heartbeat cycles according to the R wave, each cycle has about 600 timestamps, and is input into the model trained with the BPNL method of the present invention as the loss function. Finally, the TSR method is used to generate the disease type heat map of different heartbeat cycles, as shown in FIG. Figure 5 and Figure 6As shown in the figure. The horizontal axis represents the timestamp, and the vertical axis represents the 12-lead channel. The feature importance is quantified by the color brightness gradient, ranging from 0 to 1. The closer the value is to 1, the brighter it is, indicating that the model determines that the feature is more important; the closer the value is to 0, the darker it is, indicating that the model determines that the feature is less important.

[0053] exist Figure 5 In the image, the bright areas are mainly concentrated near the QRS wave and the fibrillation wave, especially in the V1 lead. This is consistent with the diagnostic criteria for atrial fibrillation: the normal P wave disappears, replaced by fibrillation waves of varying sizes and shapes, usually most prominent in the V1 lead. Figure 6 In the figure, the bright areas mainly appear in the ST segments of the V1 and V4 leads, and show the characteristics of ST segment depression: that is, the ST segment is horizontally or downwardly sloping and moves downward by more than 0.1mV, which is consistent with the diagnostic criteria for ST depression. The visualization results further verify that the heat map generated by the TSR method can not only accurately associate the heartbeat areas that the model considers important with the diagnostic categories, providing a basis for model decision-making, but also locate in specific lead areas, reflecting the health status of different parts of the heart. This provides clinicians with a reliable diagnostic aid to help them more accurately judge the patient's condition, thereby effectively reducing the clinical diagnostic burden of doctors.

[0054] The step S4 specifically includes taking 20% ​​of the data as a test data set in the test verification stage and inputting it into the trained model for multi-label class imbalance classification of the electrocardiogram. In addition, the present invention selects two sub-datasets G12EC and Chapman-Shaoxing from the data set as the test cross-dataset model generalization for verification.

[0055] In summary, the electrocardiogram classification and interpretability method based on unbalanced multi-label learning proposed in the present invention is of positive help in studying the imbalance problem of positive and negative labels in ECG data in reality, and is of great significance in clinical research. The present invention focuses on the multi-label ECG classification task, mainly solving the imbalance problem of positive and negative labels and the interpretability problem of the model. For the former problem, the present invention proposes a strategy of selecting only one pair of positive and negative labels for optimization each time, and derives a new loss function BPNL from it. Experiments on standard ECG data sets have verified the effectiveness of BPNL in solving the imbalance problem of positive and negative labels. For the latter problem, the present invention considers the importance of features from both the time domain and the spatial domain, and introduces the TSR method to help locate the disease and explain the basis for diagnosis. The heat map analysis on the patient's real ECG data verifies the rationality of the TSR method in explaining ECG abnormalities. The experimental results show that compared with the most advanced multi-label ECG classification method, the method of the present invention achieves better performance.

[0056] The above specific embodiments are only for explaining the principle and technical method of the method proposed in the present invention, and are not intended to limit the implementation of the technical solution of the present invention. For those skilled in the art, the technical solution of the method proposed in the present invention can be reasonably adjusted and replaced according to needs, and these adjustments and replacements are protected by the claims of the present invention.

Claims

1. An electrocardiogram classification and interpretability method based on unbalanced multi-label learning, characterized in that: The method at least comprises: Step S1: resample the original ECG signal to 500 Hz, normalize and clean the original signal, and divide it into fixed-size time windows as the input of the neural network; Step S2: using the loss function newly proposed by the present invention, that is, only one pair of markers is selected for optimization each time during model training, so that the positive and negative markers are balanced during the training process; Step S3: Visualizing the diseases of the multi-marker electrocardiogram by using a time-domain significance scaling method to assist in locating and explaining different diseases; Step S4: In the test verification phase, the test ECG data sample is input into the trained model for abnormality classification and performance evaluation.

2. The electrocardiogram (ECG) classification process of unbalanced multi-label learning according to claim 1, characterized in that: Step S1 includes using the PhysioNet Challenge 2021 standard ECG dataset for the data set, which contains 88,253 12-lead ECG data records from 8 different databases: CPSC2018 database, CPSC additional database (CPSC-EXTRA), Physical Technology Federal Database (PTB), PTB Extended Database (PTB-XL), Georgia 12-lead ECG Challenge Database (G12EC), Chapman University and Shaoxing People's Hospital Database (Chapman-Shaoxing), Ningbo First Hospital Database (Ningbo) and St. Petersburg Database (St Petersburg). In the experiment, the data set was divided into training set and test set in a ratio of 8:

2. During data preprocessing, all ECG signals were resampled to 500Hz and split into 10-second time windows, and finally Z-score normalization was applied. According to the setting of the PhysioNet / CinC 2020 / 2021 challenge, this paper adopts the officially provided SNOMED CT code and uses 26 categories for model evaluation.

3. The electrocardiogram signal data set introduction and preprocessing process according to claim 2, characterized in that: Step S2 uses the newly proposed loss function of the present invention during model training, that is, only one pair of markers is selected for optimization each time, so that the positive and negative markers are balanced during the training process, specifically including: Given a multi-label ECG training dataset x i represents the i-th ECG sample, y i is its corresponding mark, N represents the total number of samples, where F is the number of feature channels, T is the length of the timestamp, y i =[y i1 ,y i2 , ..., y iC ]∈{0,1} 1×C , where y i Each element of corresponds to a type of arrhythmia, and C is the number of labels. ij =1, indicating that sample x i Contains the jth label, called a positive label, otherwise y ij = 0, called a negative label. The goal of multi-label ECG classification is to train a model to predict the label of the test sample in the input space χ. There is a serious imbalance problem between positive and negative labels in ECG data. Existing ECG classification algorithms often ignore this problem, and most of them use Binary Cross Entropy (BCE) as the loss function, but BCE cannot solve the imbalance problem. To this end, the present invention proposes a novel balanced positive-negative label (BPNL) loss function. The motivation of this loss function comes from a very direct strategy, that is, only one pair of positive and negative labels is selected for optimization each time, so that the positive and negative labels are balanced during training. The key to the success of this strategy lies in what pair of labels to choose for optimization? Inspired by the large margin theory, the present invention first defines the interval between positive and negative labels, that is, the distance between the positive label with the smallest score and the negative label with the largest score. Here, the score refers to the logits output of the neural network (without sigmoid activation). The formal definition of the interval Δg is as follows: where s p and n are the scores of positive and negative labels, respectively, and Ω p and Ω n are the index sets corresponding to the positive and negative labels respectively. The interval Δ defines the positive and negative markers. g Then, based on the large margin theory, maximize Δ g , we can get the following formula: Where θ is the parameter of the neural network. Since min(a, b) = -max(-a, -b), the above formula can be converted into the following equivalent form: To facilitate the derivation of the loss function, replace the first max in the above formula with min, and you can get the following formula: Considering that the max function is not differentiable, its corresponding smooth approximate log sum exp function is used here instead. The definition of log sum exp is as follows: Among them, a i ∈(-∞, +∞), i = 1, 2, ..., k. log(·) is a logarithmic function, and e(·) is an exponential function. Substituting (5) into (4), we get the following objective function: Naturally, The loss function is unbounded, so minimize This will cause the loss to tend to negative infinity, making it unstable and difficult to optimize when used for neural network training. In order to solve this problem, the present invention adds two constants 1 to the loss function term, making the loss function non-negative, that is, bounded. The final balanced positive and negative labeled BPNL loss function is as follows:

4. Model training is performed according to the proposed BPNL loss function according to claim 3, characterized in that: In step S3, after the model is trained using the loss function proposed by the present invention, the temporal saliency rescaling method (TSR) is used to visualize the diseases of the multi-marker electrocardiogram to assist in locating and explaining different diseases. The present invention adopts the temporal significance scaling method for interpretability analysis. Different from the interpretability method that only applies GradCAM (Gradient-weighted Class Activation Mapping) in the temporal domain, the TSR method considers the importance of both the temporal and spatial domains (leads) and calculates the features that are considered to be important. The model relevance score of the channel corresponding to the timestamp. The specific method of the TSR method is shown in Algorithm 1: It should be noted that Δ t The correlation score corresponding to the absolute value of the difference before and after the mask is obtained based on the comprehensive evaluation of the accuracy and recall rate in the model training. This method can help doctors locate which channels the disease mainly manifests in (different channels correspond to different parts of the heart), thereby improving the accuracy of diagnosis.

5. The method of claim 4 for visualizing diseases in multi-marker electrocardiograms using a time-domain significance scaling method to assist in locating and explaining different diseases, characterized in that: In step S4, 20% of the multi-label unbalanced ECG data is used as a test data set in the test verification stage and input into the trained model for multi-label classification of the ECG. In addition, two sub-datasets G12EC and Chapman-Shaoxing are selected from the data set of the present invention as test data sets for verification. Specifically, the experiment adopts an alternating test method: when G12EC is used as a test data set, the remaining data sets are used as training sets; similarly, when Chapman-Shaoxing is used as a test data set, the remaining data sets are used as training sets. Through this setting, the adaptability of the method of the present invention in processing cross-dataset tasks can be more comprehensively evaluated.

Images