Arrhythmia classification method based on multi-scale space-time learning and adaptive selection neural network

Abstract

The invention provides an arrhythmia classification method based on multi-scale space-time learning and an adaptive selection neural network. Aiming at the problems that electrocardiosignals have different complex wave band information of various scales, pathological changes have the characteristics of transience and intermittency in time sequence, and different channels in space contain different pathological change information, a multi-scale thought is introduced in the invention to extract the complex wave band information; a multi-scale sparse time sequence attention module is designed to capture local tight and remote sparse time sequence feature information, a multi-scale residual convolution module is designed to interact and integrate spatial feature information in different channels, and an adaptive selection module is designed to dynamically select different scales and time steps. In some public data sets, the method has better classification performance and higher diagnosis speed.

Description

Technical Field

[0001] The present invention belongs to the technical field of machine learning and pattern recognition. A neural network for arrhythmia classification is designed and trained for a given 12-lead electrocardiogram signal, and then new electrocardiogram signals are automatically classified to detect all possible arrhythmia diseases. Background Art

[0002] According to the World Health Organization, cardiovascular disease is currently the leading cause of death worldwide, with an estimated 17.9 million lives lost annually. Arrhythmias are the most common and important cardiovascular disease, referring to a group of conditions in which the heart beats too slowly, too quickly, or irregularly. Arrhythmias are often present in the early stages of cardiovascular disease. If arrhythmias can be diagnosed promptly and accurately in the early stages of the disease, targeted treatment and management measures can be implemented early on, significantly improving treatment effectiveness and patient survival rates.

[0003] An electrocardiogram (ECG) is a graphical representation of the heart's electrical activity. A normal ECG signal consists of different types of wave bands, including the P wave, QRS complex, T wave, and PR segment. A heart attack can affect the heart's conductive system. An ECG can be used to identify specific heart conditions based on the characteristics and morphology of the waves, allowing for health monitoring. However, accurately identifying these complex arrhythmia-related ECG rhythms requires considerable clinical experience and expertise. The introduction of computer-aided diagnosis aims to assist doctors in making accurate ECG diagnoses and improve efficiency. Currently, many computer-aided studies have been used in ECG diagnosis, with the main research methods including machine learning and deep learning.

[0004] In traditional machine learning, feature extraction is often an independent step. These features can be manually designed or obtained through some automatic feature extraction algorithms. Common features include P-QRS-T features and morphological features. These features are then classified using methods such as support vector machines and k-nearest neighbors. Deep learning methods learn features and perform classification tasks directly from preprocessed ECG signals through end-to-end learning. They perform well in processing large-scale and complex ECG data and have gradually become mainstream in ECG analysis. Based on the inherent properties of the main feature extractors within the neural network, deep learning models for diagnosing arrhythmias can be mainly divided into the following categories: (1) Convolutional neural networks (CNN): The convolution operation can efficiently capture local patterns in the ECG, such as waveforms and peaks, which helps to accurately identify cardiac events and abnormalities. (2) Recurrent neural networks (RNN) or Transformer: Since ECG signals are time series, the temporal correlation within the signal is expected to better reveal the characteristics of arrhythmia categories. (3) Combined models: Combining CNN, RNN or Transformer aims to fully utilize the local patterns in the ECG and the correlation of signals in different time periods.

[0005] Most current studies overlook the often transient and intermittent nature of ECG lesions. For example, atrial fibrillation typically manifests as the disappearance of P waves and the appearance of fibrillation waves. These lesions are localized and sparsely distributed in time series. Different leads in space also contain different lesion information. Ignoring these characteristics may not fully extract the key features of the lesions. Furthermore, ECGs contain signals of various frequencies and scales, such as QRS complexes, P waves, and T waves. The local duration of lesions can vary, and a single-scale convolution may not be able to effectively capture these diverse scale features. Some multi-scale networks do not adaptively select these features. Summary of the Invention

[0006] To address the problems that ECG signals have complex band information of various durations, lesions are short-lived and intermittent in time, and different lesion information contained in different leads in space, we proposed an arrhythmia classification method based on multi-scale spatiotemporal learning and adaptive selection neural networks based on multi-scale thinking. This method can adaptively model the characteristic information of ECG signals in both the temporal and spatial dimensions.

[0007] This paper discloses an arrhythmia classification method based on multi-scale spatiotemporal learning and adaptive selection neural networks. The method incorporates multi-scale concepts to extract complex band information. A multi-scale sparse temporal attention module is designed to sparsely model the dependencies of the time periods most relevant to the lesion, capturing both locally dense and remotely sparse lesion information in the time series. A multi-scale residual convolution module is designed to model the interrelationships between different channels, enabling interaction and integration of features across these channels. An adaptive selection module is designed to dynamically select feature maps containing information at different scales, choosing the scale and time step that best represent the lesion. The network's input is electrocardiogram (ECG) signal data, and its output is the probability of the class to which it belongs.

[0008] The overall process and specific method steps of the present invention are as follows:

[0009] Step 1: Preprocess the 12-lead ECG signal;

[0010] (1) The sampling frequency of ECG data was reduced to 250 Hz; (2) Since the length of each ECG data was different, all data were trimmed to 20 seconds. If the original data exceeded 20 seconds, the excess data was trimmed, and the insufficient length was padded with zeros; (3) The ECG dataset was divided into 5-fold cross-validation for 5-fold cross-validation;

[0011] Step 2: Build an MSASNet model for 12-lead ECG signals;

[0012] In order to cope with the complex bands and different lesion information in time and space of 12-lead ECG data, the model consists of four stages. The first two stages are composed of a multi-scale sparse temporal attention module (MSSTA), a multi-scale residual convolution module (MSRCM) and an adaptive selection module (ASM). In the last two stages, MSSTA is replaced by the Transformer's multi-head self-attention.

[0013] In MSSTA, a sliding window operation is used to extract sparse lesion information in time series. For the query vector at position (s) in the time series, key vectors and value vectors are sparsely selected, and self-attention is performed in a sliding window of size w centered on (s). Different hole rates dr in the window are designed to represent different degrees of sparsity, and the sparsity is adaptively selected through ASM. In MSRCM, variable-dimensional convolution and depthwise separable convolution are used to extract lesion information in the channel. Variable-dimensional convolution with a convolution kernel size of 1 is used to first increase the dimension so that the middle depthwise separable convolution has more channel information, and then reduce the dimension to integrate important information from different channels. Finally, multiple branches with different convolution kernel sizes are used to extract information at different scales, and the scale is adaptively selected through ASM.

[0014] In ASM, multiple feature maps x1-x n As input, firstly, the information of all scales is summed and fused and global average pooling is applied to the aggregate information X to generate channel-level information X g , and then obtain the feature X through convolution and channel splicing c , the weights α of different scale features are obtained through the Softmax function, and then multiplied and added with the corresponding scale features to obtain the scale-selected feature X s On the other hand, the feature dimension of the aggregated information X is reduced to 1 by convolution to obtain the time-step level information X t , get the weight β of different time steps through Sigmoid function, and then add it to X s Multiply to get the features of dynamically selected scale and time step;

[0015] Step 3: Train the constructed MSASNet model and predict the ECG signals in the test set. Perform five such cross-validation experiments and take the average to obtain the arrhythmia classification results.

[0016] Compared with traditional methods, the present invention has the following advantages: (i) it does not require complex data processing and can perform end-to-end automatic diagnosis, which is convenient and fast; (ii) it fully utilizes the key information of the lesion in time and space, integrates multi-scale features and adaptively selects features, thereby improving the classification performance of the model. In addition, the model has a low number of parameters and computational complexity, a fast model inference speed, and can be deployed in electrocardiogram monitoring equipment. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] Figure 1 is a flow chart of the present method;

[0018] Figure 2 It is a neural network structure diagram designed by the present invention;

[0019] Figure 3 This is the structure diagram of the multi-scale residual convolution module designed by the present invention, where C and L represent the number of channels and time step of the feature map after convolution and other operations respectively;

[0020] Figure 4 This is the structure diagram of the multi-scale sparse temporal attention module designed by the present invention, where C and L represent the number of channels and time steps of the feature map after convolution and other operations respectively;

[0021] Figure 5 This is the structural diagram of the adaptive selection module designed by the present invention, where C and L respectively represent the number of channels and time step of the feature map after operations such as convolution, and sr represents the channel scaling rate. DETAILED DESCRIPTION

[0022] In order to make the purpose, technical solutions and advantages of the present invention clearer, the following is a detailed description of the specific details involved in the technical invention solution:

[0023] The present invention proposes an arrhythmia classification method based on multi-scale spatiotemporal learning and adaptive selection neural network. The electrocardiogram signal data used is from the 2018 China Physiological Signal Challenge (2018CPSC) and the 2021 Computational Challenge in Cardiology (2021CinC). The 2018 CPSC dataset, sourced from 11 hospitals, contains 6,877 publicly available ECG records, each containing 12 standard leads. The signal is sampled at a frequency of 500 Hz and lasts for 6-60 seconds. The dataset encompasses eight arrhythmias and one normal rhythm, for a total of nine ECG types. The 2021 CinC dataset, comprised of six sources from four countries across three continents, has over 88,000 officially publicly available ECG records, each containing 12 standard leads. Due to the diverse data sources and varying signal sampling frequencies and durations, the dataset includes 133 diagnostic categories, 26 of which are relatively common and clinically meaningful for algorithm evaluation. Therefore, only the classification models for these 26 categories are evaluated during scoring. Details are shown in Table 1. Each ECG record in the dataset contains one or more labels, indicating that the patient suffers from one or more arrhythmias. The goal of this invention is to construct a multi-label classification algorithm that can correctly identify all types of arrhythmias in patients.

[0024] The overall process of the present invention is shown in the appendix of the invention. Figure 1 As shown, the specific implementation is as follows:

[0025] Step 1: Preprocess the 12-lead ECG signal;

[0026] (1) The sampling frequency of ECG data was reduced to 250 Hz; (2) Since the length of each ECG data was different, all data were trimmed to 20 seconds. If the original data exceeded 20 seconds, the excess data was trimmed, and the insufficient length was padded with zeros; (3) The ECG dataset was divided into 5-fold cross-validation for 5-fold cross-validation;

[0027] Step 2: Build an MSASNet model for 12-lead ECG signals;

[0028] As shown in the accompanying drawings Figure 2The model shown consists of four stages. The first two stages use the multi-scale sparse temporal attention module (MSSTA), the multi-scale residual convolution module (MSRCM) and the adaptive selection module (ASM). In the last two stages, the MSSTA is replaced with the Transformer's multi-head self-attention, which enables the model to fully utilize locality and sparsity when processing low-level information, and simulate long-distance dependencies when processing high-level information.

[0029] The MSRCM structure is as shown in the accompanying figure of the specification. Figure 3 As shown in the figure, variable-dimensional convolution and depth-wise separable convolution are used to extract the lesion information in the channel. The variable-dimensional convolution with a convolution kernel size of 1 is first used to increase the dimension so that the middle depth-wise separable convolution has more channel information, and then the dimension is reduced to integrate the important information of different channels. Finally, multiple branches with different convolution kernel sizes are used to extract information of different scales.

[0030] Table 1 Datasets used in the invention

[0031]

[0032] The MSSTA structure is shown in the accompanying figure. Figure 4 As shown in the figure, a sliding window operation is used to extract remote sparsity information. For the query vector at position (s) in the time series, the key vector and value vector are sparsely selected, and self-attention is performed in a sliding window of size w centered on (s). In order to express different sparsity levels and different remote distances and to satisfy the acquisition of multi-scale feature information, a hole rate dr is designed based on the sliding window. When dr is 1, it is locally dense, and when dr gradually increases, it satisfies the remote sparsity characteristics. The calculation of the sparse operation and sparse attention is as follows.

[0033] Sparse Operation=Select(Q, K, V; w, dr)

[0034]

[0035] Where Q, K, V represent queries, keys and values, K dr 、V dr Represents sparse selection of keys and values

[0036] The ASM structure is shown in the accompanying figure. Figure 5 As shown, multiple feature maps x of different scales are nAs input, we first reduce the feature dimension by convolution to reduce the number of parameters and add the information of all levels to fuse them. Then we use global average pooling on the aggregate information X to generate channel-level information X g , and then obtain the feature X through convolution and channel splicing c , the weights α of different scale features are obtained through the Softmax function, and then multiplied and added with the corresponding scale features to obtain the scale-selected feature X s , the process can be described as:

[0037] X=F sum (F conv (x1),...,F conv (x n ))

[0038] X g =F gap (X)

[0039] X c =F concat (F conv (X g ),..., F conv (X g ))

[0040] α=Softmax(X c )

[0041]

[0042] Then, the feature dimension of the aggregated information X is reduced to 1 by convolution to obtain the time-step level information X t , get the weight β of different time steps through Sigmoid function, and then add it to X s Multiplying together the features of the dynamically selected scale and time step can be described as:

[0043] X t =F conv (X)

[0044] β=Sigmoid(X t )

[0045] Output=βX s

[0046] Finally, the probability of each type of arrhythmia in ECG data is obtained through a classification layer consisting of global average pooling, global maximum pooling and fully connected layer and a Sigmoid function.

[0047] Step 3: Optimize and train the constructed MSASNet model;

[0048] The loss function used is the asymmetric loss function (ASL), which is mainly used to solve the imbalance problem of positive and negative samples in the multi-label classification task and reduce the contribution of the easy negative class loss. The calculation process is as follows.

[0049]

[0050] p m =max(pm,0)

[0051] Among them L + is the loss of the positive class, L - is the loss of the negative class, p is the predicted probability of the model, γ + is the focusing factor of the positive class, γ - is the focus factor of the negative class, m is the offset of the negative class probability, p m is the negative class probability after offset.

[0052] Then predict the 12-lead ECG signal in the test set:

[0053] (1) Preprocess each 12-lead ECG signal record in the test set using the method described in step 1;

[0054] (2) Input the preprocessed data into the trained MSASNet model, and finally obtain the probability vector P of each category through the forward propagation of the model. n ={p n 1 , p n 2 ,...,p n k}, where n represents the nth ECG data, w n k Indicates the probability that the nth data belongs to category k;

[0055] (3) Set a fixed classification threshold, if w n k >threshold, then the nth data belongs to category k, and all categories to which the nth data belongs are obtained, that is, all types of arrhythmia diseases suffered by the patient are obtained.

[0056] Table 2 Comparison of results of different methods

[0057]

[0058] In order to verify the effectiveness and feasibility of the method of the present invention, the 12-lead ECG signal datasets of the 2018CPSC dataset and the 2021CinC dataset were used for verification, and the F1 value and the challenge score in 2021CinC (a weight matrix was added on the basis of F1, and similar categories were also judged to have scores) were used to evaluate the model performance. The comparison with popular methods is shown in Table 2. The F1 value of this method on the 2018CPSC dataset is 0.838, and the challenge score in the 2021CinC dataset is 0.728, which proves that this method has better classification performance than other methods. In addition, the complexity comparison of this invention with some classic classification models is shown in Table 3, which also proves that the model parameter amount and computational complexity of the present invention are low, and it has a faster training and testing speed.

[0059] Table 3. Comparison of the complexity of different methods. There are 5502 ECG records in the training set and 1375 ECG records in the test set.

[0060]

[0061] [1]HUANG Y, YEN GG, TSENG V S. Snippet Policy Network V2: Knee-GuidedNeuroevolution for Multi-Lead ECG Early Classification[J / OL]. IEEE Transactions on Neural Networks and Learning Systems, 2022: 1-15. DOI: 10.1109 / TNNNLS.2022.3187741.

[0062] [2] JYOTISHI D, DANDAPAT S. An Attentive Spatio-Temporal Learning-BasedNetwork for Cardiovascular Disease Diagnosis[J / OL]. IEEE Transactions on Systems, Man, and Cybernetics: Systems, 2023, 53(8): 4661-4671. DOI: 10.1109 / TSMC.2023.3257022.

[0063] [3]BUI T H.Automatic varied-length ECG classification using alighhtweight DenseNet model[J].Biomedical Signal Processing and Control,2023.

[0064] [4]QIN L, XIE Y, LIU X, et al.An End-to-End 12-Leading Electrocardiogram Diagnosis System Based on Deformable Convolutional Neural Network With Good Antinoise Ability[J / OL].IEEE Transactions on Instr umentation and Measurement,2021.70:1-13.DOI:10.1109 / TIM.2021.3073707.

[0065] [5]NEJEDLY P,IVORA A,SMISEK R, et al.Classification of ECG Using Ensemble of Residual CNNs with Attention Mechanism[C / OL] / / 2021 Computing in Cardiology(CinC):Volume 48.2021:1-4[2024-10-23].https: / / ieeexplore.ieee.org / abstract / document / 9662723.DOI:10.23919 / CinC53138.2021.9662723.

[0066] [6]HAN H,PARK S,MIN S, et al.Towards High Generalization Performance on Electrocardiogram Classific ation[C / OL] / / 2021 Computing in Cardiology(CinC):Volume 48.2021:1-4[2024-10-23].https: / / ieeexplore.ieee.org / abstact / document / 9662737.DOI:10.23919 / CinC53138.2021.9662737.

[0067] [7]HAN H, PARK S, MIN S, et al. Improving generalization performance of electrocardiogram classification models[J / OL]. Physiological Measurement, 2023, 44(5): 054003. DOI: 10.1088 / 1361-6579 / acb30f.

[0068] [8]BORTOLAN G. 3D ECG display with deep learning approach for identification of cardiac abnormalities from a variable number of leads[J / OL]. Physiological Measurement, 2023, 44(2): 024002. DOI: 10.1088 / 1361-6579 / acb4dc.

[0069] [9]HE K, ZHANG X, REN S, et al. Deep Residual Learning for Image Recognition[C / OL] / / Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition. 2016: 770-778[2024-04-13]. https: / / openaccess.thecvf.com / content_cvpr_2016 / html / He_Deep_Residual_Learning_CVPR_2016_paper.html.

[0070]

[10] SZEGEDY C, WEI LIU, YANGQING JIA, et al. Going deeper with convolutions[C / OL] / / 2015 IEEE Conference on Computer Vision and Pattern Recognition (CVPR). Boston, MA, USA: IEEE, 2015: 1-9[2024-03-31]. http: / / ieeexplore.ieee.org / document / 7298594 / . DOI: 10.1109 / CVPR.2015.7298594.

[0071]

[11] HOCHREITER S, SCHMIDHUBER J. Long Short-Term Memory[J / OL]. Neural Computation, 1997, 9(8): 1735-1780. DOI: 10.1162 / neco.1997.9.8.1735.

[0072]

[12] DEY R, SALEM F M. Gate-variants of Gated Recurrent Unit (GRU) neural networks[C / OL] / / 2017 IEEE 60th International Midwest Symposium on Circuits and Systems (MWSCAS). 2017: 1597-1600[2024-04-13]. https: / / ieeexplore.ieee.org / abstract / document / 8053243. DOI: 10.1109 / MWSCAS.2017.8053243.

[0073]

[13] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An Image is Worth 16x16 Words: Transformers for Image Recognition at Scale[A / OL]. arXiv, 2021[2024-04-13]. http: / / arxiv.org / abs / 2010.11929. DOI: 10.48550 / arXiv.2010.11929.

[0074]

[14] WANG W, XIE E, LI X, et al. Pyramid Vision Transformer: A Versatile Backbone for Dense Prediction Without Convolutions[C / OL] / / Proceedings of the IEEE / CVF International Conference on Computer Vision. 2021: 568-578[2024-04-13]. https: / / openaccess.thecvf.com / content / ICCV2021 / html / Wang_Pyramid_Vision_Transformer_A_Versatile_Backbone for_Dense_Prediction_Without_ICCV_2021_paper.html.

Claims

1. A method for arrhythmia classification based on multi-scale spatiotemporal learning and adaptive selection neural network, MSASNet, is characterized by: The method comprises the following steps: Step 1: Preprocess the 12-lead ECG signal; (1) The sampling frequency of the electrocardiogram (ECG) data was reduced to 250 Hz; (2) Since the length of each ECG data was different, all data were trimmed to 20 seconds. If the original data exceeded 20 seconds, the excess data was trimmed, and the data with insufficient length was padded with zeros; (3) The ECG dataset was divided into 5-fold cross-validation for 5-fold cross-validation; Step 2: Build an MSASNet model for 12-lead ECG signals; In order to cope with the complex bands and different lesion information in time and space of 12-lead ECG data, the model consists of four stages. The first two stages are composed of a multi-scale sparse temporal attention module (MSSTA), a multi-scale residual convolution module (MSRCM) and an adaptive selection module (ASM). In the last two stages, MSSTA is replaced by the Transformer's multi-head self-attention. In MSSTA, a sliding window operation is used to extract sparse lesion information in time series. For the query vector at position (s) in the time series, key vectors and value vectors are sparsely selected, and self-attention is performed in a sliding window of size w centered on (s). Different hole rates dr in the window are designed to represent different degrees of sparsity, and the sparsity is adaptively selected through ASM. In MSRCM, variable-dimensional convolution and depthwise separable convolution are used to extract lesion information in the channel. Variable-dimensional convolution with a convolution kernel size of 1 is used to first increase the dimension so that the middle depthwise separable convolution has more channel information, and then reduce the dimension to integrate important information from different channels. Finally, multiple branches with different convolution kernel sizes are used to extract information at different scales, and the scale is adaptively selected through ASM. In ASM, multiple feature maps x1-x n As input, firstly, the information of all scales is summed and fused and global average pooling is applied to the aggregate information X to generate channel-level information X g , and then obtain the feature X through convolution and channel splicing c , the weights α of different scale features are obtained through the Softmax function, and then multiplied and added with the corresponding scale features to obtain the scale-selected feature X s On the other hand, the feature dimension of the aggregated information X is reduced to 1 by convolution to obtain the time-step level information X t , get the weight β of different time steps through Sigmoid function, and then add it to X s Multiply to get the features of dynamically selected scale and time step; Step 3: Train the constructed MSASNet model and predict the ECG signals in the test set. Perform five such cross-validation experiments and take the average to obtain the arrhythmia classification results.

Images