Three-lead electrocardiogram diagnosis method based on multiple views and multiple models

Abstract

The invention provides a three-lead electrocardiogram diagnosis method based on multiple views and multiple models. Aiming at the problem of poor algorithm applicability in a low-resource environment of a few-lead electrocardiogram, multi-angle and multi-modal data are obtained by constructing a single-lead view, a multi-lead view and a patient information view; the method comprises the following steps of: extracting single-lead specificity information, multi-lead integrity information and patient personal semantic information respectively by using a lightweight recursive gating model, a multi-scale self-adaptive model and a self-encoder, finally fusing multi-view information, and balancing contribution of each view through multi-loss joint optimization. Verification is carried out in some public data sets, and the method has good classification performance and low complexity under the condition of 3 leads.

Description

Technical Field

[0001] This invention belongs to the field of machine learning and pattern recognition technology. It designs and trains a neural network for ECG diagnosis for a given 3-lead ECG signal, and then automatically diagnoses new ECG signals to detect all possible heart diseases. Background Technology

[0002] According to a report by the World Health Organization, cardiovascular disease is currently the leading cause of death worldwide, estimated to claim approximately 17.9 million lives annually. Cardiac arrhythmia is the most common and important cardiovascular disease, referring to a group of conditions where the heart beats too slowly, too fast, or irregularly. Cardiac arrhythmia is often present in the early stages of cardiovascular disease. If arrhythmia can be diagnosed promptly and accurately in the early stages of the disease, targeted treatment and management can be implemented in the early stages, significantly improving treatment effectiveness and patient survival rates.

[0003] An electrocardiogram (ECG) is a graphical representation of the heart's electrical activity. A normal ECG signal consists of different types of wave segments, including the P wave, QRS complex, T wave, and PR segment. Heart disease can affect the heart's electrical conductivity. ECGs can identify specific cardiac conditions based on wave characteristics and morphology, thus enabling health monitoring. However, accurately identifying these complex arrhythmia-related ECG rhythms requires considerable clinical experience and expertise. The introduction of computer-aided diagnosis aims to assist physicians in making accurate ECG diagnoses and improve efficiency. Currently, numerous computer-aided studies have been applied to ECG diagnosis, primarily using machine learning and deep learning methods.

[0004] In traditional machine learning, feature extraction is often an independent step. These features can be designed manually or obtained through some automatic feature extraction algorithms. Common features include P-QRS-T features and morphological features. Then, methods such as support vector machines and k-nearest neighbors are used to classify these features. Deep learning methods learn features and perform classification tasks directly from preprocessed ECG signals through end-to-end learning. They perform well in processing large-scale and complex ECG data and have gradually become the mainstream in ECG analysis. Based on the inherent properties of the main feature extractors in the neural network, deep learning models for diagnosing arrhythmias can be mainly divided into the following categories: (1) Convolutional Neural Network (CNN): Convolution operations can efficiently capture local patterns in ECG, such as waveforms and peaks, which helps to accurately identify cardiac events and abnormalities. (2) Recurrent Neural Network (RNN) or Transformer: Since ECG signals are time series, the temporal correlation within the signal is expected to better reveal the characteristics of arrhythmia categories. (3) Combination Model: Combining CNN, RNN or Transformer aims to make full use of the local patterns in ECG and the correlation of signals at different time periods.

[0005] However, most deep learning-based methods default to using 12-lead ECG data. These methods often prioritize improving diagnostic performance, neglecting the model's applicability in low-resource environments, such as its suitability with fewer leads, the number of parameters, and computational complexity. Using fewer leads can make ECG applications more widespread because ECG devices with fewer leads are relatively inexpensive, easy to maintain, and can be used for long-term dynamic recording via portable or wearable devices. However, fewer leads may also lead to the loss of some subtle ECG information. In some studies, using fewer leads has failed to fully explore the potential information in the remaining leads, resulting in less than ideal diagnostic performance. Summary of the Invention

[0006] To address the issue of poor applicability of algorithms in low-resource environments with fewer leads in ECG, this paper explores the use of deep learning in the case of 3-lead ECG and proposes a lightweight multi-view and multi-model-based ECG diagnostic method with fewer leads. This method can perform comprehensive diagnosis using detailed information from single leads of ECG, comprehensive information from multiple leads, and patient information with low complexity.

[0007] This invention discloses a three-lead electrocardiogram (ECG) diagnostic method based on multiple views and multiple models. Each single-lead ECG signal is treated as a view, and a recursive gating model is used to extract specific detailed features from each lead, learning the diversity information of each lead in the ECG. Multiple leads are treated as a whole as a view, and a multi-scale adaptive model is used to extract complementary features between cross leads, learning the integrity information of the multi-lead ECG. Patient information is treated as a view, and an autoencoder is used to vectorize metadata, learning the semantic information of the metadata. Finally, the information from multiple views is fused and subjected to joint optimization using multiple losses. The network's input is ECG signal data and patient personal information data, and the output is the probability of the patient's corresponding cardiac disease category.

[0008] The overall process and specific steps of this invention are as follows:

[0009] Step 1: Preprocess the 12-lead ECG signal;

[0010] (1) Reduce the sampling frequency of electrocardiogram (ECG) data to 250 Hz; (2) Since the length of each ECG data is different, all data are cropped to 20 s long. For original data that are longer than 20 s, the extra data is cropped, and for data that are shorter than 20 s, 0 is used to fill the gaps; (3) Perform 5-fold cross-validation on the ECG dataset for 5-fold cross-validation.

[0011] Step 2: Construct MVMMNet for 3-lead ECG signals;

[0012] MVMMNet mainly consists of multiple different views and corresponding models, a view fusion module, and multiple loss functions. MVMMNet starts with single-lead ECG signal views, multi-lead ECG signal views, and patient information views to acquire comprehensive patient data from multiple angles and modalities. For the single-lead view, a recursively gated model is used, employing recursively gated convolutions and recursively gated multilayer perceptrons to mine subtle information in the single-lead ECG signal and extract specific details. For the multi-lead view, a multi-scale adaptive model is used, employing inter-lead information interaction through cascaded depthwise separable convolutions to extract complementary information from multiple leads. For the patient information view, an autoencoder based on fully connected layers is used to model the semantic information of patient metadata. After extracting information from multiple views, the view fusion module fuses the view information. Finally, the final fused information, single-lead view information, and multi-lead view information are jointly optimized using three loss functions.

[0013] Step 3: Train the constructed MVMMNet model and predict the ECG signals in the test set. Perform five such cross-validation experiments and take the average to obtain the ECG diagnosis result.

[0014] Compared with traditional methods, the advantages of this invention are: (i) it does not require complex data processing and can perform end-to-end automatic diagnosis, which is convenient and fast; (ii) it comprehensively utilizes key information and patient personal information in fewer leads, integrates multi-view and multi-modal features, improves the diagnostic performance of the model, and the model has a low number of parameters and computational complexity, making it applicable to ECG monitoring devices with fewer leads. Attached Figure Description

[0015] Figure 1 This is a flowchart of the method;

[0016] Figure 2 This is a diagram of the neural network structure designed in this invention;

[0017] Figure 3 This is a structural diagram of the recursive gating model designed in this invention;

[0018] Figure 4 This is a structural diagram of the recursive gated convolution and recursive gated multilayer perceptron modules designed in this invention;

[0019] Figure 5 This is a structural diagram of the multi-scale adaptive model designed in this invention;

[0020] Figure 6 This is a structural diagram of the multi-scale residual convolution module designed in this invention;

[0021] Figure 7 This is a structural diagram of the adaptive selection module designed in this invention;

[0022] Figure 8 This is a structural diagram of the self-encoder designed in this invention; Detailed Implementation

[0023] To make the objectives, technical solutions, and advantages of this invention clearer, the following detailed descriptions of the various specific details involved in the technical invention are provided:

[0024] This invention proposes a 3-lead electrocardiogram (ECG) diagnostic method based on multiple views and multiple models. The ECG signal data used are from the 2018 Chinese Physiological Signal Challenge (2018CPSC) and the 2021 Computational Cardiology Challenge (202|CinC). The 2018CPSC dataset comes from 11 hospitals and contains 6,877 publicly available ECG records. Each record contains 12 standard leads, with a signal sampling frequency of 500 Hz and a duration of 6-60 seconds. This dataset includes 8 types of arrhythmias and 1 type of normal rhythm, for a total of 9 types of ECGs. The 2021CinC dataset consists of six sources from four countries across three continents. The official source has released more than 88,000 ECG records, each containing 12 standard leads. Due to the diverse data sources, the signal sampling frequency and duration vary. This dataset includes 133 diagnostic categories, of which 26 are relatively common and clinically significant for algorithm evaluation. Therefore, the scoring only evaluates the classification models for these 26 categories. Each ECG record in the dataset contains one or more labels, indicating that the patient has one or more arrhythmia conditions. The goal of the invention is to build a multi-label classification algorithm to correctly identify all types of heart disease in patients.

[0025] The overall process of this invention is as described in the appendix to the invention document. Figure 1 As shown, the specific implementation method is as follows:

[0026] Step 1: Preprocess the 3-lead ECG signal;

[0027] (1) Reduce the sampling frequency of ECG data to 250Hz; (2) Since the length of each ECG data is different, all data are cropped to 20s. For original data that exceeds 20s, the extra data is cropped, and for data that is not long enough, 0 is used to fill it; (3) Perform 5-fold cross-validation on the ECG dataset for 5-fold cross-validation.

[0028] Step 2: Construct MVMMNet for 3-lead ECG signals;

[0029] The structure of the proposed method (MVMMNet) is shown in the attached figure in the manual. Figure 2 As shown, the network mainly consists of multiple different views and corresponding models, view fusion, and multiple loss functions. For single-lead views, a recursive gating-based model is used to achieve high-order information interaction in single leads through continuous recursion and gating mechanisms, extracting specific detailed information in single leads. For multi-lead views, a multi-scale adaptive selection model is used to extract complementary information in multi-leads. For patient personal information views, an autoencoder is used to model the semantic information of patient metadata and fuse the information from multiple views. Then, three losses are used to jointly optimize the network for the final fused information, single-lead view information, and multi-lead view information.

[0030] The structure of the recursive gating model is shown in the attached diagram in the manual. Figure 3 As shown, the overall structure consists of four stages. In each stage, the Transformer structure suitable for temporal information modeling is adopted, namely LayerNorm+MHSA+LayerNorm+MLP. However, the multi-head self-attention is replaced with recursive gated convolution (RgConv). The main reasons are as follows: (1) The computational complexity of MHSA is proportional to the square of the sequence length, while the computational complexity of recursive gated convolution is proportional to the sequence length, thus reducing the computational complexity. (2) The introduction of the recursive gate mechanism enables the convolution to have the ability to adapt and mine subtle information, which is conducive to capturing specific details in a single lead. Similarly, MLP is replaced with recursive gated multilayer perceptron (RgMLP) in order to capture more complex feature patterns in high-dimensional space.

[0031] The structure of RgConv is shown in the attached diagram in the manual. Figure 4 As shown in a), assume the input feature map is First, a convolution operation is performed to double the number of channels of x. Then, the feature map is divided into p1, p2, p3, and p4 by channels (C / 4, C / 4, C / 2, C), with the number of channels as shown in equation (1). Then, p1 and p2, which has undergone depthwise separable convolution, are multiplied by a dot product and then convolved to obtain the first-order gated output y1, as shown in equation (2). The above operations are then performed sequentially to obtain the output y after recursive gated convolution, as shown in equations (3) and (4). The structure of RgMLP is shown in the attached diagram of the manual. Figure 4 As shown in b), its general process is the same as that of RgConv.

[0032] p1, p2, p3, p4 = Split(F conv (x)) (1)

[0033] y1=F conv (p1*F DWconv (p2)) (2)

[0034] y2=F conv (y1*F DWconv (p3)) (3)

[0035] y = F conv (y2*F DWcon (p4)) (4)

[0036] The attached diagram of the instruction manual Figure 5The structure of the multi-scale adaptive model (MSASNet) is shown. This model also uses the multi-stage idea in traditional vision models. The main structure consists of 4 stages, and each stage is composed of a multi-scale residual convolutional module (MSRCM). This module is mainly used to perform inter-channel information interaction in ECG and learn complementary information of the lead dimension.

[0037] The structure of the multi-scale residual convolution module is shown in the attached diagram in the instruction manual. Figure 6 As shown, different branches in this module represent different scales. The specific process is as follows: first, channel expansion is performed by ordinary convolution (Conv) with a kernel size of 1. Then, the feature map is divided into four groups in the channel dimension. Next, information interaction between channels and extraction of complementary lesion information in different channels are realized by depthwise separable convolution (DWConv) and inter-group concatenation. Then, the dimension is restored by ordinary convolution with a kernel size of 1. Finally, feature selection is performed by an adaptive selection module.

[0038] The structure of the Adaptive Selection Module (ASM) is shown in the attached diagram in the manual. Figure 7 As shown, feature maps from N different sources As input, the feature dimension is first reduced to 1 / sr of the original by using N ordinary convolutions with a kernel size of 1 to reduce the number of parameters. Then, the information from all sources is added and fused, as shown below.

[0039] X = F sum (F conv (x1),...,F conv (x n (5)

[0040] Then, in the branch above, global average pooling is first applied to the aggregated information X to generate channel-level information X. g Then, feature X is obtained by performing N ordinary convolutions with kernel size of 1 and concatenating the channels. c Then, the weights α of the features from different sources are obtained through the Softmax function, multiplied by the corresponding features and summed to obtain the source-selected features X. s This process can be represented as:

[0041] X g =F gap (X) (6)

[0042] X c =F concat (F conv (X g ),...,F conv (X g (7)

[0043] α = Softmax(X) c (8)

[0044]

[0045] Next, in the following branch, the feature dimension of the aggregated information X is reduced to 1 by using a regular convolution with a kernel size of 1 to generate time-step level information X. t Then, the weights β at different time steps are obtained through the Sigmoid function, and finally summed with X. s Multiplying these features yields the output feature map for dynamically selected sources and time steps. This process can be represented as:

[0046] X t =F conv (X) (10)

[0047] β = sigmoid(X) t (11)

[0048] Out = βX s (12)

[0049] As shown in the attached diagram of the instruction manual. Figure 8 The diagram shows an autoencoder that models patient information (the dataset only provides patients' gender and age information), mainly composed of multiple fully connected layers. First, the feature dimensionality is continuously increased through fully connected layers to learn higher-level and more abstract feature information. Then, the feature dimensionality is reduced to reconstruct the information, and finally, high-level semantic information of the patient information is output.

[0050] Finally, the information from multiple views is fused. First, the feature information from multiple single-lead views is fused. When there are too many leads, directly stitching features on the channel may result in an excessively large final feature dimension. Therefore, for multi-source information with consistent feature map size, an adaptive selection module (ASM) is used for adaptive selection fusion. Second, for multi-source information with inconsistent feature map size, including patient information, single-lead information, and multi-lead information, pooling is used to retain only the channel dimension information. Then, the patient's individual view information is fused into the single-lead and multi-lead features through channel stitching. Finally, the single-lead and multi-lead features are stitched together through the channel to obtain the final feature. In order to obtain more comprehensive information from different data perspectives, three loss functions are introduced for joint optimization of the final fused information, single-lead view information, and multi-lead view information. The total loss of the model is shown in Equation (13), which enables the model to effectively utilize data from different perspectives, balance the influence of each perspective, and thus improve the learning effect and robustness.

[0051] Loss = Loss a +Losss +Loss m =λ1ASL a +λ2ASL s +λ3ASL mn (13)

[0052] Where ASL is the asymmetric loss and λ is the adjustment factor.

[0053] Step 3: Optimize and train the constructed MSASNet model;

[0054] The loss function used is the asymmetric loss function (ASL), which is mainly used to solve the problem of imbalance between positive and negative samples in this multi-label classification task and to reduce the contribution of negative class loss. The calculation process is as follows:

[0055]

[0056] p m =max(pm, 0) (15)

[0057] Where L + For the loss of the positive class, L - The loss is for the negative class, p is the predicted probability of the model, and γ is the loss for the negative class. + For the positive class, γ is the focusing factor. - p is the focusing factor for the negative class, m is the offset of the negative class probability, and p is the focusing factor for the negative class. m This represents the negative class probability after the shift.

[0058] Then, predictions were made on the 3-lead ECG signals in the test set:

[0059] (1) Preprocess each 3-lead ECG signal recording in the test set using the method described in step one;

[0060] (2) Input the preprocessed data into the trained MVMMNet model, and after the model's forward propagation, obtain the probability vector P for each class. n ={p n 1 p n 2 , ..., p n k}, where n represents the nth ECG data point, w n k This represents the probability that the nth data item belongs to category k;

[0061] (3) Set a fixed classification threshold, if w n kIf the threshold is greater than 1, then the nth data point belongs to category k. This gives us all the categories to which the nth data point belongs, which means we have all the types of heart disease that the patient has.

[0062] Table 1. Comparison of classification performance of MVMMNet and advanced methods on the CPSC2018 dataset.

[0063]

[0064] Table 2 Comparison of classification performance of MVMMNet and advanced methods on the CinC2021 dataset

[0065]

[0066] Tables 1 and 2 show the comparison of the classification performance of the proposed model and state-of-the-art methods on the CPSC2018 and CinC2021 datasets, respectively. The proposed method, MVMMNet, uses multiple views and corresponding models to acquire multi-angle and multi-modal information from ECG. On the CPSC2018 dataset, MVMMNet achieves the best classification performance using the 3-lead method, with an F1 score of 0.837. Furthermore, MVMMNet outperforms most state-of-the-art methods using 12 leads, second only to PearsonUnet. On the CinC2021 dataset, MVMMNet also achieves the best classification performance using the 3-lead method, with a CS score of 0.726. MVMMNet again outperforms most state-of-the-art methods using 12 leads, with an F1 score second only to Multimodal. This demonstrates the effectiveness of the proposed model MVMMNet in 3-lead classification.

[0067] Table 3. Comparison of complexity between MVMMNet and state-of-the-art methods

[0068]

[0069] Table 3 shows the various complexity metrics of different models under the same machine environment settings. MVMMNet has the best classification performance among all models, and its network parameter count is 3.18M, ​​second only to MSASNet and Deform-CNN. Its computational complexity Flops is 0.47G, second only to MSASNet. Compared with the MSASNet network, the MVMMNet single-lead view recursive gating model uses fewer network layer iterations and channels, and the autoencoder layer is composed of multiple simple fully connected layers. Therefore, the parameter count and computational complexity have not increased much. The F1 and CS scores of the classification performance have both improved by 2%. However, due to the introduction of multi-loss joint optimization, the training speed is slightly slower, but the testing speed is faster. The above shows that MVMMNet is a lightweight and accurate ECG diagnostic model.

[0070] [1] DOSOVITSKIY A, BEYER L, KOLESNIKOV A, et al. An Image is Worth 16x16Words: Transformers for Image Recognition at Scale[A].arXiv, 2021[2024-04-13].http: / / arxiv.org / abs / 2010.11929.DOl:10.48550 / arXiv.2010.11929.

[0071] [2] QIN L, XIE Y, LIU

[0072] [3]GE R.Convolutional squeeze-and-excitation network for ECGarrhythmia detection[J].Artificial Intelligence In Medicine, 2021.

[0073] [4]HUANG Y,YEN G G,TSENG V S.Snippet Policy Network V2:Knee-GuidedNeuroevolution for Multi-Lead ECG Early Classification[J].IEEE Transactionson Neural Networks and Learning Systems.2024,35(2):2167-2181.DOI:10.1109 / TNNLS.2022.3187741.

[0074] [5]LE K H,PHAM H H,NGUYEN T B T,et al.LightX3ECG:A Lightweight andeXplainable Deep Learning System for 3-lead Electrocardiogram Classification[J].Biomedical Signal Processing and Control,2023,85:104963.DOI:10.1016 / j.bspc.2023.104963.

[0075] [6]JYOTISHI D,DANDAPAT S.An Attentive Spatio-Temporal Learning-BasedNetwork for Cardiovascular Disease Diagnosis[J].IEEE Transactions on Systems,Man,and Cybernetics:Systems,2023,53(8):4661-4671.DOI:10.1109 / TSMC.2023.3257022.

[0076] [7]JIANG S,LI D,ZHANG Y.A deep neural network based on multi-modeland multi-scale for arrhythmia classification[J].Biomedical Signal Processingand Control,2023,85:105060.DOI:10.1016 / j.bspc.2023.105060.

[0077] [8]RAN S,L1 ×,ZHAO B,et al.Label correlation embedding guidednetwork for multi-label ECG arrhythmla diagnosis[J].Knowledge-Based Systems,2023,270:110545.DOI:10.1016 / j.knosys.2023.110545.

[0078] [9]CHEN C,JIN B,CHE C,et al.OSGAN:Omni-scale and Global-aware ECGarrhythmia diagnostic network[J].Biomedical Signal Processing and Control,2024,96:106602.DOI:10.1016 / j.bspc.2024.106602.

[0079]

[10] YANG H,ZHAO Y,YANG X.PearsonUnet:ECG Classification ModelIntegrating Pearson Related Beats[J].IEEE Transactions on Instrumentation andMeasurement,2025,74:1-11.DOI:10.1109 / TIM.2025.3541691.

[0080]

[11] YU W,ZHOU P,YAN S,et al.InceptionNeXt:When Inception MeetsConvNeXt[C] / / Proceedings of the IEEE / CVF Conference on Computer Vision andPattern Recognition.2024:5672-5683.

[0081]

[12] HAN H,PARK S,MIN S,et al.Improving generalization performance ofelectrocardiogram classification models[J].Physiological Measurement,2023,44(5):054003.DOI:10.1088 / 1361-6579 / acb30f.

[0082]

[13] BORTOLAN G.3D ECG display with deep learning approach foridentification of cardiac abnormalities from a variable number of leads[J].Physiological Measurement,2023,44(2):024002.DOI:10.1088 / 1361-6579 / acb4dc.

[0083]

[14] KILADZE M R,LYAKHOVA U A,LYAKHOV P A,et al.Multimodal NeuralNetwork for Recognition of Cardiac Arrhythmias Based on 12-LoadElectrocardiogram Signals[J].2023,11。

Claims

1. A 3-lead electrocardiogram recognition method based on multiple views and multiple models, MVMMNet, characterized in that: The method includes the following steps; Step 1: Preprocess the ECG signal: (1) Reduce the sampling frequency of electrocardiogram (ECG) data to 250 Hz; (2) Since the length of each ECG data is different, all data are cropped to 20 s long. For original data that are longer than 20 s, the extra data is cropped, and for data that are shorter than 20 s, 0 is used to fill the gaps; (3) Perform 5-fold cross-validation on the ECG dataset for 5-fold cross-validation. Step 2: Construct MVMMNet for 3-lead ECG signals; MVMMNet mainly consists of multiple different views and corresponding models, a view fusion module, and multiple loss functions. Starting from single-lead ECG signal views, multi-lead ECG signal views, and patient information views, MVMMNet obtains comprehensive patient data from multiple angles and modalities. Then, for single-lead views, a recursive-gated model is used to mine subtle information in single-lead ECG signals through recursive-gated convolution and recursive-gated multilayer perceptron, extracting specific details in single leads; for multi-lead views, a multi-scale adaptive model is used to realize inter-lead information interaction through inter-group cascaded depthwise separable convolution, extracting complementary information in multi-leads. For the patient personal information view, an autoencoder based on a fully connected layer is used to model the semantic information of the patient metadata; After extracting multiple view information, the view information is fused through the view fusion module. Finally, the final fused information, single-lead view information and multi-lead view information are jointly optimized using three loss functions. Step 3: Train the constructed MVMMNet model and predict the ECG signals in the test set. Perform five such cross-validation experiments and take the average to obtain the ECG diagnosis result.

Images