Research on the application of Mamba-Transformer fusion model in ECG classification

Abstract

The application discloses an application research method of a Mamba-Transformer fusion model in ECG classification. In view of the problems of limited local receptive field, high computational complexity or insufficient global modeling capability of existing CNN, RNN and Transformer in processing long-term ECG signals, the application first introduces the Mamba architecture into the ECG classification field, and proposes a hybrid coding architecture of alternately stacking Mamba and Transformer modules. The Mamba module captures the local dynamic characteristics in the ECG signal with linear complexity, the Transformer module models the global dependency relationship, and the two are deeply fused through residual connection and layer normalization. The application also proposes a differentiated optimization strategy for unstable training of the hybrid model. The experimental results on the PTB-XL and MIT-BIH public data sets show that the method of the application is significantly better than existing baseline models in classification accuracy and computational efficiency, and provides a new technical scheme for clinical electrocardiogram auxiliary diagnosis and wearable device application.

Description

Technical Field

[0001] This invention belongs to the field of biomedical signal processing and artificial intelligence deep learning technology, specifically involving a research method for electrocardiogram (ECG) classification based on the Mamba and Transformer fusion model, which is applicable to clinical and wearable device scenarios such as arrhythmia detection, ECG-assisted diagnosis, and remote ECG monitoring. Background Technology

[0002] Electrocardiogram (ECG) signal classification is one of the core tasks in intelligent diagnosis of cardiovascular diseases. ECG signals have complex characteristics such as long duration, high dimensionality, strong nonlinearity, and the coexistence of local waveform morphology and long-range rhythm dependence. Existing deep learning methods face the following main problems when handling ECG classification tasks:

[0003] Convolutional Neural Networks (CNNs): They extract local features through one-dimensional convolution. Although they can effectively capture local morphologies such as QRS groups, their receptive field is limited, making it difficult to model the global dependencies of ultra-long sequences.

[0004] Recurrent Neural Networks (RNN / LSTM): Naturally suitable for time-series data, theoretically capable of capturing dependencies of arbitrary length, but suffer from gradient vanishing or exploding problems, and serial computation leads to low training efficiency.

[0005] Transformer model: It models global dependencies in one go through a self-attention mechanism, but its computational complexity is O(n²), and its memory consumption increases quadratically with the sequence length, making it difficult to handle long-term ECG signals on the order of minutes.

[0006] The Mamba model, proposed in recent years, is based on the Selective State Space Model (SSM), featuring linear complexity O(n) and content adaptation mechanisms, enabling efficient processing of extremely long sequences. However, the Mamba model alone still has limitations in its ability to model global dependencies, and its application in ECG classification is still in its infancy.

[0007] Therefore, there is an urgent need for a research method for ECG classification that combines local feature capture with global dependency modeling and is computationally efficient. Summary of the Invention

[0008] This invention addresses the core technical contradictions in existing ECG classification methods by proposing a novel solution. Specifically, while traditional CNN methods can effectively capture local waveform morphology in ECG signals (such as QRS complexes, P waves, and T waves), their receptive field is limited, making it difficult to model global long-range dependencies spanning several minutes or even longer. RNN / LSTM methods, while theoretically capable of handling sequences of arbitrary length, are prone to gradient vanishing or exploding problems during actual training, and their serial computation leads to low training efficiency. Transformer methods achieve powerful global modeling capabilities through self-attention mechanisms, but their computational complexity is O(n²), and memory and computational overhead explode quadratically with increasing sequence length, making them inefficient for handling the minute-long ECG records commonly encountered in clinical practice. While the Mamba model possesses linear complexity O(n) and a content adaptation mechanism, its ability to model global abstract dependencies is still insufficient when used alone. Therefore, this invention aims to provide an ECG classification application research method that balances local feature capture and global dependency modeling, offering high computational efficiency and stable training, thus overcoming the bottlenecks of existing technologies.

[0009] The core technical solution of this invention is to construct a hybrid encoder architecture based on alternating stacks of Mamba and Transformer blocks. This architecture breaks away from the traditional single-model design approach, achieving deep collaborative extraction of local dynamic features and global long-range dependencies. Specifically, the hybrid encoder consists of multiple Mamba Blocks and Transformer Blocks connected in series. The Mamba Block is based on the Selective State Space Model (SSM), whose core innovation lies in the fact that the state space parameters are no longer fixed constants, but functions of the input sequence. This allows the model to dynamically adjust its feature extraction strategy according to different regions of the ECG signal: "focusing" on extracting fine features in key morphological regions such as QRS complexes, and "quickly skipping" in noisy or stable regions, achieving efficient and accurate local dynamic modeling. The Transformer Block, based on a multi-head self-attention mechanism, performs global relationship modeling on the feature sequences output by the Mamba Block, capturing cross-timescale dependencies such as heart rate variability and long-range rhythm patterns. Each Block employs a standardized LayerNorm and residual connection design, which accelerates model convergence and ensures stable gradient propagation in the deep network. Furthermore, the architecture of this invention is highly flexible, supporting the free configuration of the number, stacking order, and module ratio of Mamba and TransformerBlocks according to specific task requirements, which greatly facilitates subsequent ablation experiments and structural optimization.

[0010] The specific implementation of this invention comprises four key steps. The first step is the preprocessing and embedding of the ECG signal. The original multi-lead or single-lead ECG signal is first subjected to baseline drift filtering (typically a 0.5–40Hz bandpass filter) to remove low-frequency noise and high-frequency interference; then, Z-score normalization is performed to standardize the signal amplitude distribution; finally, it is uniformly resampled to a fixed frequency (e.g., 250Hz) to ensure timing alignment. The preprocessed signal is mapped to a high-dimensional feature space through a linear projection layer to generate a serialized embedding representation, and optionally, learnable positional encoding is added to preserve temporal order information. The second step is the forward computation of the hybrid encoder. The embedded sequence is sequentially passed through alternately stacked Mamba Blocks and Transformer Blocks. Each Block, while preserving input information, progressively extracts features at different granularities. The Mamba Block leverages its linear complexity to efficiently scan long sequences and extract local features; the Transformer Block performs global interaction on small-scale feature maps, avoiding the quadratic computational overhead of directly processing ultra-long sequences. The third step is feature aggregation and classification output. The feature sequence output by the hybrid encoder is processed by a global average pooling layer, compressing the variable-length sequence into a fixed-dimensional feature vector. This operation reduces the number of parameters and enhances the model's robustness to sequence length. Finally, the feature vector is passed through one or more fully connected layers and a Softmax activation function to output the classification probability distribution of each ECG category. The fourth step is model training and optimization. This invention uses the AdamW optimizer combined with a cosine annealing learning rate scheduling strategy, using the cross-entropy loss function as the optimization objective. To address the training oscillation problem that may occur when Mamba and Transformer modules are cascaded, this invention proposes a set of differentiated optimization strategies: including setting differentiated initial learning rates for Mamba and Transformer modules, inserting an additional LayerNorm layer in the network to stabilize the feature distribution, using gradient pruning to prevent gradient explosion, and using mixed precision training to accelerate computation and save GPU memory. At the same time, an early stopping mechanism is introduced to prevent model overfitting and ensure that the model has good generalization ability.

[0011] This invention has three core innovations compared to existing technologies. First, it introduces the Mamba architecture into ECG classification research for the first time. Previously, no work systematically explored the applicability of Mamba in long-sequence ECG classification tasks. This invention fills this technological gap, verifying the significant advantages of Mamba's linear complexity and content adaptation mechanism in processing long-term clinical ECG signals, providing a new technological paradigm for this field. Second, it proposes and implements an alternating stacking fusion architecture of Mamba and Transformer. Unlike simple feature concatenation or serial cascading, the alternating stacking design of this invention allows for dynamic fusion of local features and global dependencies at each layer of the network. Local information can be enhanced with the help of global context, and global abstraction can be continuously refined from fine-grained local features, achieving true deep collaborative modeling. Third, it proposes a systematic optimization scheme to address the training instability problem of hybrid models. Through multiple techniques such as differentiated learning rates, additional LayerNorm, gradient pruning, and mixed-precision training, this invention effectively solves the gradient flow mismatch and loss oscillation problems caused by direct concatenation of heterogeneous modules, ensuring reliable model convergence. In terms of technical advantages, this invention combines linear computational complexity (O(n)), constant-level inference memory usage (O(1)), powerful long-range modeling capabilities, and flexible module configurability. It significantly outperforms CNN, LSTM, Transformer, and pure Mamba baselines in both classification accuracy and computational efficiency.

[0012] This invention can be widely applied to various ECG-assisted diagnostic scenarios. In clinical settings, it can be used for tasks such as arrhythmia classification, myocardial ischemia identification, and ventricular hypertrophy detection on 12-lead static ECG. In dynamic ECG monitoring scenarios, it can perform end-to-end analysis of long-term ECG signals recorded by Holter, without the need for manual truncation or segmentation. In wearable devices and edge computing scenarios, leveraging the constant-level inference memory usage of the Mamba module (O(1),) this invention's method can be lightweightly deployed on resource-constrained devices such as portable ECG monitors and smartwatches to achieve real-time ECG analysis. In terms of experimental verification, this invention achieved a classification performance of Macro-F1 of 0.624 and AUC of 0.943 on the PTB-XL public dataset (21,799 10-second 12-lead records), significantly outperforming the pure CNN baseline (Macro-F1=0.586) and the pure Transformer baseline (Macro-F1=0.601). On the MIT-BIH arrhythmia dataset, for long recordings of approximately 30 minutes, this invention also achieved classification results superior to existing baselines. Ablation experiments further confirmed the necessity of the alternating stacking design of Mamba and Transformer, and the crucial role of optimization strategies such as additional LayerNorm in training stability. In summary, this invention provides a novel technical solution for intelligent analysis of electrocardiogram signals that combines high accuracy, high efficiency, and high flexibility, possessing significant clinical application value and industrialization prospects. Attached Figure Description

[0013] Figure 1 Schematic diagram of the Mamba-Transformer hybrid encoder structure.

Claims

1. A research method for applying the Mamba-Transformer fusion model to ECG classification, characterized in that, Includes the following steps: Preprocessing and sequence embedding of the raw ECG signal; The embedded sequence is input into a hybrid encoder consisting of alternating stacks of Mamba Blocks and Transformer Blocks, which performs collaborative extraction of local features and global dependencies. ECG classification results are output through global average pooling and fully connected layers.

2. The method according to claim 1, characterized in that, The Mamba Block employs a selective state-space model, where the state-space parameters are functions of the input sequence, enabling content-adaptive local dynamic feature capture.

3. The method according to claim 1, characterized in that, The hybrid encoder supports flexible configuration of the number, stacking order, and module ratio of Mamba and Transformer Blocks for structural optimization and ablation verification.

4. The method according to claim 1, characterized in that, During training, additional LayerNorm layers, differential parameter initialization, gradient pruning, and mixed-precision training are introduced to improve training stability when heterogeneous modules are cascaded.

5. The method according to claim 1, characterized in that, The ECG signal includes single-lead or multi-lead signals and is applicable to publicly available datasets such as PTB-XL and MIT-BIH, as well as clinically collected data.

6. The method according to claim 1, characterized in that, The method is applied to at least one of the following ECG-assisted diagnostic tasks: arrhythmia classification, heartbeat detection, heart rate variability analysis, and myocardial ischemia identification.

7. The method according to claim 1, characterized in that, The model supports deployment on edge computing devices and utilizes the O(1) constant-level inference memory footprint of the Mamba module to achieve lightweight inference.

8. An ECG classification system based on the method of any one of claims 1 to 7, characterized in that, include: Data preprocessing module: used for filtering, normalizing, and resampling ECG signals; Hybrid encoding module: Consists of alternating stacks of Mamba Block and Transformer Block; Classification output module: used for feature aggregation and class probability output.

Images