Long sequence electrocardiogram signal disease recognition system based on transformer architecture

Abstract

The application relates to the technical field of electrocardiosignal analysis, and discloses a long-sequence electrocardiosignal disease recognition system based on a Transformer architecture; the application overcomes the core defects of fixed window segmentation in traditional electrocardio analysis, such as damaging waveform integrity, neglecting lead spatial topological relationship and low efficiency of long-sequence modeling, the P-QRS-T waveform structure is completely reserved through a heart beat self-adaptive segmentation technology, and the risk of fixed window truncation is eliminated; the lead anatomical topological relationship and space-time coding are fused, and multi-lead electro-physiological correlation is modeled; long-sequence efficient processing is realized based on hierarchical sparse Transformer, the recognition sensitivity of complex pathologies such as arrhythmia and myocardial ischemia is improved; gradient directional regulation and flow processing mechanism are combined, the clinical real-time demand is met while the diagnostic accuracy is ensured, and finally, the light-weight deployment is adapted to a mobile terminal, thereby providing reliable, efficient and universal intelligent decision support for early warning of heart diseases.

Description

Technical Field

[0001] This invention relates to the field of electrocardiogram (ECG) signal analysis technology, specifically to a long-sequence ECG signal disease identification system based on the Transformer architecture. Background Technology

[0002] Current ECG signal analysis techniques mainly rely on fixed-length window segmentation strategies and local feature extraction models. For example, they use a sliding window of preset duration to extract signal segments and perform segment-by-segment classification based on convolutional neural networks or recurrent neural networks. While such methods can capture the morphological features of a single heartbeat, they are limited by the rigid segmentation mechanism of the fixed window and cannot dynamically adjust the analysis interval according to the patient's actual heart rate, resulting in the forced separation of pathological correlations across heartbeats. At the same time, traditional models typically use channel stacking or simple pooling to process multi-lead signals, ignoring the physiological spatial topological relationships between leads, making the fusion of spatial dimension information superficial.

[0003] However, existing technologies have the following drawbacks: First, fixed window segmentation disrupts the physiological integrity of ECG waveforms, especially in cases with large RR interval variability such as arrhythmias, easily causing P / T wave truncation or aliasing, significantly reducing the recognizability of key waveform features; Second, the ability to model spatiotemporal features of multi-lead systems is weak, failing to encode the anatomical positional relationships of the standard 12-lead system (such as the angle between limb lead vectors and the spatial coordinates of chest leads) into the feature learning process, thus limiting the depth of spatial correlation mining; Third, long sequence computation is inefficient, the sequence dependency characteristics of traditional RNNs cannot process ultra-long ECG records in parallel, and the layer stacking of CNNs makes it difficult to model minute-level long-term dependencies due to limited receptive fields, failing to meet the needs of real-time clinical analysis. Therefore, a long sequence ECG signal disease recognition system based on the Transformer architecture is proposed. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a long-sequence electrocardiogram (ECG) signal disease identification system based on the Transformer architecture, thereby resolving the problems mentioned in the background.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a long-sequence electrocardiogram signal disease recognition system based on the Transformer architecture, comprising:

[0006] (a) Multi-lead signal preprocessing: Configured to receive raw multi-lead ECG signals, baseline drift is eliminated by a low-frequency cutoff high-pass filter, power frequency notch filter is used to eliminate power supply interference, and each lead signal is normalized to zero mean and unit variance by z-score normalization.

[0007]

[0008] in denotes the original sampling value of the i-th lead at time t (i = 1, 2,..., 12), μ i and σ i are its mean and standard deviation, respectively;

[0009] (b) Heartbeat cycle adaptive segmentation: the normalized signal output by step (a) is dynamically segmented into sub-segments based on R-wave detection, and the sub-segment length L is determined by physiological parameters:

[0010]

[0011] wherein is the mean of historical RR intervals; a is a preset proportion coefficient; f s is the sampling rate and satisfies f s ≥ 250 Hz;

[0012] The adjacent sub-segments are set with a preset proportion of overlap to generate a sequence of sub-segments that are continuous in time;

[0013] (c) Spatio-temporal embedding generation: the sub-segment features are extracted by a convolutional network to generate an embedding vector After superimposing the learnable position encoding, a spatio-temporal embedding vector is formed;

[0014] (d) Multi-lead spatio-temporal coding: the spatio-temporal embedding vectors of each lead are arranged in a time-lead two-dimensional matrix, wherein the time dimension position encoding E time is generated by a trainable sine function, and the lead dimension position encoding E lead is initialized according to the standard lead anatomical topology to construct a joint spatio-temporal coding matrix;

[0015] (e) Hierarchical sparse Transformer processing: the spatio-temporal coding matrix is input into a multi-layer improved encoder, and each layer performs:

[0016] Local block full attention: taking the number of sampling points covering a single heartbeat cycle as a basic block, multi-head self-attention is calculated within the block;

[0017] Global key frame aggregation: key frames are dynamically selected according to signal features, global context nodes are generated by pooling operation and interacted with each block to output deep feature representation;

[0018] (f) Disease classification output: the hidden state vector of the [CLS] flag in the final layer of the Transformer is extracted to output the probability distribution of multiple cardiac diseases.

[0019] Preferably, the heartbeat cycle adaptive segmentation of step (b) further comprises the following contents:

[0020] The R-wave detection adopts a differential-threshold joint algorithm: if the signal point s tsatisfy and simultaneously satisfy then determine that the time t is the R-wave peak position, wherein κ is a differential threshold coefficient, λ is an amplitude threshold coefficient, β, γ are analysis window starting indexes, δ, ∈ are amplitude detection range indexes;

[0021] The subsegment boundary is extended forward and backward based on the R-wave position, ensuring that each subsegment contains an integer number of complete cardiac cycles.

[0022] Preferably, the lead position code E lead The initialization method includes differentiated processing of limb leads and chest leads:

[0023] The limb leads generate codes according to vector angles based on the Einthoven triangle theory:

[0024] E lead (I) = cos(θ I )·P x + sin(θ I )·P y

[0025] E lead (II) = cos(θ II )·P x + sin(θ II )·P y

[0026] where θ I and θ II are standard vector angles of the lead, P x and P y are trainable projection basis vectors;

[0027] The chest leads are initialized according to human anatomical coordinates:

[0028]

[0029] where x j is the anatomical coordinate vector of lead j, W coord is a trainable weight matrix, and b is a bias vector.

[0030] Preferably, the global key frame aggregation operation of step (e) includes:

[0031] Dynamic selection mechanism of key frames: when the QRS complex energy of a certain signal block exceeds twice the global average value, it is forced to be a key frame, and the energy calculation formula is:

[0032]

[0033] where Eblock is the signal block energy, is the global average energy, and η is the energy threshold coefficient;

[0034] The generation of the global context node G adopts the max-pooling operation:

[0035]

[0036] wherein is the pooling function, is the normalization function, and H k is the kth key frame hidden state vector, is the key frame index set.

[0037] Preferably, the system comprises a gradient directional regulation module, which performs the following operations in the model training phase:

[0038] The gradient of the time step corresponding to the QRS complex is amplified to strengthen the sensitivity of the model to the electrocardiogram key features:

[0039]

[0040] wherein ρ is the gradient amplification coefficient, is the parameter gradient corresponding to the QRS period;

[0041] The dynamic gradient clipping strategy prevents training divergence: if the gradient norm of the parameter W exceeds the threshold τ, it is scaled by a certain proportion:

[0042]

[0043] wherein τ is the gradient norm threshold, is the L2 norm of the gradient vector.

[0044] Preferably, the disease classification of step (f) adopts a multi-task joint loss function, which comprises:

[0045] The main loss function is the frequency-weighted cross-entropy, which solves the class imbalance problem:

[0046]

[0047] wherein f c is the frequency of class c in the training set, ν is the class balance coefficient; C is the total number of disease classes; y c is the true label of class c; p c is the predicted probability of class c;

[0048] The time sequence consistency loss function constrains the prediction smoothness of adjacent sub-sections:

[0049]

[0050] where T is the total number of subsegments, p t is the prediction probability vector of the t-th subsegment;

[0051] The total loss function is the weighted sum:

[0052]

[0053] where ξ is the regularization weight coefficient.

[0054] Preferably, the system supports real-time streaming processing, realized by a hidden state caching mechanism:

[0055] Continuously cache the hidden state matrices of the 6th and 12th layers for the history subsegments

[0056] and where M is the number of processed subsegments, D is the dimension of hidden state;

[0057] When a new subsegment X new is inputted, the hidden state of the first 6 layers is taken as the input of the first 6 layers of the Transformer, and only the output of the 7th to 12th layers is calculated for the new subsegment.

[0058] The Least Recently Used (LRU) strategy is adopted to manage the cache: when the number of cached subsegments exceeds a predetermined threshold N max , the hidden states corresponding to the K oldest subsegments are removed.

[0059] Preferably, the preprocessing stage integrates a multi-lead cooperative denoising module, including:

[0060] Motion artifact detection: calculate the high-frequency band energy E HF of the signal =∑(s*h HPF ) 2 , if E HF > ζ·σ HF , it is determined as a noise segment, where σ HF is the standard deviation of the high-frequency energy of the whole sequence, s is the time series of the lead signal, h HPF is the impulse response of the high-pass filter, and ζ is a predetermined coefficient (positive real number);

[0061] Signal repair based on lead spatial relationship: if lead i is determined to be damaged, its signal is reconstructed using healthy leads:

[0062]

[0063] where is the reconstructed signal of lead i at time t, Ω is the set of healthy leads, d ijis the standard electrode distance between lead i and j (according to AHA lead position map), p is the distance decay index (positive integer), w ij is the repair weight of lead j to lead i.

[0064] Preferably, the system deployment phase adopts a model compression framework with electrocardiogram feature perception, containing three-level optimization:

[0065] Structural compression: replace the original 12-layer Transformer with a 6-layer encoder-decoder architecture, in which:

[0066] The encoder layer retains the local-global attention mechanism;

[0067] The decoder layer adopts lightweight linear attention, with complexity reduced to O(N);

[0068] Physiological-guided weight pruning:

[0069] Calculate the saliency map S of each attention head to QRS wave features h , if , then prune the head, where S h is the saliency score of the hth attention head, H is the total number of attention heads, and φ is a predetermined threshold coefficient;

[0070] Pathological knowledge distillation:

[0071] The teacher model (original 12 layers) guides the output probability distribution of the student model (6 layers):

[0072]

[0073] where is the logits output of the teacher model for class c, is the logits output of the student model for class c, T temp is the temperature coefficient, and σ is the softmax function.

[0074] Compared with the prior art, the present application has the following beneficial effects:

[0075] The present application overcomes the core defects of fixed window segmentation in traditional electrocardio analysis, such as destroying waveform integrity, ignoring lead space topological relationship and low efficiency of long sequence modeling, retains P-QRS-T waveform structure through heart beat adaptive segmentation technology, eliminates fixed window truncation risk; fuses lead anatomical topology and space-time coding, models multi-lead electro-physiological correlation; realizes efficient processing of long sequence based on hierarchical sparse Transformer, improves recognition sensitivity of complex pathologies such as arrhythmia and myocardial ischemia; combines gradient directional regulation and streaming processing mechanism, guarantees diagnostic accuracy while meeting clinical real-time requirements, finally through lightweight deployment adaptation to mobile terminals, provides reliable, efficient and universal intelligent decision support for early warning of heart diseases.

[0076] Other features and advantages of the present application will be set forth in the following description, and in part will become apparent to those skilled in the art from the description, or can be learned by practice of the application. The objects and other advantages of the application will be realized and attained by the structure particularly pointed out in the written description and claims thereof as well as the appended drawings. BRIEF DESCRIPTION OF DRAWINGS

[0077] Figure 1 The system overall processing flowchart of the present application is shown in the figure;

[0078] Figure 2 The heart beat cycle adaptive segmentation flowchart of the present application is shown in the figure;

[0079] Figure 3 The space-time embedding generation flowchart of the present application is shown in the figure;

[0080] Figure 4 The hierarchical sparse Transformer processing flowchart of the present application is shown in the figure;

[0081] Figure 5 The gradient directional regulation training flowchart of the present application is shown in the figure;

[0082] Figure 6 The model compression deployment flowchart of the present application is shown in the figure;

[0083] Figure 7 The real-time streaming processing flowchart of the present application is shown in the figure. DETAILED DESCRIPTION

[0084] The technical solutions in the embodiments of the present application will be described clearly and completely below with reference to the drawings in the embodiments of the present application. Obviously, the described embodiments are only part of the embodiments of the present application, rather than all the embodiments. Based on the embodiments in the present application, all other embodiments obtained by those skilled in the art without creative labor fall within the scope of protection of the present application.

[0085] Please refer to Figures 1-7The long sequence electrocardio signal disease recognition system based on the Transformer architecture in the application comprises:

[0086] Step 1: Multi-lead signal preprocessing

[0087] This step carries out standardized preprocessing on the original 12-lead electrocardio signal, eliminates noise interference and unifies the signal scale, to provide high-quality input data for subsequent processing. The preprocessing includes three core operations: baseline drift elimination, power frequency interference elimination and signal standardization, to ensure that all lead signals have the characteristics of zero mean and unit variance.

[0088] 1. Baseline drift elimination:

[0089] A second-order Butterworth high-pass filter with a cutoff frequency of 0.5 Hz is used;

[0090] Low-frequency components are eliminated by sliding average filtering:

[0091]

[0092] Where y t is the output signal after filtering at time point t, s t is the original input signal at time point t, N is the sliding window length (number of sampling points), and k is the sampling point index in the window;

[0093] Low-frequency baseline drift caused by respiratory motion (0.15-0.3 Hz) and body movement is eliminated.

[0094] 2. Power frequency interference elimination:

[0095] A 50Hz (or 60Hz) notch filter is used.

[0096] Transfer function:

[0097]

[0098] Where H(z) is the transfer function of the filter, z- 1 is the unit delay operator, ω0 is the normalized angular frequency of the target interference frequency, and r is the pole radius,

[0099] Deep attenuation is achieved at a specific frequency point to eliminate power supply interference and its harmonics.

[0100] 3. Signal standardization:

[0101] The z-score standardization formula is used to eliminate individual differences of each lead signal, so that all lead signals have the characteristics of zero mean and unit variance:

[0102]

[0103] where Xi(t) represents the original sampling value of the ith lead at time t (i = 1, 2,..., 12), μi i and σi i are its mean and standard deviation, respectively;

[0104] Calculation rules:

[0105] Mean calculation window: 5-second sliding window

[0106] Standard deviation calculation: global calculation

[0107] Eliminate signal DC offset, center the waveform to zero; unify the signal amplitude range, solve the gain difference of different leads, eliminate individual differences, and make all lead signals have zero mean and unit variance.

[0108] The pseudo code is as follows:

[0109]

[0110]

[0111] The steps are as follows:

[0112] Initialize the coefficients of 0.5Hz high-pass filter and 50Hz notch filter;

[0113] Process each lead independently (12-lead loop);

[0114] Apply high-pass filter to eliminate baseline drift;

[0115] Apply notch filter to eliminate power frequency interference;

[0116] Calculate the mean and standard deviation of the lead signal;

[0117] Perform z-score standardization: (signal value-mean) / standard deviation;

[0118] Return the processed 12-lead signal matrix.

[0119] Example:

[0120] Input: Lead II original signal segment [0.15, 1.32, -0.28, 2.15, 0.98] mV;

[0121] Processing process:

[0122] Baseline filtered: [0.18, 1.30, -0.25, 2.12, 1.02] (eliminate 0.3Hz drift);

[0123] After power frequency filtering: [0.17, 1.31, -0.26, 2.13, 1.01] (eliminating 50Hz interference);

[0124] Standardization (μ) i =0.87,σ i =0.92): [-0.76, 0.48, -1.23, 1.37, 0.15];

[0125] Output characteristics: All values ​​are distributed between -1.23 and 1.37, with a mean of 0 and a standard deviation of 1.

[0126] This step eliminates baseline drift and power line interference in the ECG signal through three levels of processing, and then eliminates individual differences through z-score standardization, outputting a standardized 12-lead signal with zero mean and unit variance. This processing provides clean input that conforms to physiological characteristics for subsequent dynamic segmentation, ensuring the comparability of signals from different patients and different devices.

[0127] Step 2: Adaptive segmentation of the cardiac cycle

[0128] This step involves dynamically segmenting the ECG signal based on R-wave detection, adaptively determining the segment length according to the patient's real-time heart rate to ensure that each segment contains a complete cardiac cycle. It includes two core operations: precise R-wave localization and dynamic segmentation.

[0129] 1. R-wave detection: A differential-threshold joint algorithm is used to accurately locate the R-wave peak position.

[0130]

[0131] Amplitude conditions:

[0132]

[0133] Determine the position of the R-wave peak at time t;

[0134] Where κ is the differential threshold coefficient (which can be set to the range of 0.5-0.8), λ is the amplitude threshold coefficient (which can be set to the range of 0.6-0.9), β,γ are the starting index of the analysis window, and δ,∈ is the amplitude detection range index;

[0135] It captures the steep rising edge of the R-wave and dynamically adapts to changes in signal amplitude; points that simultaneously meet the slope and amplitude conditions are identified as R-wave peaks, with a minimum R-wave interval of 200ms to avoid artifacts.

[0136] 2. Dynamic segmentation:

[0137] Length calculation:

[0138]

[0139] in is the average of historical RR intervals; a is a preset proportion coefficient; f s is the sampling rate and satisfies f s ≥ 250 Hz.

[0140] Boundary determination:

[0141] Take the R-wave peak as the center point, expand 0.4L forward and 0.6L backward, and ensure to contain the complete P-QRS-T complex.

[0142] Overlap setting:

[0143] Set the proportion of adjacent subsegments as 30%, and set the start point of the overlap region as, for example, the position of the previous R-wave + 0.7L, to generate a sequence of time-continuous subsegments.

[0144] The pseudo code is as follows:

[0145]

[0146]

[0147]

[0148] The steps are as follows:

[0149] Initialize the R-wave position list and the minimum R-wave interval (200 ms);

[0150] Slide the window to traverse the signal (keep a 300-point buffer at the boundary);

[0151] Calculate the maximum difference value in the range of 150 points before and after the current point;

[0152] Calculate the maximum amplitude in the range of 300 points before and after the current point;

[0153] Check whether the slope condition and the amplitude condition are both met;

[0154] If the conditions are met and the interval from the previous R-wave is > 200 ms, record it as an R-wave;

[0155] Calculate the average of the last 10 RR intervals (use the default 0.8 s if there are less than 10); calculate the subsegment length L according to the formula;

[0156] Generate a subsegment (40% in front and 60% in back) with each R-wave as the center;

[0157] Add a subsegment with 30% overlap between adjacent R-waves;

[0158] Return the sequence of subsegments.

[0159] Example:

[0160] Patient parameter: average heart rate

[0161] Sampling rate: f s = 500 Hz;

[0162] Subsegment length: Sampling points;

[0163] R-wave position: t = 1000;

[0164] Subsegment range:

[0165] Main segment: [1000-192, 1000+288] = [808, 1288];

[0166] Overlap segment: previous R-wave t = 400 → [400+336, 400+336+480] = [736, 1216];

[0167] Coverage integrity: ensure to contain P-wave onset to T-wave end.

[0168] This step accurately locates the R-wave by the differential-threshold joint algorithm, dynamically calculates the subsegment length according to the real-time heart rate, generates a subsegment sequence covering the complete cardiac cycle centered on the R-wave, and sets a 30% overlap area to ensure time continuity. This adaptive segmentation strategy effectively solves the cardiac truncation problem caused by fixed windows and provides physiological structure complete input for subsequent feature extraction.

[0169] Step 3: Spatiotemporal embedding generation

[0170] This step converts the cardiac electrosegment into a high-dimensional feature vector, fuses local morphological features and spatiotemporal position information, and forms an embedding representation suitable for Transformer processing. It includes three core operations: convolution feature extraction, position encoding generation, and feature fusion:

[0171] 1. Convolution feature extraction:

[0172] Network structure:

[0173]

[0174] Output:

[0175] 2. Position encoding generation:

[0176] Temporal position encoding: generate using trainable sine function

[0177]

[0178]

[0179] where Wtime where W is the trainable weight matrix, pos is the subsegment time position index, i is the dimension index, D is the embedding dimension, and r is the wavelength base.

[0180] Lead position encoding:

[0181] Limb leads (I, II, III, aVR, aVL, aVF) are generated according to the Einthoven triangle theory according to the vector angle encoding:

[0182] E lead (I) = cos(θ I ) · P x + sin(θ I ) · P y

[0183] E lead (II) = cos(θ II ) · P x + sin(θ II ) · P y

[0184] where θ I and θ II are the standard vector angles of the lead, P x and P y are trainable projection basis vectors.

[0185] Chest leads (V1-V6) are initialized according to human anatomical coordinates:

[0186]

[0187] where x j is the anatomical coordinate vector of lead j, W coord is the trainable weight matrix, and b is the bias vector.

[0188] 3. Feature fusion:

[0189] Fusion of morphological features, time sequence positions and spatial topological information:

[0190] E final = E emb + E time + E lead

[0191] Preserve the original information of each feature to avoid interaction interference.

[0192] After fusion, perform layer normalization (LayerNorm) to stabilize the training process through normalization, and output the dimension.

[0193] The pseudo code is as follows:

[0194]

[0195]

[0196]

[0197] The steps are:

[0198] Reshape the input sub-segment into a three-dimensional tensor (1 x length x 1);

[0199] First convolutional layer: 64 filters of length 15, step 2, extract QRS wave features;

[0200] ReLU activation followed by max pooling of length 3;

[0201] Second convolutional layer: 128 filters of length 7, step 2, extract ST-T features;

[0202] Global average pooling to get a 128-dimensional embedding vector E emb ;

[0203] Generate time position encoding: generate sine and cosine encoding according to the original method of Transformer;

[0204] Generate lead position encoding according to lead type:

[0205] Limb leads: calculate according to standard vector angle projection;

[0206] Chest leads: linear transformation according to anatomical coordinates;

[0207] Add three parts of features: E emb + E time + E lead Form a spatio-temporal embedding vector;

[0208] Output the final embedding after LayerNorm normalization.

[0209] Example explanation:

[0210] Input: V4 lead sub-segment (480 points), time sequence position index = 5, wavelength base r is 10000;

[0211] Processing:

[0212] Convolutional network output E emb = [0.12, -0.35,..., 0.08];

[0213] Time encoding calculation:

[0214] E time [0] = sin(5 / 10000 0 ) ≈ 0.001

[0215] E time [1] = cos(5 / 10000) 0 )≈0.999

[0216] ...(128-dimensional vector)

[0217] Lead coding (V4 coordinates (0.2, 0.5, 0)):

[0218] Feature fusion: [0.12+0.001+0.25,-0.35+0.999-0.12,...,0.08+...];

[0219] Layer normalized output: [-0.15, 0.78, ..., 0.23];

[0220] Output: 128-dimensional normalized vector (preserving spatiotemporal topology).

[0221] Construction of joint spatiotemporal coding matrix:

[0222] Arrange all segment spatiotemporal embedding vectors of the 12 leads into a two-dimensional matrix according to the time dimension (segment sequence) and the lead dimension (anatomical location): Where 12 represents the number of standard leads and 128 represents the embedding dimension;

[0223] Each element M[t,i,:] in the matrix represents the fusion embedding vector E of the t-th sub-segment of the i-th lead. final This matrix simultaneously encodes the ECG temporal evolution and lead spatial topology, serving as input to the Transformer.

[0224] This step extracts local morphological features of cardiac electron segments using a two-layer convolutional network, and combines temporal location encoding and lead anatomical location encoding to generate a 128-dimensional embedding vector that integrates spatiotemporal information. This representation preserves the physiological structural features of the ECG signal, providing the Transformer with a structured input that includes spatial topology and temporal relationships.

[0225] Step 4: Hierarchical Sparse Transformer Processing

[0226] This step employs an improved Transformer architecture to handle spatiotemporal embedding, achieving efficient modeling of long sequences through local block attention and global keyframe aggregation. It includes four core operations: block processing, local attention, keyframe selection, and global interaction.

[0227] 1. Block processing:

[0228] Segmentation rule: Segmented according to the single heartbeat cycle (seconds).

[0229]

[0230] where f s is the sampling rate, T heartbeat is the single heart beat period (about 0.25-1.2 seconds);

[0231] Set the inter-block overlap to 10%, the input matrix is divided into N BxD blocks.

[0232] 2. Local block attention:

[0233] Attention mechanism:

[0234]

[0235] where Q, K, V are query, key, value matrices generated by linear transformation within blocks, d k is the dimension of key vector;

[0236] Multi-head setting: 8 heads, each head dimension d k = 16;

[0237] Position encoding: relative position encoding within blocks;

[0238] Add a learnable relative position encoding matrix within blocks

[0239]

[0240] where z is the maximum relative distance (default z = 50), i, j are the indices of time points within blocks, learned embed is the trainable relative position embedding matrix;

[0241] Generate position-aware embeddings for each time point within blocks, so that the model can perceive the local temporal relationship of the waveform.

[0242] 3. Key frame selection:

[0243] Select key frames based on QRS energy, focusing on areas where pathological features are significant.

[0244] Selection conditions:

[0245] When the QRS complex energy of a signal block exceeds twice the global average, it is forced to be a key frame. The energy calculation formula is:

[0246]

[0247] where E block is the signal block energy, η is the energy threshold coefficient;

[0248] 4. Global-local interaction:

[0249] The generation of global context node G adopts max-pooling operation:

[0250]

[0251] wherein is the pooling function, is the normalization function, H k is the k-th key frame hidden state vector, is the key frame index set;

[0252] Stable feature distribution, avoid amplitude difference interference, capture the most significant pathological features (such as ST segment elevation), output the key information summary representing the entire sequence;

[0253] Global-local interaction formula:

[0254] Realize the attention interaction between global node and local feature:

[0255]

[0256] wherein Qlocal is the query vector of local block, focusing on the current feature, G T is the key information of global node;

[0257] Attention weight: adjust the importance of local features (such as enhance ischemia-related features);

[0258] Residual connection: retain the original local feature (Output = LocalAttn + GlobalAttn).

[0259] In hierarchical sparse Transformer processing, the joint spatio-temporal encoding matrix is used as the input sequence of the encoder. In order to capture the global context, a learnable CLS flag embedding vector (dimension same as spatio-temporal embedding) is added at the first position of the input sequence. The flag participates in local block attention and global key frame aggregation calculation with the sequence, and its final layer hidden state vector will be used as the compressed representation of the entire electrocardiogram sequence.

[0260] The pseudo code is as follows:

[0261]

[0262]

[0263]

[0264]

[0265] The specific steps are:

[0266] Add learnable CLS flag embedding vector;

[0267] Calculate single beat block size;

[0268] Divide the input sequence into overlapping 10% blocks;

[0269] Add relative position encoding for each block;

[0270] Perform 8-head self-attention calculation (Q=K=V);

[0271] Calculate the energy E of each block block ;

[0272] Select blocks with energy > 2 times the global average as key frames;

[0273] Perform layer normalization on key frames;

[0274] Max-pooling along the time dimension to generate global node G;

[0275] For each block:

[0276] Take the center point as the query vector Q;

[0277] Calculate the attention weight of Q and G;

[0278] Generate global attention features;

[0279] Residual connection with original block features.

[0280] Recombine all blocks into complete sequence output.

[0281] Example explanation

[0282] Input:

[0283] 12-lead x 480-time-point x 128-dimensional embedding matrix (f s = 500Hz);

[0284] Block: B = 0.25 x 500 = 125 B = 0.25 x 500 = 125 → divided into 4 blocks (10% overlap);

[0285] Processing flow:

[0286] Key frame selection:

[0287] Block energy: [8.7, 5.2, 23.5, 7.3];

[0288] Global average energy:

[0289] Threshold:

[0290] Keyframe: Block 3 (Energy 23.5 > 22.35);

[0291] Global node generation:

[0292]

[0293] Global-local interaction:

[0294] Block 3 has a significantly higher attention weight (e.g., weight 0.6), enhancing pathological features (e.g., ST segment elevation).

[0295] This step reduces computational complexity through block-based processing. Within each block, self-attention is performed to capture local features. Keyframes are dynamically selected based on QRS energy to generate global context nodes. Finally, an attention mechanism is used to achieve global-local feature interaction. This hierarchical sparse structure significantly improves the efficiency of long sequence processing while preserving sensitivity to key pathological features.

[0296] Step 5. Joint Loss of Multiple Tasks

[0297] This step designs a multi-task joint loss function to address the class imbalance and temporal inconsistency issues in ECG disease identification. By combining frequency-weighted cross-entropy and temporal consistency loss, the model's sensitivity to rare pathologies and predictive stability are improved.

[0298] 1. Frequency-weighted cross-entropy loss:

[0299]

[0300] in f c ν is the frequency of category c in the training set; C is the total number of disease categories; y c The true label for category c; p c Let be the predicted probability for category c.

[0301] By assigning higher weights to low-frequency disorders (such as ventricular tachycardia) and lower weights to high-frequency disorders (such as sinus rhythm), the imbalance problem commonly found in clinical data can be alleviated.

[0302] 2. Temporal consistency loss:

[0303] Introduce a consistency constraint for adjacent segment predictions to ensure a smooth transition of model output over time:

[0304]

[0305] Where T is the total number of sub-segments, p tLet be the predicted probability vector for the t-th sub-segment;

[0306] Constraining the prediction differences between adjacent segments avoids drastic fluctuations in prediction results between adjacent heartbeats and enhances the model's stability for long sequences.

[0307] 3. Loss function fusion:

[0308] Weighted fusion has two loss functions:

[0309]

[0310] Where ξ is the regularization weight coefficient, and ξ decays linearly with the number of training rounds.

[0311] The pseudocode representation is as follows:

[0312]

[0313]

[0314] The specific steps are as follows:

[0315] Calculate the frequency of occurrence of each disease category in the batch;

[0316] According to the formula Calculate the category weights;

[0317] Calculate the weighted cross-entropy loss

[0318] Traverse all adjacent sub-segment pairs and calculate the L2 difference of the predicted probabilities;

[0319] Calculate the average to get

[0320] The two losses are fused with a weight ξ = 0.1.

[0321] Example explanation:

[0322] Disease categories: ["Normal", "Atrial fibrillation", "Ventricular tachycardia"];

[0323] Category frequency: [0.7, 0.25, 0.05];

[0324] Weight calculation:

[0325] Adjacent segment prediction:

[0326] Sub-segment t: [0.8, 0.1, 0.1];

[0327] Sub-segment t+1: [0.3, 0.6, 0.1];

[0328] Consistency loss:

[0329] Total loss:

[0330] This step solves the class imbalance problem by frequency-weighted cross-entropy, giving higher weight to rare pathologies; the smooth transition of adjacent sub-segments is constrained by the temporal consistency loss; the two are weighted and fused to form the final optimization target. This design significantly improves the recognition rate of low-frequency pathologies (such as ventricular tachycardia) by 15-20%, while reducing the temporal jitter of the prediction results.

[0331] Step 6. Real-time streaming processing

[0332] This step realizes real-time processing of long sequence ECG signals, and avoids repeated calculation through hidden state caching mechanism. By using hierarchical caching and LRU strategy, the inference efficiency is improved while ensuring temporal coherence.

[0333] 1. Hidden state caching:

[0334] Cache content:

[0335] 6th layer hidden state:

[0336] 12th layer hidden state:

[0337] Where M is the number of processed sub-segments, and D is the dimension of the hidden state.

[0338] 2. New sub-segment processing:

[0339] Calculation process:

[0340] Take as the input of the first 6 layers of Transformer, and only calculate the output of the 7th to 12th layers for the new sub-segment;

[0341] 3. Cache management (LRU strategy):

[0342] Eviction rule:

[0343] When M>N max , remove the earliest K sub-segments;

[0344] N max is the cache threshold (e.g. 100 sub-segments);

[0345] K is the removal amount each time (e.g. 10 sub-segments);

[0346] Update mechanism:

[0347] After processing the new sub-segment, it is appended to the tail of the cache;

[0348] Access timestamp record is used for LRU judgment.

[0349] Pseudo code is as follows:

[0350]

[0351]

[0352] Specific steps are as follows:

[0353] Initialize cache list and threshold parameter;

[0354] When cache exceeds N max = 100, remove the earliest K = 10 subsegments;

[0355] Splice historical cache and new subsegment into complete sequence;

[0356] Input first 6 layers of Transformer to calculate full-sequence hidden state;

[0357] Only take hidden state corresponding to new subsegment to input last 6 layers;

[0358] Add 6th layer and 12th layer hidden state of new subsegment to cache;

[0359] Return final output of new subsegment.

[0360] Example is as follows:

[0361] Input: 10-minute ECG, segmented into 200 subsegments;

[0362] Real-time processing:

[0363] 1st-100th subsegment: full-model calculation;

[0364] 101st subsegment:

[0365]

[0366] Last 6 layers only calculate new subsegment;

[0367] Cache update:

[0368] After cache reaches threshold N max = 100;

[0369] Every time 1 subsegment is newly processed, 1 earliest subsegment (K = 1) is eliminated;

[0370] Calculation amount comparison:

[0371] Traditional method: 200 subsegments x 12 layers = 2400 layers of calculation;

[0372] Streaming processing: 100×12 + 100×6 = 1800 layers (saving 25%).

[0373] This step caches the hidden state of the intermediate layer and reuses historical calculation results, requiring only high-level Transformer calculations for new data. Combined with an LRU eviction policy, it enables real-time processing of long sequences with limited memory. This mechanism reduces the system latency for processing a 10-minute 12-lead ECG to less than 2 seconds, meeting the needs of clinical real-time monitoring.

[0374] Step 7. Multi-lead collaborative noise reduction

[0375] This step integrates a multi-lead collaborative denoising module in the preprocessing stage to automatically detect and repair signal damage caused by motion artifacts. Through frequency domain analysis and spatial correlation reconstruction, it improves signal reliability in noisy environments.

[0376] 1. Motion artifact detection:

[0377] Noise segments are identified using high-frequency energy detection.

[0378] E HF =∑(s*h HPF ) 2

[0379] Judgment conditions:

[0380] If E HF >ζ·σ HF This is then identified as a noise segment;

[0381] Where σ HF Here, is the standard deviation of the high-frequency energy of the entire sequence, s is the time series of the lead signal, and h is... HPF Let ζ be the impulse response of the high-pass filter, and ζ be a predetermined coefficient (a positive real number).

[0382] Detection principle:

[0383] Motion artifacts are characterized by high-frequency abrupt changes (>5Hz);

[0384] Calculate the high-frequency energy for each 1-second window;

[0385] Items exceeding ζ times the standard deviation are considered noise.

[0386] 2. Signal Repair:

[0387] Damaged signals reconstructed based on lead spatial location weighting:

[0388]

[0389] in Let Ω be the reconstructed signal of lead i at time t, Ω be the set of healthy leads, and d be the reconstructed signal of lead i at time t. ijis the standard electrode distance between lead i and j (according to AHA lead position map), p is the distance attenuation index (positive integer), w ij is the repair weight of lead j to lead i.

[0390] The pseudo code is as follows:

[0391]

[0392]

[0393]

[0394] The specific steps are as follows:

[0395] Design a 5Hz high-pass filter (cutoff frequency 5Hz) for each lead:

[0396] Calculate the high-frequency energy E after filtering HF

[0397] Calculate the standard deviation σ of the high-frequency energy of all leads HF ; if E HF > 3.0·σ HF Mark as damaged lead; for each damaged lead i:

[0398] Get the set of healthy leads Ω;

[0399] Calculate the weight w ij according to the lead distance d ij ;

[0400] Reconstruct the signal according to the formula ;

[0401] Return the repaired 12-lead signal.

[0402] Example:

[0403] Damaged lead: V3 is judged to be damaged (high-frequency energy exceeds standard)

[0404] Healthy lead: Ω = {V1, V2, V4, V5};

[0405] Distance calculation:

[0406] d(V3-V1) = 5cm, d(V3-V2) = 3cm, d(V3-V4) = 4cm, d(V3-V5) = 6cm;

[0407] Weight calculation (p = 2):

[0408] w = [5 -2 , 3 -2 , 4 -2 , 6 -2] = [0.04, 0.11, 0.06, 0.03];

[0409] Normalization: w = [0.17, 0.46, 0.25, 0.12];

[0410] Signal reconstruction:

[0411]

[0412] This step detects motion artifacts by high-frequency energy and reconstructs the corrupted signal by spatial correlation of healthy leads. The weighting strategy based on electrode distance (distance decay exponent p = 2) conforms to the physical characteristics of electrocardio propagation, which can reduce the lead reconstruction error by 40-60% and significantly improve the system robustness in noisy environment.

[0413] Step 8. Model compression deployment

[0414] This step designs a three-level model compression framework to address the resource constraints of clinical devices. By simplifying the structure, pruning guided by physiology, and knowledge distillation, the 12-layer original model is compressed into a 6-layer lightweight model, reducing the computational load.

[0415] 1. Structural compression:

[0416] Compression scheme:

[0417] Original 12-layer encoder → 6-layer codec (4 encoding layers + 2 decoding layers);

[0418] Decoder uses linear attention:

[0419] LinearAttn(Q, K, V) = V·softmax(K) T Q

[0420] Complexity comparison:

[0421] Original: O(N 2 ) → compressed: O(N);

[0422] Parameter amount: example 120M → 35M.

[0423] 2. Physiological pruning:

[0424] Pruning based on attention head's saliency map score on QRS wave:

[0425]

[0426] where S h is the saliency map score of the hth attention head, H is the total number of attention heads, and φ is the predetermined threshold coefficient (typical value 0.4, range 0.3-0.6);

[0427] Pruning rules:

[0428] Compute sensitivity of each head to QRS wave;

[0429] Remove heads with sensitivity lower than 40% of the maximum.

[0430] 3. Pathology knowledge distillation:

[0431] Guiding the compressed model learning with the original model:

[0432]

[0433] where is the logits output of the teacher model for class c, is the logits output of the student model for class c, T temp is the temperature coefficient, and σ is the softmax function.

[0434] The pseudo-code is represented as:

[0435]

[0436]

[0437]

[0438] The specific steps are:

[0439] Build a 6-layer student model (4-layer encoder + 2-layer linear attention decoder);

[0440] Compute the saliency map score S h of each attention head of the teacher model to QRS wave;

[0441] Remove all attention heads with S h <0.4×max(S k );

[0442] Distillation training loop:

[0443] The teacher model generates soft labels p tea = softmax(z tea / 2.0);

[0444] The student model outputs p stu = softmax(z stu / 2.0);

[0445] Compute the KL divergence loss

[0446] Update the student model combined with 30% cross-entropy loss.

[0447] Example:

[0448] Salience map score:

[0449] Head 1: S1 = 0.8 (sensitive to QRS)

[0450] Head 2: S2 = 0.3 (sensitive to noise)

[0451] Head 3: S3 = 0.9 (max value)

[0452] Pruning decision: φ = 0.4 → threshold = 0.4 x 0.9 = 0.36

[0453] Head 2 is pruned (0.3 < 0.36)

[0454] Distillation effect:

[0455] Teacher model accuracy: 95.2%

[0456] Student model after distillation: 93.7% (direct training only 91.2%)

[0457] This step achieves three-level optimization through codec architecture compression, physiological guided pruning, and knowledge distillation, reducing the computational load by 70% while maintaining model performance (accuracy decrease < 2%). The compressed model can run in real-time on mobile devices (such as ECG monitors) with an inference speed of 50 frames per second, meeting the needs of clinical deployment.

[0458] Step 9. Gradient Directional Regulation Module

[0459] This step implements a gradient directional regulation strategy during model training, enhancing the model's sensitivity to ECG pathological features through key feature gradient amplification and dynamic clipping mechanisms, while ensuring training stability.

[0460] 1. QRS Gradient Amplification:

[0461] Identify the gradient of the time step corresponding to the QRS complex:

[0462]

[0463] where ρ is the gradient amplification coefficient, is the parameter gradient corresponding to the QRS period;

[0464] Strengthen the model's ability to recognize QRS waveform distortions (such as premature ventricular contractions and bundle branch block).

[0465] 2. Dynamic Gradient Clipping:

[0466] Monitor the gradient norm to prevent training divergence:

[0467]

[0468] where τ is the gradient norm threshold, is the L2 norm of the gradient vector;

[0469] Scale down the gradient when it explodes.

[0470] The pseudo code is represented as:

[0471]

[0472]

[0473] The specific steps are:

[0474] Perform regular forward propagation and loss computation;

[0475] Get the gradient of all parameters;

[0476] Locate the gradient of the QRS complex corresponding time step (100-150);

[0477] Amplify the QRS region gradient by 2 times (p = 2.0);

[0478] Check if the gradient L2 norm exceeds the threshold τ = 1.0;

[0479] If it exceeds, scale it down;

[0480] Update the gradient after regulation.

[0481] Example:

[0482] Training scenario: model learns to identify ventricular premature beat;

[0483] QRS period: time steps 120-125 in a certain segment are abnormal QRS;

[0484] Gradient change:

[0485] Original gradient value: 0.3 (QRS region);

[0486] After amplification: 0.3 x 2.0 = 0.6;

[0487] Gradient norm: 0.85 (<1.0) → no need for clipping;

[0488] Training effect: model sensitivity to abnormal QRS is improved by 40%.

[0489] This step enhances the model's ability to identify key ECG pathologies by amplifying the gradient of QRS features in a targeted manner, and prevents training divergence by combining dynamic gradient clipping. This allows the model to improve the accuracy of identifying key pathological features such as ventricular premature beat and ST-T changes, while reducing training shock phenomena.

[0490] Application Example: Early diagnosis of acute myocardial infarction patients

[0491] Patient Information:

[0492] Gender Age: xx-year-old male

[0493] Complaint: Sudden severe retrosternal squeezing pain, lasting for 30 minutes without relief, accompanied by profuse sweating, nausea

[0494] Initial signs: Blood pressure 160 / 100 mmHg, heart rate 110 beats / min (sinus tachycardia), slightly accelerated respiration

[0495] Clinical suspicion: Acute coronary syndrome (ACS), highly suspected acute myocardial infarction (AMI)

[0496] System processing flow:

[0497] 1. Multi-lead signal preprocessing (reflecting technical points):

[0498] Input: Connect the patient to a standard 12-lead ECG machine, collect the initial signal (sampling rate 500 Hz); the signal baseline is significantly drifting (the patient is agitated), there is significant muscle artifact (muscle tremor) in V3 lead.

[0499] Processing:

[0500] Standardization: The system automatically applies a 0.5 Hz high-pass filter to eliminate baseline drift, a 50 Hz notch filter to eliminate power supply interference, and z-score standardization to all leads, making the signal mean 0 and variance 1, eliminating individual and device differences.

[0501] Collaborative denoising: The system detects that the high-frequency energy of V3 lead is abnormally high (more than 3 times the global standard deviation), determining that it is a motion artifact damage. Using the spatial relationship of multi-lead, the system automatically calculates the repair weight of V1, V2, V4, V5 leads to V3 according to the AHA standard lead spacing (the closer the lead to V3, the greater the weight), and real-time reconstructs the clean V3 lead signal, restoring the ST segment morphology obscured by noise.

[0502] 2. Heartbeat period adaptive segmentation:

[0503] The patient's heart rate is fast (~ 110 bpm) and there are occasional premature beats, with unstable RR intervals. Fixed window segmentation is prone to truncating P waves or T waves, destroying the integrity of the waveform.

[0504] Processing:

[0505] Adaptive segmentation: A differential-threshold joint algorithm is used to locate each R-peak accurately. Based on the real-time calculated historical average RR interval and a preset scaling factor a, the current sub-segment length (e.g., covering about 1.15 cardiac cycles) is determined dynamically. The sub-segment boundaries are extended forward and backward (40% forward, 60% backward) based on the R-peak, ensuring that each sub-segment contains complete P-QRS-T waveforms. Adjacent sub-segments are set with a 30% overlap to ensure temporal continuity. Finally, a 10-minute recording is segmented into 165 continuous and complete waveform sub-segments.

[0506] 3. Spatio-temporal embedding generation:

[0507] Processing:

[0508] Embedding fusion: For each cardiac sub-segment:

[0509] Local morphological features (e.g., QRS wave width, ST segment morphology, T wave polarity) are extracted through a lightweight convolutional network (e.g., two-layer CNN), generating a 128-dimensional embedding vector (E emb ).

[0510] A learnable temporal position encoding (E time ) is superimposed to mark the temporal order of the sub-segment in the entire sequence.

[0511] An anatomical topology-based lead position encoding (E lead ) is superimposed:

[0512] For limb leads (I, II, III, aVR, aVL, aVF), the encoding is initialized according to the standard vector angles of Einthoven's triangle.

[0513] For chest leads (V1-V6), the encoding is initialized according to their standard anatomical coordinates on the human chest wall (e.g., V4 is located at the left midclavicular line at the 5th intercostal space).

[0514] The three are added together and layer-normalized to generate the final spatio-temporal fusion embedding vector (E final ), which contains ECG morphology, temporal information, and lead spatial position relationships.

[0515] 4. Multi-lead spatio-temporal encoding:

[0516] Processing: The spatio-temporal fusion embedding vectors (E final ) of all sub-segments of the 12 leads are arranged into a structured two-dimensional matrix according to the "time dimension (sub-segment sequence) x lead dimension (anatomical position)". This matrix is the joint representation of the system to understand the evolution of long sequence ECG signals in space (multi-lead) and time, serving as the input of the subsequent Transformer.

[0517] 5. Hierarchical Sparse Transformer Processing:

[0518] 10-minute long sequence with 165 sub-segments, full-attention computation is huge.

[0519] Processing:

[0520] Chunking: The long sequence is divided into multiple smaller chunks (e.g., each covering about 1 heartbeat) based on the number of sampling points covering a single heartbeat.

[0521] Local chunk full attention: Within each chunk, the Transformer layer performs full multi-head self-attention computation, finely capturing the waveform features and their mutual relationships (e.g., the association between QRS and T waves) within the chunk (single heartbeat or adjacent heartbeats).

[0522] Global key frame aggregation: The system dynamically analyzes the physiological features (e.g., QRS complex energy) of each signal chunk. When the QRS energy of a chunk is significantly higher than the global average (e.g., more than 2 times), the chunk is forced to be labeled as a "key frame" (Key Frame), which usually corresponds to the area with the most significant pathological features (e.g., the period of severe ST segment elevation in this example).

[0523] Global-local interaction: The hidden state of all key frames is normalized and max-pooled to generate a "global context node" (Global Context Node) representing the key pathological information of the entire sequence. This global node then interacts with each local chunk through an attention mechanism, injecting key global information (e.g., anterior myocardial ischemia pattern) into the local features, guiding the model to more accurately interpret each heartbeat, especially those with less obvious pathological features. The final output contains a feature representation with deep context information.

[0524] 6. Disease Classification Output:

[0525] Processing:

[0526] Extract the hidden state vector of the [CLS] flag specially added by the final Transformer layer, which aggregates the comprehensive information of the entire long sequence ECG.

[0527] This vector is input into the classification layer, which outputs the probability distribution of each type of heart disease.

[0528] Multi-task joint loss (reflected in the training phase): During model training, the system uses:

[0529] Frequency-weighted cross-entropy loss: Assign higher weights (based on their frequency in the training set) to relatively rare critical categories such as acute myocardial infarction, to address the data imbalance problem and improve the recognition rate of these key pathologies.

[0530] Temporal consistency loss: constrain the prediction results of adjacent sub-segments not to have dramatic jumps (e.g. the previous sub-segment is judged as normal, and the next sub-segment is suddenly judged as myocardial infarction), to ensure the smoothness and stability of long sequence diagnosis results. The two losses are weighted and summed as the total optimization objective.

[0531] 7. Real-time streaming processing:

[0532] Requirement: Clinically, there is a need to continuously monitor the changes in the patient's electrocardio.

[0533] Processing:

[0534] The system continuously caches the hidden state matrices of the historical processed sub-segments at the intermediate layers (e.g. layer 6) and the final layer (layer 12) of the Transformer.

[0535] When a newly collected electrocardio sub-segment is input:

[0536] The system splices the cached intermediate layer hidden states with the new sub-segment.

[0537] Only the outputs of the high layers (e.g. layers 7 to 12) of the Transformer are calculated for the new sub-segment, greatly saving the amount of calculation.

[0538] The Least Recently Used (LRU) strategy is used to manage the cache space, and when the number of cached sub-segments exceeds the threshold (e.g. 100 segments), the oldest cached sub-segment state is removed.

[0539] Effect: In this example, the system's processing delay for each new heartbeat (sub-segment) is controlled within 1 second, achieving near real-time monitoring and early warning of acute electrocardio events.

[0540] 8. Gradient directional regulation module:

[0541] Purpose: To improve the sensitivity of the model to key pathological features of electrocardio (such as ST segment changes in this example).

[0542] Processing (when training):

[0543] QRS gradient amplification: when updating parameters through backpropagation, the system identifies the gradients generated by model parameters at time steps corresponding to QRS complex. The gradients in these key areas are directionally amplified (e.g. by 2 times), forcing the model to pay more attention to QRS shape distortion (such as pathological Q wave) and the subsequent ST-T segment changes.

[0544] Dynamic gradient clipping: to prevent gradient explosion caused by amplification or training itself, the system monitors the gradient size (L2 norm) of all parameters in real time. Once the gradient exceeds the pre-set safety threshold, the gradient is scaled proportionally to ensure the stability of the training process.

[0545] Effect: In this example, this mechanism significantly enhances the model's ability to identify subtle ST segment elevation (key feature in this example) and pathological Q waves.

[0546] 9. Model Compression Deployment:

[0547] Requirement: Real-time operation on resource-limited bedside monitors or mobile terminals.

[0548] Process:

[0549] Structural Compression: Replace the original 12-layer Transformer encoder with a lighter 6-layer architecture (e.g., 4-layer encoder + 2-layer decoder with linear attention), reducing computational complexity from O(N 2 ) to O(N).

[0550] Physiological Guided Pruning: Analyze the contribution of each "attention head" in the model to key electrocardiogram features such as QRS waves (salience score). Remove those attention heads that are not sensitive to key features (heads with scores significantly lower than the maximum value), further simplifying the model.

[0551] Pathological Knowledge Distillation: Use the trained and powerful original 12-layer model (teacher model) to guide the training of the compressed 6-layer model (student model). The student model not only learns the true labels, but more importantly, learns the "soft labels" (containing rich information such as similarity between classes) output by the teacher model, enabling the small model to approximate the diagnostic ability of the large model.

[0552] Effect: The compressed model runs efficiently on the embedded system of the bedside monitor, with an inference speed of more than 50 frames per second, meeting the real-time requirements of clinical practice, while maintaining high diagnostic accuracy (accurately identifying acute myocardial infarction in this example).

[0553] System Diagnosis Output and Clinical Verification:

[0554]

[0555]

[0556] Summary of the Example:

[0557] This example fully demonstrates the workflow of the ECG disease recognition system based on the Transformer architecture: starting from the preprocessing of multi-lead signals, noise interference is eliminated through adaptive filtering and standardization; the dynamic heart beat segmentation technology is applied to preserve the complete physiological structure; the spatio-temporal coding is fused to generate ECG feature representation; the hierarchical sparse Transformer is used to capture long sequence pathological correlation; the gradient regulation and multi-task optimization are combined to improve the sensitivity of key features; finally, the clinical real-time diagnosis is realized through streaming processing and model compression. The system successfully identifies the key pathological patterns such as ST segment persistent depression in patients with coronary heart disease and generates emergency intervention suggestions, the whole process significantly improves the analysis efficiency while ensuring the accuracy of diagnosis, verifying its clinical application value in early warning of complex heart diseases.

Claims

1. A long sequence electrocardiogram signal disease recognition system based on a Transformer architecture, characterized in that, Comprise: (a) Multi-lead signal preprocessing: configured to receive multi-lead raw ECG signals, eliminate baseline drift through a low-frequency cutoff high-pass filter, eliminate power supply interference through a power frequency notch filter, and normalize each lead signal to zero mean unit variance through z-score standardization: ， wherein denotes the raw sample value of the =1,2,...,12; and are its mean and standard deviation, respectively; (b) Heartbeat cycle adaptive segmentation: dynamically segment the normalized signal output by step (a) into subsegments based on R-wave detection, with the subsegment length L determined by physiological parameters: ， wherein is the historical RR interval mean; and a is a preset proportionality coefficient; is the sampling rate and satisfies ; Adjacent subsegments are set with a preset proportion of overlap to generate a time-continuous subsegment sequence; (c) Spatio-temporal embedding generation: a convolutional neural network is used to extract features of the sub-segments, and the generated embedding vector Post-stacking learnable position encoding is added to form a spatio-temporal embedding vector; (d) Multi-lead spatio-temporal encoding: The spatio-temporal embedding vectors of each lead are arranged in a time-lead two-dimensional matrix, where the time dimension position encodes Adopting a trainable sinusoidal function generation, lead dimension position encoding According to the standard lead anatomical topology initialization, the joint spatio-temporal encoding matrix is constructed; (e) Hierarchical sparse Transformer processing: input the spatio-temporal coding matrix into a multi-layer improved encoder, each layer performs: Local block full attention: take the number of sampling points covering a single heartbeat cycle as the basic block, and calculate the multi-head self-attention within the block; Global key frame aggregation: dynamically select key frames according to signal characteristics, generate global context nodes through pooling operations and interact with each block, output deep feature representation; (f) Disease classification output: extract the hidden state vector of the [CLS] flag bit in the final layer of the Transformer, and output the probability distribution of multiple cardiac diseases.

2. The long sequence electrocardiosignal disease recognition system based on the Transformer architecture according to claim 1, characterized in that, Step (b) further comprises the following: The R-wave detection adopts a differential-threshold combined algorithm: if a signal point satisfies and simultaneously satisfies , then it is determined that the t moment is the R-wave peak position, wherein is a differential threshold coefficient, is an amplitude threshold coefficient, is an analysis window starting index, is an amplitude detection range index. The subsegment boundary is extended forward and backward based on the R-wave position to ensure that each subsegment contains an integer number of complete heartbeat cycles.

3. The long sequence electrocardiosignal disease recognition system based on the Transformer architecture according to claim 1, characterized in that, The lead dimension position code in step (d) The initialization method includes differentiated processing of limb leads and chest leads: Limb leads generate codes according to vector angles based on Einthoven's triangle theory: ， ， wherein and is a lead standard vector angle, and is a trainable projection basis vector; Chest leads are initialized according to human anatomical coordinates: ， wherein is an anatomical coordinate vector for lead j, is a trainable weight matrix, is a bias vector.

4. The long sequence electrocardiosignal disease recognition system based on the Transformer architecture according to claim 1, characterized in that, The global key frame aggregation operation in step (e) includes: Dynamic key frame selection mechanism: when the QRS complex energy of a certain signal block exceeds twice the global average, it is forced to be a key frame, and the energy calculation formula is: ， wherein is a signal block energy, is a global average energy, is an energy threshold coefficient; The global context node G is generated using the max-pooling operation: ， wherein is a pooling function, is a normalization function, is the k-th keyframe latent state vector, is a set of keyframe indices.

5. The long sequence electrocardiosignal disease recognition system based on the Transformer architecture according to claim 1, characterized in that, The system includes a gradient directional regulation module, which performs the following operations during model training: Amplify the gradient of the QRS wave corresponding time step to enhance the sensitivity of the model to ECG key features: ， wherein is a gradient amplification coefficient, is a parameter gradient corresponding to the QRS period; Dynamic gradient clipping strategy prevents training divergence: if the gradient norm of a parameter exceeds a threshold , then scale it down by a factor of : ， wherein is a gradient norm threshold, is a gradient vector's norm.

6. The long sequence electrocardiosignal disease recognition system based on the Transformer architecture according to claim 1, characterized in that, Step (f) uses a multi-task joint loss function for disease classification, which includes: The main loss function is frequency-weighted cross-entropy to solve the class imbalance problem: ， wherein , is the frequency of class c in the training set, is the class balancing coefficient; is the total number of disease classes; is the true label of class c; is the predicted probability of class c; The time consistency loss function constrains the prediction smoothness of adjacent subsegments: ， where T is the total number of sub-sections, is the prediction probability vector for the tth sub-section. The total loss function is the weighted sum: ， wherein is a regularization weight coefficient.

7. The long sequence electrocardiosignal disease recognition system based on the Transformer architecture according to claim 1, characterized in that, The system supports real-time streaming processing through the hidden state caching mechanism: Persistent cache history sub-segment in 6th and 12th layer hidden state matrices and where M is the number of processed sub-segments and D is the hidden state dimensionality. When a new subsegment At input time, the As input to the first 6 layers of the Transformer, only the output of layers 7-12 is computed for the new subsegment; Managing the cache using a least recently used LRU policy: when the number of cache segments exceeds a predetermined threshold the oldest K segments are removed.

8. The long sequence electrocardiosignal disease recognition system based on the Transformer architecture according to claim 1, characterized in that, The preprocessing stage integrates a multi-lead collaborative denoising module, which includes: Motion artifact detection: high frequency band energy of the signal is computed if then a noise segment is determined, where is the standard deviation of the high frequency energy over the entire sequence, s is the time series of the lead signal, is the impulse response of the high pass filter, is a predetermined coefficient, and the predetermined coefficient is a positive real number; Signal reconstruction based on lead spatial relationship: if lead i is determined to be damaged, its signal is reconstructed using healthy leads: , wherein is a reconstructed signal for lead i at time t; is a set of healthy leads; is a standard electrode spacing for lead i and j, the standard electrode spacing being determined according to the AHA lead position map; is a distance decay index, the distance decay index being a positive integer; is a repair weight of lead j on lead i.

9. The long sequence electrocardiosignal disease recognition system based on the Transformer architecture according to claim 1, characterized in that, The system deployment stage uses an ECG feature-aware model compression framework, which includes three levels of optimization: Structural compression: replace the original 12-layer Transformer with a 6-layer encoder-decoder architecture, where: The encoder layer retains local-global attention mechanism; The decoder layer uses lightweight linear attention, reducing complexity to O(N); Physiological guided weight pruning: computing a saliency map for each attention head on QRS wave features , if then pruning the head, wherein is a saliency map score of the h-th attention head, is a total number of attention heads, is a predetermined threshold coefficient; Pathological knowledge distillation: The teacher model is the original 12-layer model, which guides the output probability distribution of the student model, and the student model is the 6-layer model: ， where is the logits output of the teacher model for class c, is the logits output of the student model for class c, is the temperature coefficient, is the softmax function.

Images