Long-sequence electrocardiosignal disease recognition system based on Transform architecture

Abstract

The invention relates to the technical field of electrocardiosignal analysis, and discloses a long-sequence electrocardiosignal disease recognition system based on a Transform architecture. The core defects that in traditional electrocardiogram analysis, waveform integrity is damaged by fixed window segmentation, a lead space topological relation is neglected, and long sequence modeling efficiency is low are overcome, a P-QRS-T waveform structure is completely reserved through the heart beat adaptive segmentation technology, and the fixed window truncation risk is eliminated; the lead anatomical topology and the space-time coding are fused, and multi-lead electrophysiological association is modeled; long sequence efficient processing is realized based on hierarchical sparse Transform, and the recognition sensitivity of complex pathologies such as arrhythmia and myocardial ischemia is improved; in combination with gradient directional regulation and control and a streaming processing mechanism, the clinical real-time requirement is met while the diagnosis accuracy is guaranteed, and finally, reliable, efficient and universal intelligent decision support is provided for early warning of heart diseases through lightweight deployment of an adaptive mobile terminal.

Description

Technical Field

[0001] This invention relates to the field of electrocardiogram (ECG) signal analysis technology, specifically to a long-sequence ECG signal disease identification system based on the Transformer architecture. Background Technology

[0002] Current ECG signal analysis techniques mainly rely on fixed-length window segmentation strategies and local feature extraction models. For example, they use a sliding window of preset duration to extract signal segments and perform segment-by-segment classification based on convolutional neural networks or recurrent neural networks. While such methods can capture the morphological features of a single heartbeat, they are limited by the rigid segmentation mechanism of the fixed window and cannot dynamically adjust the analysis interval according to the patient's actual heart rate, resulting in the forced separation of pathological correlations across heartbeats. At the same time, traditional models typically use channel stacking or simple pooling to process multi-lead signals, ignoring the physiological spatial topological relationships between leads, making the fusion of spatial dimension information superficial.

[0003] However, existing technologies have the following drawbacks: First, fixed window segmentation disrupts the physiological integrity of ECG waveforms, especially in cases with large RR interval variability such as arrhythmias, easily causing P / T wave truncation or aliasing, significantly reducing the recognizability of key waveform features; Second, the ability to model spatiotemporal features of multi-lead systems is weak, failing to encode the anatomical positional relationships of the standard 12-lead system (such as the angle between limb lead vectors and the spatial coordinates of chest leads) into the feature learning process, thus limiting the depth of spatial correlation mining; Third, long sequence computation is inefficient, the sequence dependency characteristics of traditional RNNs cannot process ultra-long ECG records in parallel, and the layer stacking of CNNs makes it difficult to model minute-level long-term dependencies due to limited receptive fields, failing to meet the needs of real-time clinical analysis. Therefore, a long sequence ECG signal disease recognition system based on the Transformer architecture is proposed. Summary of the Invention

[0004] To address the shortcomings of existing technologies, this invention provides a long-sequence electrocardiogram (ECG) signal disease identification system based on the Transformer architecture, thereby resolving the problems mentioned in the background.

[0005] To achieve the above objectives, the present invention provides the following technical solution: a long-sequence electrocardiogram signal disease recognition system based on the Transformer architecture, comprising:

[0006] (a) Multi-lead signal preprocessing: Configured to receive raw multi-lead ECG signals, baseline drift is eliminated by a low-frequency cutoff high-pass filter, power frequency notch filter is used to eliminate power supply interference, and each lead signal is normalized to zero mean and unit variance by z-score normalization.

[0007]

[0008] in μ represents the original sampled value of lead i at time t (i = 1, 2, ..., 12). i and σ i These are its mean and standard deviation, respectively;

[0009] (b) Adaptive segmentation of the cardiac cycle: The normalized signal output from step (a) is dynamically segmented into sub-segments based on R-wave detection, and the sub-segment length L is determined by physiological parameters.

[0010]

[0011] in The mean of historical RR intervals; α is the preset proportional coefficient; f s The sampling rate is f and satisfies f s ≥250Hz;

[0012] Adjacent segments are set to overlap by a preset ratio to generate a temporally continuous sequence of segments;

[0013] (c) Spatiotemporal embedding generation: Extracting features from segments and passing them through a convolutional network to generate embedding vectors. Learnable positional codes are then superimposed to form a spatiotemporal embedding vector;

[0014] (d) Multi-lead spatiotemporal coding: The spatiotemporal embedding vectors of each lead are arranged in a time-lead two-dimensional matrix, where the time dimension position code E time It uses a trainable sine function to generate lead-dimensional position encoding E. lead Based on the standard lead anatomy topology initialization, a joint spatiotemporal coding matrix is ​​constructed;

[0015] (e) Layer-based sparse Transformer processing: The spatiotemporal coding matrix is ​​input into a multi-layer improved encoder, and each layer performs the following:

[0016] Local block full attention: Using the number of sampling points covering a single heartbeat cycle as the basic block, multi-head self-attention is calculated within the block;

[0017] Global keyframe aggregation: Dynamically select keyframes based on signal features, generate global context nodes through pooling operations and interact with each block, and output a deep feature representation;

[0018] (f) Disease classification output: Extract the hidden state vector of the [CLS] flag bit of the final layer of Transformer and output the probability distribution of multiple heart diseases.

[0019] Preferably, step (b) of adaptive cardiac cycle segmentation further includes the following:

[0020] R-wave detection employs a differential-threshold joint algorithm: if signal point s tsatisfy And simultaneously satisfy Then, time t is determined to be the position of the R-wave peak, where κ is the differential threshold coefficient, λ is the amplitude threshold coefficient, β,γ are the starting index of the analysis window, and δ,∈ is the amplitude detection range index;

[0021] The segment boundaries are extended forward and backward based on the R-wave position to ensure that each segment contains an integer number of complete cardiac cycles.

[0022] Preferably, the lead position code E in step (d) lead The initialization method includes differentiated processing for limb leads and chest leads:

[0023] Limb leads generate codes based on Einthoven's triangle theory, using vector angles:

[0024] E lead (I)=cos(θ I )·P x +sin(θ I )·P y

[0025] E lead (II)=cos(θ II )·P x +sin(θ II )·P y

[0026] Where θ I and θ II P is the standard vector angle of the lead. x and P y These are trainable projective basis vectors;

[0027] Chest leads are initialized based on human anatomical coordinates:

[0028]

[0029] Where x j Let W be the anatomical coordinate vector of lead j. coord Let b be the trainable weight matrix and b be the bias vector.

[0030] Preferably, the global keyframe aggregation operation in step (e) includes:

[0031] Dynamic keyframe selection mechanism: When the QRS composite energy of a signal block exceeds twice the global average, it is forcibly set as a keyframe. The energy calculation formula is as follows:

[0032]

[0033] Where Eblock For signal block energy, The global average energy is η, and the energy threshold coefficient is η.

[0034] The global context node G is generated using max pooling:

[0035]

[0036] in For pooling functions, H is the normalization function. k Let this be the hidden state vector of the k-th keyframe. This is a set of keyframe indexes.

[0037] Preferably, the system includes a gradient orientation control module, which performs the following operations during the model training phase:

[0038] The gradient of the QRS complex at the corresponding time step is amplified to enhance the model's sensitivity to key ECG features:

[0039]

[0040] Where ρ is the gradient amplification factor. The parameter gradient corresponding to the QRS time period;

[0041] Dynamic gradient clipping strategy to prevent training divergence: If the gradient norm of parameter W is... If the value exceeds the threshold τ, it will be scaled proportionally:

[0042]

[0043] Where τ is the gradient norm threshold. Let L2 be the gradient vector.

[0044] Preferably, the disease classification in step (f) employs a multi-task joint loss function, which includes:

[0045] The main loss function is frequency-weighted cross-entropy, which addresses the class imbalance problem.

[0046]

[0047] in f c ν is the frequency of category c in the training set; C is the total number of disease categories; y c The true label for category c; p c Let c be the predicted probability of category c;

[0048] The temporal consistency loss function constrains the prediction smoothness of adjacent segments:

[0049]

[0050] Where T is the total number of sub-segments, p t Let be the predicted probability vector for the t-th sub-segment;

[0051] The total loss function is a weighted sum:

[0052]

[0053] Where ξ is the regularization weight coefficient.

[0054] Preferably, the system supports real-time streaming processing, which is achieved through a hidden state caching mechanism:

[0055] The hidden state matrix of the continuously cached historical segments at levels 6 and 12.

[0056] and Where M is the number of processed segments, and D is the hidden state dimension;

[0057] When the new subsegment new When inputting, As input to the first 6 Transformer layers, the output of layers 7 to 12 is computed only for the new segment;

[0058] The Least Recently Used (LRU) strategy is used to manage the cache: when the number of cached segments exceeds a predetermined threshold N... max When the time comes, remove the hidden states corresponding to the earliest generated K sub-segments.

[0059] Preferably, the preprocessing stage integrates a multi-lead collaborative denoising module, comprising:

[0060] Motion artifact detection: Calculating the high-frequency band energy E of the signal HF =∑(s*h HPF ) 2 If E HF >ζ·σ HF This is then identified as a noise segment, where σ HF Here, is the standard deviation of the high-frequency energy of the entire sequence, s is the time series of the lead signal, and h is... HPF Let ζ be the impulse response of the high-pass filter, and ζ be a predetermined coefficient (a positive real number).

[0061] Signal restoration based on lead spatial relationships: If lead i is determined to be damaged, its signal is reconstructed using healthy leads.

[0062]

[0063] in Let Ω be the reconstructed signal of lead i at time t, Ω be the set of healthy leads, and d be the reconstructed signal of lead i at time t. ijThe standard electrode spacing between leads i and j (based on the AHA lead position diagram), p is the distance attenuation exponent (positive integer), and w ij Let J be the repair weight of lead j relative to lead i.

[0064] Preferably, the system deployment phase employs a model compression framework based on ECG feature perception, which includes three levels of optimization:

[0065] Structural compression: The original 12-layer Transformer is replaced with a 6-layer encoding / decoding architecture, where:

[0066] The encoder layer retains the local-global attention mechanism;

[0067] The decoder layer employs lightweight linear attention, reducing the complexity to O(N);

[0068] Physiologically guided weighted pruning:

[0069] Calculate the saliency plot S of each attention head on the QRS wave characteristics. h ,like Then cut off the head, where S h The saliency map score for the h-th attention head is given, where H is the total number of attention heads and φ is the predetermined threshold coefficient.

[0070] Distillation of Pathological Knowledge:

[0071] The teacher model (original 12 layers) guides the student model (6 layers) to output probability distribution:

[0072]

[0073] in The output of the teacher model for category c is the logits. For the student model's logits output for class c, T temp Let σ be the temperature coefficient, and σ be the softmax function.

[0074] Compared with the prior art, the present invention has the following beneficial effects:

[0075] This invention overcomes the core shortcomings of traditional ECG analysis, such as fixed window segmentation destroying waveform integrity, ignoring lead spatial topological relationships, and low efficiency in long sequence modeling. It fully preserves the P-QRS-T waveform structure through adaptive cardiac segmentation technology, eliminating the risk of fixed window truncation; it integrates lead anatomical topology and spatiotemporal coding to model multi-lead electrophysiological correlations; it achieves efficient long sequence processing based on hierarchical sparse Transformer, improving the sensitivity for identifying complex pathologies such as arrhythmias and myocardial ischemia; and it combines gradient-oriented modulation and streaming cytometry mechanisms to meet clinical real-time needs while ensuring diagnostic accuracy. Finally, through lightweight deployment adapted to mobile terminals, it provides reliable, efficient, and universally applicable intelligent decision support for early warning of cardiac diseases.

[0076] Other features and advantages of the invention will be set forth in the description which follows, and will be apparent in part from the description, or may be learned by practicing the invention. The objects and other advantages of the invention may be realized and obtained by means of the structures pointed out in the description, claims and drawings. Attached Figure Description

[0077] Figure 1 This is a flowchart of the overall processing of the system of the present invention;

[0078] Figure 2 This is a flowchart of the adaptive segmentation of the cardiac cycle according to the present invention;

[0079] Figure 3 This is a flowchart of the spatiotemporal embedding generation process of the present invention;

[0080] Figure 4 This is a flowchart of the hierarchical sparse Transformer processing of the present invention;

[0081] Figure 5 This is a flowchart of the gradient-oriented modulation training process of the present invention;

[0082] Figure 6 This is a flowchart of the compressed deployment process of the model of this invention;

[0083] Figure 7 This is a flowchart of the real-time streaming processing of the present invention. Detailed Implementation

[0084] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0085] Please see Figure 1-7The long-sequence electrocardiogram signal disease identification system based on the Transformer architecture in this invention includes:

[0086] Step 1: Multi-lead signal preprocessing

[0087] This step performs standardization preprocessing on the raw 12-lead ECG signals to eliminate noise interference and standardize signal scale, providing high-quality input data for subsequent processing. Preprocessing includes three core operations: baseline drift elimination, power line interference elimination, and signal standardization, ensuring that all lead signals have zero mean and unit variance.

[0088] 1. Baseline drift elimination:

[0089] Use a second-order Butterworth high-pass filter with a cutoff frequency of 0.5 Hz;

[0090] Low-frequency components are eliminated by moving average filtering:

[0091]

[0092] Where y t The output signal after filtering at time point t, s t Here, N represents the original input signal at time t, N is the sliding window length (number of sampling points), and k is the index of the sampling point within the window.

[0093] Eliminate low-frequency baseline drift caused by respiratory movements (0.15-0.3Hz) and body movements.

[0094] 2. Power frequency interference elimination:

[0095] A 50Hz (or 60Hz) notch filter is used.

[0096] Transfer function:

[0097]

[0098] Where H(z) is the transfer function of the filter, z- 1 Let ω0 be the normalized angular frequency of the target interference frequency, and r be the pole radius.

[0099] Achieving deep attenuation at a specific frequency point to eliminate power supply interference and its harmonics.

[0100] 3. Signal standardization:

[0101] The z-score standardization formula is used to eliminate individual differences in signals from each lead, ensuring that all signals have zero mean and unit variance.

[0102]

[0103] in μ represents the original sampled value of lead i at time t (i = 1, 2, ..., 12). i and σ i These are its mean and standard deviation, respectively;

[0104] Calculation rules:

[0105] Mean calculation window: 5-second sliding window

[0106] Standard deviation calculation: Global calculation

[0107] Eliminate DC offset of the signal to bring the waveform center to zero; unify the signal amplitude range, resolve the gain differences between different leads, eliminate individual differences, and make all lead signals have zero mean and unit variance.

[0108] The pseudocode representation is as follows:

[0109]

[0110]

[0111] The steps are as follows:

[0112] Initialize the coefficients of the 0.5Hz high-pass filter and the 50Hz notch filter;

[0113] Each lead is processed independently (12-lead cycle);

[0114] Apply a high-pass filter to eliminate baseline drift;

[0115] Notch filters are used to eliminate power frequency interference;

[0116] Calculate the mean and standard deviation of the signal in this lead;

[0117] Perform z-score standardization: (signal value - mean) / standard deviation;

[0118] Return the processed 12-lead signal matrix.

[0119] Example explanation:

[0120] Input: Raw signal segment from lead II [0.15, 1.32, -0.28, 2.15, 0.98] mV;

[0121] Processing procedure:

[0122] After baseline filtering: [0.18, 1.30, -0.25, 2.12, 1.02] (eliminating 0.3Hz drift);

[0123] After power frequency filtering: [0.17, 1.31, -0.26, 2.13, 1.01] (eliminating 50Hz interference);

[0124] Standardization (μ) i =0.87,σ i =0.92): [-0.76, 0.48, -1.23, 1.37, 0.15];

[0125] Output characteristics: All values ​​are distributed between -1.23 and 1.37, with a mean of 0 and a standard deviation of 1.

[0126] This step eliminates baseline drift and power line interference in the ECG signal through three levels of processing, and then eliminates individual differences through z-score standardization, outputting a standardized 12-lead signal with zero mean and unit variance. This processing provides clean input that conforms to physiological characteristics for subsequent dynamic segmentation, ensuring the comparability of signals from different patients and different devices.

[0127] Step 2: Adaptive segmentation of the cardiac cycle

[0128] This step involves dynamically segmenting the ECG signal based on R-wave detection, adaptively determining the segment length according to the patient's real-time heart rate to ensure that each segment contains a complete cardiac cycle. It includes two core operations: precise R-wave localization and dynamic segmentation.

[0129] 1. R-wave detection: A differential-threshold joint algorithm is used to accurately locate the R-wave peak position.

[0130]

[0131] Amplitude conditions:

[0132]

[0133] Determine the position of the R-wave peak at time t;

[0134] Where κ is the differential threshold coefficient (which can be set to the range of 0.5-0.8), λ is the amplitude threshold coefficient (which can be set to the range of 0.6-0.9), β,γ are the starting index of the analysis window, and δ,∈ is the amplitude detection range index;

[0135] It captures the steep rising edge of the R-wave and dynamically adapts to changes in signal amplitude; points that simultaneously meet the slope and amplitude conditions are identified as R-wave peaks, with a minimum R-wave interval of 200ms to avoid artifacts.

[0136] 2. Dynamic segmentation:

[0137] Length calculation:

[0138]

[0139] in The mean of historical RR intervals; α is the preset proportional coefficient; f s The sampling rate is f and satisfies f s ≥250Hz.

[0140] Boundary defined:

[0141] Centered on the R-wave peak, extend forward by 0.4L and backward by 0.6L to ensure that the complete P-QRS-T composite wave is included.

[0142] Overlap settings:

[0143] Adjacent segments are set to overlap by a preset ratio of 30%, with the starting point of the overlapping area being, for example, the previous R-wave position + 0.7L, generating a temporally continuous segment sequence.

[0144] The pseudocode representation is as follows:

[0145]

[0146]

[0147]

[0148] The steps are as follows:

[0149] Initialize the R-wave location list and minimum R-wave interval (200ms);

[0150] Sliding window traverses the signal (with a 300-point buffer reserved at the boundary);

[0151] Calculate the maximum difference within a range of 150 points before and after the current point;

[0152] Calculate the maximum amplitude within a range of 300 points before and after the current point;

[0153] Check whether both the slope condition and the amplitude condition are met simultaneously;

[0154] If the condition is met and the interval between the current R-wave and the previous R-wave is >200ms, it is recorded as an R-wave.

[0155] Calculate the average of the last 10 RR intervals (use the default value of 0.8s if there are fewer than 10); calculate the segment length L according to the formula;

[0156] Segments (40% front, 60% back) are generated centered on each R wave;

[0157] Add subsegments with 30% overlap between adjacent R waves;

[0158] Returns the sequence of sub-segments.

[0159] Example explanation:

[0160] Patient parameters: mean heart rate

[0161] Sampling rate: f s =500Hz;

[0162] Subsegment length: Sampling points;

[0163] R-wave location: t = 1000;

[0164] Sub-segment range:

[0165] Main segment: [1000-192,1000+288]=[808,1288];

[0166] Overlapping segment: The preceding R wave t = 400 → [400 + 336, 400 + 336 + 480] = [736, 1216];

[0167] Coverage integrity: Ensure coverage from the start of the P wave to the end of the T wave.

[0168] This step uses a differential-threshold joint algorithm to accurately locate the R wave, dynamically calculates the segment length based on real-time heart rate, and generates a segment sequence covering the entire cardiac cycle centered on the R wave, with a 30% overlap region to ensure temporal continuity. This adaptive segmentation strategy effectively solves the cardiac truncation problem caused by a fixed window, providing a physiologically complete input for subsequent feature extraction.

[0169] Step 3: Spatiotemporal Embedding Generation

[0170] This step converts the cardiac electron segment into a high-dimensional feature vector, fusing local morphological features with spatiotemporal location information to form an embedding representation suitable for Transformer processing. It includes three core operations: convolutional feature extraction, positional encoding generation, and feature fusion.

[0171] 1. Convolutional feature extraction:

[0172] Network structure:

[0173]

[0174] Output:

[0175] 2. Location code generation:

[0176] Temporal position encoding: generated using a trainable sine function.

[0177]

[0178]

[0179] Among them Wtime is the trainable weight matrix, pos is the sub-segment time position index, i is the dimension index, D is the embedding dimension, and r is the wavelength cardinality.

[0180] Lead position coding:

[0181] Limb leads (I, II, III, aVR, aVL, aVF) are encoded according to Einthoven's triangle theory, based on vector angles:

[0182] E lead (I)=cos(θ I )·P x +sin(θ I )·P y

[0183] E lead (II)=cos(θ II )·P x +sin(θ II )·P y

[0184] Where θ I and θ II P is the standard vector angle of the lead. x and P y These are trainable projective basis vectors;

[0185] Chest leads (V1-V6) are initialized based on human anatomical coordinates:

[0186]

[0187] Where x j Let W be the anatomical coordinate vector of lead j. coord Let b be the trainable weight matrix and b be the bias vector.

[0188] 3. Feature fusion:

[0189] Integrating morphological features, temporal location, and spatial topological information:

[0190] E final =E emb +E time +E lead

[0191] Preserve the original information of each feature and avoid interactive interference;

[0192] After fusion, layer normalization (LayerNorm) is performed to stabilize the training process and output the dimension.

[0193] The pseudocode representation is as follows:

[0194]

[0195]

[0196]

[0197] The steps are as follows:

[0198] Reshape the input segment into a three-dimensional tensor (1 × length × 1);

[0199] First convolutional layer: 64 filters of length 15, stride 2, extracting QRS wave features;

[0200] Max pooling of length 3 after ReLU activation;

[0201] The second convolutional layer consists of 128 filters of length 7 with a stride of 2, extracting ST-T features.

[0202] Global average pooling yields a 128-dimensional embedding vector E. emb ;

[0203] Generate time position encoding: Generate sine and cosine encodings using the original Transformer method;

[0204] Generate lead position codes based on lead type:

[0205] Limb leads: calculated using standard vector angle projection;

[0206] Chest leads: linearly transformed according to anatomical coordinates;

[0207] The sum of the three features: E emb +E time +E lead Forming spatiotemporal embedding vectors;

[0208] The final embedding is output after LayerNorm normalization.

[0209] Example explanation:

[0210] Input: V4 lead subsegment (480 points), timing position index = 5, wavelength base r = 10000;

[0211] deal with:

[0212] Convolutional network output E emb = [0.12, -0.35, ..., 0.08];

[0213] Time code calculation:

[0214] E time [0] = sin(5 / 10000) 0 )≈0.001

[0215] E time [1] = cos(5 / 10000) 0 )≈0.999

[0216] ...(128-dimensional vector)

[0217] Lead coding (V4 coordinates (0.2, 0.5, 0)):

[0218] Feature fusion: [0.12+0.001+0.25,-0.35+0.999-0.12,...,0.08+...];

[0219] Layer normalized output: [-0.15, 0.78, ..., 0.23];

[0220] Output: 128-dimensional normalized vector (preserving spatiotemporal topology).

[0221] Construction of joint spatiotemporal coding matrix:

[0222] Arrange all segment spatiotemporal embedding vectors of the 12 leads into a two-dimensional matrix according to the time dimension (segment sequence) and the lead dimension (anatomical location): Where 12 represents the number of standard leads and 128 represents the embedding dimension;

[0223] Each element M[t,i,:] in the matrix represents the fusion embedding vector E of the t-th sub-segment of the i-th lead. final This matrix simultaneously encodes the ECG temporal evolution and lead spatial topology, serving as input to the Transformer.

[0224] This step extracts local morphological features of cardiac electron segments using a two-layer convolutional network, and combines temporal location encoding and lead anatomical location encoding to generate a 128-dimensional embedding vector that integrates spatiotemporal information. This representation preserves the physiological structural features of the ECG signal, providing the Transformer with a structured input that includes spatial topology and temporal relationships.

[0225] Step 4: Hierarchical Sparse Transformer Processing

[0226] This step employs an improved Transformer architecture to handle spatiotemporal embedding, achieving efficient modeling of long sequences through local block attention and global keyframe aggregation. It includes four core operations: block processing, local attention, keyframe selection, and global interaction.

[0227] 1. Block processing:

[0228] Segmentation rule: Segmented according to the single heartbeat cycle (seconds).

[0229]

[0230] Where f s T is the sampling rate. heartbeat This refers to the duration of a single heartbeat cycle (approximately 0.25-1.2 seconds).

[0231] Setting the inter-block overlap to 10% will divide the input matrix into N B×D blocks in the output.

[0232] 2. Local block attention:

[0233] Attention mechanism:

[0234]

[0235] Where Q, K, and V are the query, key, and value matrices, respectively, generated by intra-block embedding linear transformations, and d k The dimension of the key vector;

[0236] Multi-head setup: 8 heads, each head with dimension d k =16;

[0237] Position encoding: Relative position encoding within a block;

[0238] Add a learnable relative position encoding matrix within the block.

[0239]

[0240] Where z is the maximum relative distance (default z = 50), i, j are the indices of the time points within the block, and learned embed For trainable relative position embedding matrices;

[0241] Generate position-aware embeddings for each time point within the block, enabling the model to perceive the local temporal relationships of the waveform.

[0242] 3. Keyframe selection:

[0243] QRS energy dynamic selection of keyframes focuses on areas with significant pathological features.

[0244] Selection criteria:

[0245] When the QRS composite wave energy of a signal block exceeds twice the global average, it is forcibly set as a keyframe. The energy calculation formula is as follows:

[0246]

[0247] Where E block For signal block energy, The global average energy is η, and the energy threshold coefficient is η.

[0248] 4. Global-local interaction:

[0249] The global context node G is generated using max pooling:

[0250]

[0251] in For pooling functions, H is the normalization function. k Let this be the hidden state vector of the k-th keyframe. A set of keyframe indexes;

[0252] Stable feature distribution, avoid interference from amplitude differences, capture the most significant pathological features (such as ST segment elevation), and output a summary of key information representing the entire sequence;

[0253] Global-local interaction formula:

[0254] Implement attention-based interaction between global nodes and local features:

[0255]

[0256] Where Qlocal is the query vector for the local block, focusing on the current feature, and G... T This is key information for global nodes;

[0257] Attention weighting: Adjusting the importance of local features (e.g., enhancing ischemia-related features);

[0258] Residual connection: Preserves the original local features (Output = LocalAttn + GlobalAttn).

[0259] In hierarchical sparse Transformer processing, the joint spatiotemporal encoding matrix serves as the input sequence for the encoder. To capture the global context, a learnable CLS flag embedding vector (with the same dimension as the spatiotemporal embedding) is added to the beginning of the input sequence. This flag participates in local block attention and global keyframe aggregation calculations along with the sequence, and its final layer hidden state vector serves as a compressed representation of the entire ECG sequence.

[0260] The pseudocode representation is as follows:

[0261]

[0262]

[0263]

[0264]

[0265] The specific steps are as follows:

[0266] Add a learnable CLS flag embedding vector;

[0267] Calculate the size of a single heartbeat.

[0268] Divide the input sequence into blocks that overlap by 10%;

[0269] Add relative position encoding to each block;

[0270] Perform 8-head self-attention calculations (Q=K=V);

[0271] Calculate the energy E of each block block ;

[0272] Select blocks with energy greater than twice the global average as keyframes;

[0273] Perform layer normalization on keyframes;

[0274] Generate a global node G using max pooling along the time dimension;

[0275] For each block:

[0276] Take the center point as the query vector Q;

[0277] Calculate the attention weights for Q and G;

[0278] Generate global attention features;

[0279] Connect to the feature residuals of the original block.

[0280] Reassemble all blocks into a complete sequence output.

[0281] Example description

[0282] enter:

[0283] 12-lead × 480 time points × 128-dimensional embedding matrix (f s =500Hz);

[0284] Block division: B = 0.25 × 500 = 125 → Divided into 4 blocks (10% overlap);

[0285] Processing flow:

[0286] Keyframe selection:

[0287] Block energy: [8.7, 5.2, 23.5, 7.3];

[0288] Global average energy:

[0289] Threshold:

[0290] Keyframe: Block 3 (Energy 23.5 > 22.35);

[0291] Global node generation:

[0292]

[0293] Global-local interaction:

[0294] Block 3 has a significantly higher attention weight (e.g., weight 0.6), enhancing pathological features (e.g., ST segment elevation).

[0295] This step reduces computational complexity through block-based processing. Within each block, self-attention is performed to capture local features. Keyframes are dynamically selected based on QRS energy to generate global context nodes. Finally, an attention mechanism is used to achieve global-local feature interaction. This hierarchical sparse structure significantly improves the efficiency of long sequence processing while preserving sensitivity to key pathological features.

[0296] Step 5. Joint Loss of Multiple Tasks

[0297] This step designs a multi-task joint loss function to address the class imbalance and temporal inconsistency issues in ECG disease identification. By combining frequency-weighted cross-entropy and temporal consistency loss, the model's sensitivity to rare pathologies and predictive stability are improved.

[0298] 1. Frequency-weighted cross-entropy loss:

[0299]

[0300] in f c ν is the frequency of category c in the training set; C is the total number of disease categories; y c The true label for category c; p c Let be the predicted probability for category c.

[0301] By assigning higher weights to low-frequency disorders (such as ventricular tachycardia) and lower weights to high-frequency disorders (such as sinus rhythm), the imbalance problem commonly found in clinical data can be alleviated.

[0302] 2. Temporal consistency loss:

[0303] Introduce a consistency constraint for adjacent segment predictions to ensure a smooth transition of model output over time:

[0304]

[0305] Where T is the total number of sub-segments, p tLet be the predicted probability vector for the t-th sub-segment;

[0306] Constraining the prediction differences between adjacent segments avoids drastic fluctuations in prediction results between adjacent heartbeats and enhances the model's stability for long sequences.

[0307] 3. Loss function fusion:

[0308] Weighted fusion has two loss functions:

[0309]

[0310] Where ξ is the regularization weight coefficient, and ξ decays linearly with the number of training rounds.

[0311] The pseudocode representation is as follows:

[0312]

[0313]

[0314] The specific steps are as follows:

[0315] Calculate the frequency of occurrence of each disease category in the batch;

[0316] According to the formula Calculate the category weights;

[0317] Calculate the weighted cross-entropy loss

[0318] Traverse all adjacent sub-segment pairs and calculate the L2 difference of the predicted probabilities;

[0319] Calculate the average to get

[0320] The two losses are fused with a weight ξ = 0.1.

[0321] Example explanation:

[0322] Disease categories: ["Normal", "Atrial fibrillation", "Ventricular tachycardia"];

[0323] Category frequency: [0.7, 0.25, 0.05];

[0324] Weight calculation:

[0325] Adjacent segment prediction:

[0326] Sub-segment t: [0.8, 0.1, 0.1];

[0327] Sub-segment t+1: [0.3, 0.6, 0.1];

[0328] Consistency loss:

[0329] Total loss:

[0330] This step addresses the class imbalance problem by using frequency-weighted cross-entropy to assign higher weights to rare pathologies; it also constrains the smooth transition of predictions for adjacent sub-segments through temporal consistency loss; the two are then weighted and fused to form the final optimization objective. This design significantly improves the model's recognition rate for low-frequency pathologies (such as ventricular tachycardia) (by 15-20%), while reducing temporal jitter in the prediction results.

[0331] Step 6. Real-time streaming processing

[0332] This step enables real-time processing of long-sequence ECG signals, avoiding redundant computations through a hidden-state caching mechanism. A hierarchical caching and LRU strategy are employed to improve inference efficiency while maintaining temporal coherence.

[0333] 1. Implicit state cache:

[0334] Cache content:

[0335] Hidden state at level 6:

[0336] Hidden state at level 12:

[0337] Where M is the number of processed segments, and D is the hidden state dimension;

[0338] 2. New sub-segment processing:

[0339] Calculation process:

[0340] Will As input to the first 6 Transformer layers, the output of layers 7 to 12 is computed only for the new segment;

[0341] 3. Cache management (LRU strategy):

[0342] Elimination rules:

[0343] When M>N max Remove the earliest K segments at time;

[0344] N max Set the cache threshold (e.g., 100 segments);

[0345] K represents the amount removed each time (e.g., 10 segments);

[0346] Update mechanism:

[0347] The new segment is processed and appended to the end of the cache;

[0348] Access timestamp records are used for LRU (Least Recently Used) determination.

[0349] The pseudocode representation is as follows:

[0350]

[0351]

[0352] The specific steps are as follows:

[0353] Initialize the cache list and threshold parameters;

[0354] When the cache exceeds N max When K=100, remove the earliest K=10 sub-segments;

[0355] Concatenate the historical cache with the new subfield to form a complete sequence;

[0356] Calculate the hidden states of the entire sequence by inputting the first 6 layers of Transformers;

[0357] Only the hidden state input corresponding to the new sub-segment is taken from the last 6 layers;

[0358] Add the hidden states of the new sub-segment at levels 6 and 12 to the cache;

[0359] Returns the final output of the new sub-segment.

[0360] Example explanation:

[0361] Input: 10-minute ECG, divided into 200 segments;

[0362] Real-time processing:

[0363] Sub-segments 1-100: Full model calculation;

[0364] Sub-segment 101:

[0365]

[0366] The last 6 layers only calculate the new sub-segments;

[0367] Cache update:

[0368] Cache reaches threshold N max =After 100;

[0369] For each new sub-segment processed, the oldest sub-segment (K=1) is discarded;

[0370] Comparison of computational complexity:

[0371] Traditional method: 200 sub-segments × 12 layers = 2400 layers for calculation;

[0372] Streaming processing: 100×12 + 100×6 = 1800 layers (saving 25%).

[0373] This step caches the hidden state of the intermediate layer and reuses historical calculation results, requiring only high-level Transformer calculations for new data. Combined with an LRU eviction policy, it enables real-time processing of long sequences with limited memory. This mechanism reduces the system latency for processing a 10-minute 12-lead ECG to less than 2 seconds, meeting the needs of clinical real-time monitoring.

[0374] Step 7. Multi-lead collaborative noise reduction

[0375] This step integrates a multi-lead collaborative denoising module in the preprocessing stage to automatically detect and repair signal damage caused by motion artifacts. Through frequency domain analysis and spatial correlation reconstruction, it improves signal reliability in noisy environments.

[0376] 1. Motion artifact detection:

[0377] Noise segments are identified using high-frequency energy detection.

[0378] E HF =∑(s*h HPF ) 2

[0379] Judgment conditions:

[0380] If E HF >ζ·σ HF This is then identified as a noise segment;

[0381] Where σ HF Here, is the standard deviation of the high-frequency energy of the entire sequence, s is the time series of the lead signal, and h is... HPF Let ζ be the impulse response of the high-pass filter, and ζ be a predetermined coefficient (a positive real number).

[0382] Detection principle:

[0383] Motion artifacts are characterized by high-frequency abrupt changes (>5Hz);

[0384] Calculate the high-frequency energy for each 1-second window;

[0385] Items exceeding ζ times the standard deviation are considered noise.

[0386] 2. Signal Repair:

[0387] Damaged signals reconstructed based on lead spatial location weighting:

[0388]

[0389] in Let Ω be the reconstructed signal of lead i at time t, Ω be the set of healthy leads, and d be the reconstructed signal of lead i at time t. ijThe standard electrode spacing between leads i and j (based on the AHA lead position diagram), p is the distance attenuation exponent (positive integer), and w ij Let J be the repair weight of lead j relative to lead i.

[0390] The pseudocode representation is as follows:

[0391]

[0392]

[0393]

[0394] The specific steps are as follows:

[0395] Design a 5Hz high-pass filter (cutoff frequency 5Hz) for each lead:

[0396] Calculate the high-frequency energy E after filtering HF

[0397] Calculate the standard deviation σ of the high-frequency energy in all leads. HF If E HF >3.0·σ HF Mark as damaged leads; for each damaged lead i:

[0398] Obtain the set of health leads Ω;

[0399] According to the lead spacing d ij Calculate weight w ij ;

[0400] According to the formula Reconstructing the signal;

[0401] Return the repaired 12-lead signal.

[0402] Example explanation:

[0403] Damaged lead: V3 is determined to be damaged (high-frequency energy exceeds limit).

[0404] Health leads: Ω = {V1, V2, V4, V5};

[0405] Distance calculation:

[0406] d(V3-V1)=5cm, d(V3-V2)=3cm, d(V3-V4)=4cm, d(V3-V5)=6cm;

[0407] Weight calculation (p=2):

[0408] w = [5 -2 ,3 -2 4 -2 6 -2= [0.04, 0.11, 0.06, 0.03];

[0409] Normalized: w = [0.17, 0.46, 0.25, 0.12];

[0410] Signal reconstruction:

[0411]

[0412] This step utilizes high-frequency energy detection of motion artifacts and spatial correlation of healthy leads to reconstruct damaged signals. A weighted strategy based on electrode distance (distance attenuation index p=2) aligns with the physical characteristics of ECG propagation, reducing lead reconstruction errors by 40-60% and significantly improving system robustness in noisy environments.

[0413] Step 8. Model Compression Deployment

[0414] This step addresses the limitations of clinical equipment resources by designing a three-level model compression framework. Through structural simplification, physiologically guided pruning, and knowledge distillation, the original 12-layer model is compressed into a lightweight 6-layer model, reducing computational load.

[0415] 1. Structural compression:

[0416] Compression scheme:

[0417] Original 12-layer encoder → 6-layer encoder-decoder (4 encoding layers + 2 decoding layers);

[0418] The decoder uses linear attention:

[0419] LinearAttn(Q,K,V)=V·softmax(K) T Q

[0420] Complexity comparison:

[0421] Original: O(N) 2 → After compression: O(N);

[0422] Parameter quantity: Example 120M→35M.

[0423] 2. Physiologically guided pruning:

[0424] Attention head-based saliency map scoring pruning of QRS waves:

[0425]

[0426] Where S h The saliency map score for the h-th attention head is given, where H is the total number of attention heads and φ is the predetermined threshold coefficient (typical value 0.4, range 0.3-0.6).

[0427] Pruning rules:

[0428] Calculate the sensitivity of each head to the QRS wave;

[0429] Remove heads with sensitivity below 40% of the maximum value.

[0430] 3. Distillation of Pathological Knowledge:

[0431] Using the original model to guide the learning of the compressed model:

[0432]

[0433] in The output of the teacher model for category c is the logits. For the student model's logits output for class c, T temp Let σ be the temperature coefficient, and σ be the softmax function.

[0434] The pseudocode representation is as follows:

[0435]

[0436]

[0437]

[0438] The specific steps are as follows:

[0439] Construct a 6-layer student model (4-layer encoder + 2-layer linear attention decoder);

[0440] Calculate the saliency map score S for each attentional head of the teacher model on the QRS wave. h ;

[0441] Remove all S h <0.4×max(S k (attention head);

[0442] Distillation training cycle:

[0443] Teacher model generates soft labels p tea =softmax(z) tea / 2.0);

[0444] Student model output p stu =softmax(z) stu / 2.0);

[0445] Calculate KL divergence loss

[0446] The student model is updated by incorporating a 30% cross-entropy loss.

[0447] Example explanation:

[0448] Saliency plot score:

[0449] Head 1: S1 = 0.8 (sensitive to QRS);

[0450] Head 2: S2 = 0.3 (Sensitive to noise)

[0451] First 3: S3 = 0.9 (maximum value);

[0452] Pruning decision: φ = 0.4 → threshold = 0.4 × 0.9 = 0.36;

[0453] The first two heads were removed (0.3 < 0.36);

[0454] Distillation effect:

[0455] Teacher model accuracy: 95.2%;

[0456] Student model distillation: 93.7% (direct training: only 91.2%).

[0457] This step employs a three-tiered optimization approach—encoding / decoding architecture compression, physiologically guided pruning, and knowledge distillation—to reduce computational load by 70% while maintaining model performance (accuracy decrease <2%). The compressed model can run in real-time on mobile devices (such as ECG monitors) with an inference speed of 50 frames per second, meeting clinical deployment requirements.

[0458] Step 9. Gradient-oriented control module

[0459] This step implements a gradient-oriented control strategy during the model training phase. By amplifying key feature gradients and using dynamic pruning mechanisms, it enhances the model's sensitivity to ECG pathological features while ensuring training stability.

[0460] 1. QRS gradient amplification:

[0461] Identify the gradient of the QRS complex at the corresponding time step:

[0462]

[0463] Where ρ is the gradient amplification factor. The parameter gradient corresponding to the QRS time period;

[0464] Enhance the model's ability to identify QRS waveform distortions (such as premature ventricular contractions and bundle branch block).

[0465] 2. Dynamic gradient clipping:

[0466] Monitoring gradient norm to prevent training divergence:

[0467]

[0468] Where τ is the gradient norm threshold. Let L2 norm be the gradient vector;

[0469] When gradient explosion occurs, the gradient is scaled down proportionally.

[0470] The pseudocode representation is as follows:

[0471]

[0472]

[0473] The specific steps are as follows:

[0474] Perform standard forward propagation and loss calculation;

[0475] Obtain the gradient of all parameters;

[0476] Locate the gradient of the QRS complex at the corresponding time step (100-150);

[0477] The gradient in the QRS region is amplified by a factor of 2 (ρ = 2.0);

[0478] Check if the gradient L2 norm exceeds the threshold τ = 1.0;

[0479] If it exceeds the limit, it will be scaled proportionally.

[0480] Used to update parameters after gradient adjustment.

[0481] Example explanation:

[0482] Training scenario: The model learns to identify premature ventricular contractions (PVCs);

[0483] QRS period: Time steps 120-125 in a certain sub-segment are abnormal QRS;

[0484] Gradient change:

[0485] Original gradient value: 0.3 (QRS region);

[0486] After magnification: 0.3 × 2.0 = 0.6;

[0487] Gradient norm: 0.85 (<1.0) → No cropping required;

[0488] Training results: The model's sensitivity to anomalous QRS was improved by 40%.

[0489] This step enhances the model's ability to identify key ECG pathologies by directionally amplifying the QRS feature gradient, and combines dynamic gradient clipping to prevent training divergence, thereby improving the model's accuracy in identifying key pathological features such as premature ventricular contractions and ST-T changes, while reducing training oscillations.

[0490] Application example: Early diagnosis of patients with acute myocardial infarction

[0491] Patient Information:

[0492] Gender and Age: Male, xx years old;

[0493] Chief complaint: Sudden onset of severe, squeezing pain behind the sternum, lasting for 30 minutes without relief, accompanied by profuse sweating and nausea;

[0494] Initial physical signs: blood pressure 160 / 100 mmHg, heart rate 110 beats / min (sinus tachycardia), slightly rapid breathing;

[0495] Clinical suspicion: Acute coronary syndrome (ACS), with a high suspicion of acute myocardial infarction (AMI).

[0496] System processing flow:

[0497] 1. Multi-lead signal preprocessing (highlighting key technical points):

[0498] Input: A standard 12-lead electrocardiograph connected to the patient was used to acquire the initial signal (sampling rate 500Hz); the signal baseline showed significant drift (patient agitation), and significant electromyographic interference (muscle tremor) was present in lead V3.

[0499] deal with:

[0500] Standardization: The system automatically applies a 0.5Hz high-pass filter to eliminate baseline drift and a 50Hz notch filter to eliminate power frequency interference. It also performs z-score standardization on all leads to make the signal mean 0 and the variance 1, eliminating individual and equipment differences.

[0501] Collaborative denoising: The system detected an abnormally high frequency energy increase in lead V3 (exceeding 3 times the global standard deviation), which was determined to be motion artifact damage. Utilizing the spatial relationship of multiple leads, the system automatically calculated the repair weights of leads V1, V2, V4, and V5 on V3 based on the AHA standard lead spacing (leads closer to V3 have a higher weight), reconstructing a clean lead V3 signal in real time and restoring the ST segment morphology masked by noise.

[0502] 2. Adaptive segmentation of the cardiac cycle:

[0503] The patient has a rapid heart rate (~110 bpm) and occasional premature beats, with an unstable RR interval. Fixed window segmentation easily truncates the P wave or T wave, disrupting the waveform integrity.

[0504] deal with:

[0505] Adaptive Segmentation: The system employs a differential-threshold joint algorithm to precisely locate each R-wave peak. Based on real-time calculated historical average RR intervals and a preset scaling factor α, the current segment length is dynamically determined (e.g., covering approximately 1.15 cardiac cycles). Segment boundaries are extended forward and backward based on the R-wave (40% before and 60% after), ensuring each segment contains a complete P-QRS-T waveform. Adjacent segments have a 30% overlap to guarantee temporal continuity. Ultimately, the 10-minute recording is segmented into 165 continuous and waveform-complete segments.

[0506] 3. Spatiotemporal embedding generation:

[0507] deal with:

[0508] Embedded fusion: for each cardiac segment:

[0509] Local morphological features (such as QRS wave width, ST segment morphology, and T wave polarity) are extracted using a lightweight convolutional network (e.g., a two-layer CNN) to generate a 128-dimensional embedding vector (E). emb ).

[0510] Superimposed with a learnable temporal location code (E time ), marking the temporal order of the sub-segment within the entire sequence.

[0511] Superimposed with an anatomical topology-based lead location code (E lead ):

[0512] For limb leads (I,II,III,aVR,aVL,aVF), the encoding is based on Einthoven's triangle theory and initialized according to its standard vector angle.

[0513] For the thoracic leads (V1-V6), the coding is initialized based on their standard anatomical coordinates on the human chest wall (e.g., V4 is located at the 5th intercostal space on the left midclavicular line).

[0514] The three are added together and then layer-normalized to generate the final spatiotemporal fusion embedding vector (E). final This vector simultaneously contains ECG morphology, temporal information, and lead spatial relationships.

[0515] 4. Multi-lead spatiotemporal coding:

[0516] Processing: Spatiotemporal fusion embedding vector (E) of all segments of the 12 leads final The data are arranged into a structured two-dimensional matrix according to the "time dimension (segment sequence) x lead dimension (anatomical location)". This matrix is ​​a joint representation of the evolution of long-sequence ECG signals in space (multi-lead) and time, and serves as the input for the subsequent Transformer.

[0517] 5. Hierarchical sparse Transformer processing:

[0518] A long sequence of 165 segments lasting 10 minutes requires a huge amount of computation for fully connected attention.

[0519] deal with:

[0520] Block processing: Divide the long sequence into multiple smaller blocks based on the number of sampling points covering a single heartbeat (e.g., each block covers about 1 heartbeat).

[0521] Local block full attention: Within each block, the Transformer layer performs full multi-head self-attention computation, finely capturing the waveform features and their interrelationships within the block (single heartbeat or adjacent heartbeats) (such as the correlation between QRS waves and T waves).

[0522] Global keyframe aggregation: The system dynamically analyzes the physiological characteristics (such as QRS complex energy) of each signal block. When the QRS energy of a block is detected to be significantly higher than the global average (e.g., more than twice), the block is forcibly marked as a "key frame," which usually corresponds to the region with the most significant pathological features (such as the period of severe ST segment elevation in this case).

[0523] Global-Local Interaction: The hidden states of all keyframes are normalized and max-pooled to generate a "Global Context Node" representing key pathological information of the entire sequence. This global node then interacts with each local block through an attention mechanism, injecting key global information (such as anterior wall myocardial ischemia patterns) into the local features, guiding the model to more accurately interpret each heartbeat, especially those with less obvious pathological features. The final output is a feature representation containing deep contextual information.

[0524] 6. Disease classification output:

[0525] deal with:

[0526] Extract the hidden state vector of the [CLS] flag specially added to the final layer of the Transformer, which gathers comprehensive information from the entire long-sequence electrocardiogram.

[0527] This vector is input into the classification layer, and the output is the probability distribution of various heart diseases.

[0528] Multi-task joint loss (reflected during training): During the model training phase, the system uses:

[0529] Frequency-weighted cross-entropy loss: assigns higher weights (based on their frequency in the training set) to relatively rare critical categories such as acute myocardial infarction, solves the data imbalance problem, and improves the recognition rate of these key pathologies.

[0530] Temporal consistency loss: This constraint prevents drastic jumps in prediction results between adjacent segments (e.g., a segment previously predicted as normal, but the next segment suddenly predicted as myocardial infarction), ensuring the smoothness and stability of long-sequence diagnostic results. The weighted sum of the two losses serves as the overall optimization objective.

[0531] 7. Real-time streaming processing:

[0532] Need: Clinical practice requires continuous monitoring of patients' electrocardiogram changes.

[0533] deal with:

[0534] The system continuously caches the hidden state matrix of historically processed segments in the intermediate layers (such as layer 6) and the final layer (layer 12) of the Transformer.

[0535] When newly acquired cardiac electron segments are input:

[0536] The system concatenates the cached intermediate hidden state with the new sub-segment.

[0537] The output of the Transformer is calculated only for new segments (such as layers 7 to 12), which greatly reduces the amount of computation.

[0538] The cache space is managed using the Least Recently Used (LRU) strategy. When the number of cached segments exceeds a threshold (e.g., 100 segments), the oldest cached segment is removed.

[0539] Results: In this example, the system's processing delay for each new heartbeat (segment) is controlled within 1 second, achieving near real-time monitoring and early warning of acute electrocardiographic events.

[0540] 8. Gradient-oriented control module:

[0541] Objective: To improve the model's sensitivity to key pathological features of electrocardiograms (such as ST segment changes in this case).

[0542] Processing (during training):

[0543] QRS gradient amplification: During backpropagation parameter updates, the system identifies the gradients generated by the model parameters at the time steps corresponding to the QRS complex wave. These gradients in key regions are amplified in a targeted manner (e.g., amplified by a factor of 2), forcing the model to pay more attention to QRS morphological distortions (such as pathological Q waves) and the subsequent ST-T segment changes.

[0544] Dynamic gradient clipping: To prevent gradient explosion caused by scaling operations or training itself, the system monitors the gradient magnitude (L2 norm) of all parameters in real time. Once the gradient exceeds a preset safety threshold, it is scaled proportionally to ensure the stability of the training process.

[0545] Results: In this case, this mechanism significantly enhanced the model’s ability to identify subtle ST segment elevation (a key feature in this case) and pathological Q waves.

[0546] 9. Model compression deployment:

[0547] Requirement: To operate in real time on bedside monitors or mobile devices with limited resources.

[0548] deal with:

[0549] Structural compression: Replacing the original 12-layer Transformer encoder with a lighter 6-layer architecture (e.g., a 4-layer encoder + 2-layer decoder with linear attention) reduces the computational complexity from O(N) layers. 2 ) decreased to O(N).

[0550] Physiologically guided pruning: The contribution of each "attention head" in the model to key ECG features such as the QRS wave was analyzed (saliency map score). Attention heads that were insensitive to key features (heads with scores significantly lower than the maximum value) were removed to further simplify the model.

[0551] Pathological knowledge distillation: This method utilizes a well-trained, high-performance original 12-layer model (the teacher model) to guide the training of a compressed 6-layer model (the student model). The student model not only learns the real labels but, more importantly, learns the "soft labels" output by the teacher model (containing rich information such as inter-category similarity), enabling the smaller model to approximate the diagnostic capabilities of the larger model.

[0552] Results: The compressed model ran efficiently on the embedded system of the bedside monitor, with an inference speed of over 50 frames per second, meeting the real-time requirements of clinical practice while maintaining high diagnostic accuracy (in this case, acute myocardial infarction was accurately identified).

[0553] System diagnostic output and clinical validation:

[0554]

[0555]

[0556] Summary of Implementation Examples:

[0557] This embodiment fully demonstrates the workflow of an ECG disease recognition system based on the Transformer architecture: starting with multi-lead signal preprocessing, noise interference is eliminated through adaptive filtering and standardization; dynamic heartbeat segmentation technology is applied to preserve the complete physiological structure; spatiotemporal coding is fused to generate ECG feature representations; hierarchical sparse Transformer is used to capture long-sequence pathological associations; gradient modulation and multi-task optimization are combined to improve the sensitivity of key features; and finally, real-time clinical diagnosis is achieved through streaming processing and model compression. The system successfully identified key pathological patterns such as persistent ST-segment depression in patients with coronary artery disease and generated emergency intervention suggestions. The entire process significantly improves analysis efficiency while ensuring diagnostic accuracy, verifying its clinical application value in early warning of complex cardiac diseases.

Claims

1. A disease identification system based on long-sequence electrocardiogram (ECG) signals using the Transformer architecture, characterized in that, include: (a) Multi-lead signal preprocessing: Configured to receive raw multi-lead ECG signals, baseline drift is eliminated by a low-frequency cutoff high-pass filter, power frequency notch filter is used to eliminate power supply interference, and each lead signal is normalized to zero mean and unit variance by z-score normalization. in This represents the original sampled value of lead i at time t. (i = 1, 2, ..., 12), μ i and σ i These are its mean and standard deviation, respectively; (b) Adaptive segmentation of the cardiac cycle: The normalized signal output from step (a) is dynamically segmented into sub-segments based on R-wave detection, and the sub-segment length L is determined by physiological parameters. in The mean of the historical RR intervals; α is the preset proportional coefficient; f s The sampling rate is f and satisfies f s ≥250Hz; Adjacent segments are set to overlap by a preset ratio to generate a temporally continuous sequence of segments; (c) Spatiotemporal embedding generation: Extracting features from segments and passing them through a convolutional network to generate embedding vectors. Learnable positional codes are then superimposed to form a spatiotemporal embedding vector; (d) Multi-lead spatiotemporal coding: The spatiotemporal embedding vectors of each lead are arranged in a time-lead two-dimensional matrix, where the time dimension position code E time It uses a trainable sine function to generate lead-dimensional position encoding E. lead Based on the standard lead anatomy topology initialization, a joint spatiotemporal coding matrix is ​​constructed; (e) Layer-based sparse Transformer processing: The spatiotemporal coding matrix is ​​input into a multi-layer improved encoder, and each layer performs the following: Local block full attention: Using the number of sampling points covering a single heartbeat cycle as the basic block, multi-head self-attention is calculated within the block; Global keyframe aggregation: Dynamically select keyframes based on signal features, generate global context nodes through pooling operations and interact with each block, and output a deep feature representation; (f) Disease classification output: Extract the hidden state vector of the [CLS] flag bit of the final layer of Transformer and output the probability distribution of multiple heart diseases.

2. The long-sequence electrocardiogram signal disease identification system based on Transformer architecture according to claim 1, characterized in that, Step (b) of adaptive cardiac cycle segmentation further includes the following: R-wave detection employs a differential-threshold joint algorithm: if signal point s t satisfy And simultaneously satisfy Then, time t is determined to be the position of the R-wave peak, where κ is the differential threshold coefficient, λ is the amplitude threshold coefficient, β,γ are the starting index of the analysis window, and δ,∈ is the amplitude detection range index; The segment boundaries are extended forward and backward based on the R-wave position to ensure that each segment contains an integer number of complete cardiac cycles.

3. The long-sequence electrocardiogram signal disease identification system based on the Transformer architecture according to claim 1, characterized in that, The lead position code E mentioned in step (d) lead The initialization method includes differentiated processing for limb leads and chest leads: Limb leads generate codes based on Einthoven's triangle theory, using vector angles: E lead (I)=cos(θ I )·P x +sin(θ I )·P y E lead (II)=cos(θ II )·P x +sin(θ II )·P y Where θ I and θ II P is the standard vector angle of the lead. x and P y These are trainable projective basis vectors; Chest leads are initialized based on human anatomical coordinates: Where x j Let W be the anatomical coordinate vector of lead j. coord Let b be the trainable weight matrix and b be the bias vector.

4. The long-sequence electrocardiogram signal disease identification system based on Transformer architecture according to claim 1, characterized in that, The global keyframe aggregation operation in step (e) includes: Dynamic keyframe selection mechanism: When the QRS composite energy of a signal block exceeds twice the global average, it is forcibly set as a keyframe. The energy calculation formula is as follows: Where E block For signal block energy, The global average energy is η, and the energy threshold coefficient is η. The global context node G is generated using max pooling: in For pooling functions, H is the normalization function. k Let this be the hidden state vector of the k-th keyframe. This is a set of keyframe indexes.

5. The long-sequence electrocardiogram signal disease identification system based on the Transformer architecture according to claim 1, characterized in that, The system includes a gradient orientation control module, which performs the following operations during the model training phase: The gradient of the QRS complex at the corresponding time step is amplified to enhance the model's sensitivity to key ECG features: Where ρ is the gradient amplification factor. The parameter gradient corresponding to the QRS time period; Dynamic gradient clipping strategy to prevent training divergence: If the gradient norm of parameter W is... If the value exceeds the threshold τ, it will be scaled proportionally: Where τ is the gradient norm threshold. Let L2 be the norm of the gradient vector.

6. The long-sequence electrocardiogram signal disease identification system based on the Transformer architecture according to claim 1, characterized in that, The disease classification in step (f) uses a multi-task joint loss function, which includes: The main loss function is frequency-weighted cross-entropy, which addresses the class imbalance problem. in f c ν is the frequency of category c in the training set; C is the total number of disease categories; y c The true label for category c; p c Let c be the predicted probability of category c; The temporal consistency loss function constrains the prediction smoothness of adjacent segments: Where T is the total number of sub-segments, p t Let be the predicted probability vector for the t-th sub-segment; The total loss function is a weighted sum: Where ξ is the regularization weight coefficient.

7. The long-sequence electrocardiogram signal disease identification system based on the Transformer architecture according to claim 1, characterized in that, The system supports real-time streaming processing, achieved through a hidden state caching mechanism: The hidden state matrix of the continuously cached historical segments at levels 6 and 12. and Where M is the number of processed segments, and D is the hidden state dimension; When the new subsegment new When inputting, As input to the first 6 Transformer layers, the output of layers 7 to 12 is computed only for the new segment; The Least Recently Used (LRU) strategy is used to manage the cache: when the number of cached segments exceeds a predetermined threshold N... max When the time comes, remove the hidden states corresponding to the earliest generated K sub-segments.

8. The long-sequence electrocardiogram signal disease identification system based on the Transformer architecture according to claim 1, characterized in that, The preprocessing stage integrates a multi-lead collaborative denoising module, which includes: Motion artifact detection: Calculating the high-frequency band energy E of the signal HF =∑(s*h HPF ) 2 If E HF >ζ·σ HF This is then identified as a noise segment, where σ HF Here, is the standard deviation of the high-frequency energy of the entire sequence, s is the time series of the lead signal, and h is... HPF Let ζ be the impulse response of the high-pass filter, and ζ be a predetermined coefficient (a positive real number). Signal restoration based on lead spatial relationships: If lead i is determined to be damaged, its signal is reconstructed using healthy leads. in Let Ω be the reconstructed signal of lead i at time t, Ω be the set of healthy leads, and d be the reconstructed signal of lead i at time t. ij The standard electrode spacing between leads i and j (based on the AHA lead position diagram), p is the distance attenuation exponent (positive integer), and w ij Let J be the repair weight of lead j relative to lead i.

9. The long-sequence electrocardiogram signal disease identification system based on the Transformer architecture according to claim 1, characterized in that, The system deployment phase employs a model compression framework based on ECG feature perception, which includes three levels of optimization: Structural compression: The original 12-layer Transformer is replaced with a 6-layer encoding / decoding architecture, where: The encoder layer retains the local-global attention mechanism; The decoder layer employs lightweight linear attention, reducing the complexity to O(N); Physiologically guided weighted pruning: Calculate the saliency plot S of each attention head on the QRS wave characteristics. h ,like Then cut off the head, where S h The saliency map score for the h-th attention head is given, where H is the total number of attention heads and φ is the predetermined threshold coefficient. Distillation of Pathological Knowledge: The teacher model (original 12 layers) guides the student model (6 layers) to output probability distribution: in For the teacher model, the logits output for category c. For the student model's logits output for class c, T temp Let σ be the temperature coefficient, and σ be the softmax function.

Images