Electrocardiogram atrial fibrillation prediction method based on multi-mode deep learning

Abstract

The invention relates to the technical field of electrocardiogram detection, in particular to an electrocardiogram atrial fibrillation prediction method based on multi-modal deep learning, which comprises the following steps: S1, collecting electrocardiogram signal data, S2, performing signal preprocessing and feature fusion, and S3, performing model training and atrial fibrillation prediction. According to the method, multi-modal features are extracted through a complex multi-scale attention mechanism and a feature self-adaptive attention mechanism, a deep learning model based on the combination of a convolutional neural network and a long-short term memory network is combined, multi-dimensional features and time sequence information of electrocardiosignals are fully utilized, and the multi-modal features of the electrocardiosignals are extracted. Therefore, the efficiency, the accuracy and the reliability of the atrial fibrillation prediction method are greatly improved, and the problem that the sensitivity and the specificity cannot meet the requirements when the prior art faces complex and diversified electrocardiosignals is solved.

Description

Technical Field

[0001] The present invention relates to the field of electrocardiogram detection technology, and specifically to an electrocardiogram atrial fibrillation prediction method based on multimodal deep learning. Background Art

[0002] Heart disease is a common clinical condition today. One of the possibilities of remotely monitoring heart health based on real-time heartbeats is to detect heart health. Electrocardiographs are a simple and effective method to define, detect and diagnose arrhythmias. In medicine, arrhythmias are characterized by heart rhythms that do not follow the normal sinus pattern. There are 12 main types of abnormal arrhythmias. Atrial fibrillation is one of the most important abnormal heart rhythms characterized by rapid and irregular heartbeats. Automatic detection of arrhythmias can significantly help patients get emergency treatment and avoid further trouble.

[0003] Traditional atrial fibrillation detection technologies primarily rely on waveform analysis and template matching. These methods rely heavily on specific ECG features and their preset thresholds, making it difficult to accurately identify complex, nonlinear, and variable ECG signals. Traditional methods are particularly prone to misjudgment or omission in the early stages of atrial fibrillation, as well as in high-noise, diverse individuals (such as the elderly, newborns, or those after exercise). Furthermore, while template matching is somewhat effective in detecting abnormalities with significantly different R-wave morphologies, it remains difficult to effectively capture early atrial fibrillation, which exhibits non-periodic, subtle variations. These limitations prevent traditional methods from meeting the stringent clinical requirements for high sensitivity and specificity when faced with signal complexity and diversity. Summary of the Invention

[0004] In response to the above problems, the present invention provides an electrocardiogram atrial fibrillation prediction method based on multimodal deep learning to solve the problem that the existing technology cannot meet the clinical requirements of high sensitivity and high specificity for atrial fibrillation detection when faced with complex and diverse signals.

[0005] To achieve the above objectives, the present invention provides an electrocardiogram atrial fibrillation prediction method based on multimodal deep learning, comprising the following steps: S1. Collect ECG signal data: obtain raw ECG signal data from the clinical equipment database; S2. Signal preprocessing and feature fusion: S2-1: Preprocess the original ECG signal data and extract the preprocessing signal S_pre; S2-2: Perform R wave peak detection and RR interval sequence calculation on the preprocessed signal S_pre to obtain the RR interval sequence S_rr, perform normalization and interpolation filling on the RR interval sequence S_rr to obtain time series data of uniform length; S2-3: Using a complex multi-scale attention mechanism and a feature-adaptive attention mechanism, the time series data obtained in step S2-2 is weighted and fused to generate multimodal features; The complex multi-scale attention mechanism is used to perform feature weighting and fusion on time series data. The complex multi-scale attention mechanism is used to perform multi-scale weighting on the hidden representation of time series data, capture the feature correlation at different scales, and optimize the expressiveness of features. The calculation formula of the complex multi-scale attention mechanism is as follows: in, represents the attention weight matrix, Indicates the The trainable scaling factor of each scale, represents a value vector, Indicates the scale number, Represents element-wise multiplication; The feature-adaptive attention mechanism is used to further perform dynamic feature weighting and fusion on the time series data processed by the complex multi-scale attention mechanism. Through the feature-adaptive attention mechanism, trainable attention weights are used to dynamically weight features, highlight key features and suppress redundant features. The calculation formula of the feature-adaptive attention mechanism is as follows: in, represents the feature vector, represents the trainable attention weights, Represents element-wise multiplication.

[0006] S3. Model training and atrial fibrillation prediction: Multimodal features are used to train a deep learning model based on a combination of convolutional neural networks and long short-term memory networks. ECG signals are input into the trained model to predict atrial fibrillation and output the classification results.

[0007] Furthermore, in step S2-1, preprocessing the original ECG signal data includes the following steps: a. Improve the data quality of the collected raw ECG signal data through image processing technology, perform image cropping on the raw ECG signal data, dynamically adjust the cropping area according to the lead type of the raw ECG signal data, and accurately extract the signal information of each lead; b. Perform Gaussian blur processing and binarization processing on the signal extracted in step a to extract the preprocessed signal S_pre with a clear signal contour.

[0008] Furthermore, the step S2-2 specifically includes the following steps: c. Using an improved dynamic threshold algorithm on the preprocessed signal S_pre, calculate the dV(t) / dt change rate and locate the R peak to obtain the timestamp sequence T_R; d. Based on the timestamp sequence T_R, the RR interval sequence S_rr is formed according to the adjacent time difference Δt_k = T_R(k+1)-T_R(k), and normalized and interpolated to output time series data of uniform length for multimodal feature extraction.

[0009] Furthermore, in step S2-3, the multimodal features include heart rate variability (HRV) features, topological data analysis (TDA) features, and phase space reconstruction features.

[0010] Furthermore, the heart rate variability HRV features include RMSSD, pNN50, SDNN, average heart rate and HRV triangular index; the calculation formulas of the RMSSD, the pNN50 and the SDNN are as follows: in, It is RR intervals, is the average value of the RR interval.

[0011] Furthermore, the topological data analysis (TDA) features include persistence entropy, Betti curve and persistence landscape extracted using a persistence homology algorithm.

[0012] Furthermore, in step S3, the specific steps of model training and atrial fibrillation prediction are as follows: S3-1. Spatial feature extraction: The preprocessed signal S_pre is converted into a two-dimensional electrocardiogram matrix and input into the convolutional neural network CNN. CNN extracts spatial information such as local waveform morphology and cross-lead correlation through multi-layer convolution-pooling operations to obtain a spatial feature vector, which is expanded and recorded as F_spa. S3-2, Pooling Dimensionality Reduction: Perform global or hierarchical pooling on F_spa to reduce its dimension to one-dimensional spatial features, reducing the amount of computation and the risk of overfitting; after pooling, the feature order still remains one-to-one with the original time series; S3-3. Multimodal feature fusion: The one-dimensional spatial features expanded by pooling are concatenated with the RR interval sequence features, HRV features, TDA features, and phase space reconstruction features calculated in step S2 in the feature channel dimension to form a unified high-dimensional feature vector B. This vector contains morphological, rhythmic, and topological information, providing comprehensive semantics for subsequent time series modeling. S3-4, Timing Modeling: The high-dimensional feature vector B is input into the bidirectional long short-term memory network LSTM according to the order of the heartbeat sequence; LSTM captures the temporal dependency and rhythm changes between ECG events and outputs the hidden state sequence F_tem; S3-5, attention strengthening: A complex multi-scale attention mechanism and feature adaptive attention mechanism are embedded in LSTM, and a weighting strategy is applied to the time segments in the hidden state sequence F_tem to highlight the segments that have a significant impact on the classification results; the weighted time series representation is recorded as a high-dimensional feature vector C; S3-6, Dimensionality reduction mapping and classification: The high-dimensional feature vector C is input into the fully connected dimensionality reduction mapping layer, and then the Softmax activation function is used to output the probability ŷ∈[0, 1] of atrial fibrillation AF and normal heart rate NSR; when ŷ ≥ the threshold, it is judged as atrial fibrillation, otherwise it is judged as normal rhythm.

[0013] Beneficial effects of the present invention: The present invention uses a complex multi-scale attention mechanism and a feature-adaptive attention mechanism to extract multimodal features. Combined with a deep learning model based on a convolutional neural network and a long short-term memory network, it fully utilizes the multidimensional features and temporal information of the ECG signal, thereby greatly improving the efficiency, accuracy and reliability of the atrial fibrillation prediction method and solving the problem that the sensitivity and specificity of the existing technology cannot meet the requirements when facing complex and diverse ECG signals. The present invention utilizes the fusion of multimodal features to enable the model to comprehensively analyze the multidimensional characteristics of ECG signals, thereby making full use of the complementarity between different features, improving the model's discrimination ability, and reducing misjudgments and missed judgments caused by insufficient single features; The present invention adopts a deep learning model combining CNN-LSTM to automatically learn and extract complex spatiotemporal features and time series features, further enhancing the model's ability to learn and recognize complex electrocardiogram signals. BRIEF DESCRIPTION OF THE DRAWINGS

[0014] In order to more clearly illustrate the technical solution of the present invention, the following briefly introduces the drawings required for use in the embodiments or descriptions of the prior art. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative work.

[0015] Figure 1 This is a flowchart of the electrocardiogram atrial fibrillation prediction method based on multimodal deep learning; Figure 2 This is the electrocardiogram data processing diagram; Figure 3 Schematic diagram of the multimodal feature fusion deep learning process; Figure 4 For deep learning model architecture; Figure 5 This is the R peak detection result diagram. DETAILED DESCRIPTION

[0016] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. All other embodiments obtained by ordinary technicians in this field based on the embodiments of the present invention without making any creative efforts shall fall within the scope of protection of the present invention. Example

[0017] A multimodal deep learning-based electrocardiogram atrial fibrillation prediction method, such as Figure 1 As shown, the specific steps include: S1. Collect ECG signal data: Obtain raw ECG signal data from the clinical equipment database. Raw ECG signals typically contain electrical activity data from multiple leads. Each lead records the electrophysiological activity of the heart in different directions, providing comprehensive cardiac electrical activity information. This ensures that each lead signal maintains accuracy and integrity during subsequent processing, avoiding signal confusion or information loss. S2, signal preprocessing and feature fusion, ECG processing diagram as shown Figure 2 As shown: S2-1: Preprocess the original ECG signal data and extract the preprocessed signal S_pre. The preprocessing steps are as follows: a. For the collected raw ECG signal data, image processing technology is used to improve data quality and perform image cropping. ECG signals of different leads have different time intervals and characteristics. The cropping area is dynamically adjusted according to the different lead types to accurately extract the information of each lead signal. The original ECG image is divided into multiple parts, each part corresponds to the time series data of a lead signal, ensuring the accuracy of each lead signal. b. Performing Gaussian blur processing and binarization processing on the lead signal extracted in step a to extract a preprocessed signal S_pre with a clear signal contour; Gaussian blur processing can reduce noise in the lead signal and make the signal curve of the lead signal smoother; binarization processing can convert the image into black and white, making the signal contour clearer and facilitating subsequent feature extraction; S2-2: Perform R wave peak detection and RR interval sequence calculation on the preprocessed signal S_pre to obtain the RR interval sequence S_rr. Normalize and interpolate the RR interval sequence S_rr to obtain time series data of uniform length. The specific steps include the following: c. Using an improved dynamic threshold algorithm on the preprocessed signal S_pre, calculate the dV(t) / dt change rate and locate the R peak to obtain the timestamp sequence T_R; like Figure 5 As shown, detecting the R wave peak is a key step in electrocardiogram analysis. The R wave is an important feature of the electrocardiogram signal, representing the depolarization process of the heart. In this embodiment, a dynamic threshold algorithm is used to detect the R wave peak. Specifically, the method calculates the rate of change of the electrocardiogram signal in the time domain and detects the maximum slope of the R wave, thereby accurately determining the position of the R wave. Accurate detection of the R wave provides the basis for the subsequent calculation of the RR interval sequence. The detection formula for the R wave peak is as follows: Where, Indicates the peak value of the R wave, / Indicates the rate of change of the ECG signal in the time domain. The R wave is located by detecting the maximum slope of the ECG signal change. d. Based on the timestamp sequence T_R, the RR interval sequence S_rr is formed according to the adjacent time difference Δt_k = T_R(k+1)-T_R(k), and normalized and interpolated to output time series data of uniform length for multimodal feature extraction.

[0018] S2-3: Using a complex multi-scale attention mechanism and a feature-adaptive attention mechanism, the time series data obtained in step S2-2 is weighted and fused to generate multimodal features; The complex multi-scale attention mechanism is used to perform feature weighting and fusion on time series data. It uses the complex multi-scale attention mechanism to perform multi-scale weighting on the hidden representation of time series data, capture the feature correlation at different scales, and optimize the expressiveness of features. The calculation formula of the complex multi-scale attention mechanism is as follows: in, represents the attention weight matrix, Indicates the The trainable scaling factor of each scale, represents a value vector, Indicates the scale number, Represents element-wise multiplication; The feature-adaptive attention mechanism is used to further perform dynamic feature weighting and fusion on time series data processed by the complex multi-scale attention mechanism. Through the feature-adaptive attention mechanism, trainable attention weights are used to dynamically weight features, highlight key features and suppress redundant features. The calculation formula of the feature-adaptive attention mechanism is as follows: in, represents the feature vector, represents the trainable attention weights, Represents element-wise multiplication; Multimodal features include heart rate variability (HRV) features, topological data analysis (TDA) features, and phase space reconstruction features.

[0019] HRV features are widely used in heart disease prediction and are closely related to the occurrence and development of atrial fibrillation. HRV features include RMSSD (root mean square deviation), pNN50 (heart rate variability percentage), SDNN (standard deviation), average heart rate, and HRV triangular index, which can effectively reflect the variability of heart rhythm. The calculation formulas for RMSSD, pNN50, and SDNN are as follows: in, It is RR intervals, is the average value of the RR interval.

[0020] TDA features can describe the geometric and topological structure of ECG signals. Especially when analyzing atrial fibrillation, a non-periodic and complex cardiac rhythm, they can provide more structural information than traditional features. The topological data analysis (TDA) method is used, specifically including: using the persistent homology algorithm to extract features such as persistent entropy, Betti curves, and persistent landscapes; Persistence entropy reflects the topological complexity of data. The calculation formula of persistence entropy is: in, It is The persistence (duration) of the persistence entries, n is the number of entries. The higher the persistence entropy, the more complex the topology. The Betti curve describes the changes in topological features by calculating the Betti number (topological connectivity) at each scale. The Betti number is used to represent structural features such as connected branches, loops, and holes in the data. Generally, the Betti curve plots the relationship between scale (or time) and the Betti number, which can be expressed by the following formula:

[0021] Where k represents the dimension of the Betti number, t represents the scale, represents the number of connected branches, represents the number of rings, Represents the number of holes; Persistence landscapes represent the topological characteristics of data by mapping persistence entries to generate curves. Persistence landscapes are functions of persistence entry graphs, representing topological characteristics at different scales, and are usually displayed in the form of images. The phase space feature is to generate a multi-dimensional embedding vector using the phase space reconstruction method, construct the phase space using time delay and embedding dimension, and flatten the embedding vector into a fixed dimension.

[0022] S3. Model training and atrial fibrillation prediction: Use multimodal features to train a deep learning model based on a combination of convolutional neural networks and long short-term memory networks, input ECG signals into the trained model to predict atrial fibrillation, and output classification results; Figure 3 As shown, the specific steps include: S3-1. Spatial feature extraction: The preprocessed signal S_pre is converted into a two-dimensional electrocardiogram matrix and input into the convolutional neural network CNN. CNN extracts spatial information such as local waveform morphology and cross-lead correlation through multi-layer convolution-pooling operations to obtain a spatial feature vector, which is expanded and recorded as F_spa. S3-2, Pooling Dimensionality Reduction: Perform global or hierarchical pooling on F_spa to reduce its dimension to one-dimensional spatial features, reducing the amount of computation and the risk of overfitting; after pooling, the feature order still remains one-to-one with the original time series; S3-3. Multimodal feature fusion: The one-dimensional spatial features expanded by pooling are concatenated with the RR interval sequence features, HRV features, and TDA features calculated in step S2 in the feature channel dimension to form a unified high-dimensional feature vector B. This vector contains morphological, rhythmic, and topological information, providing comprehensive semantics for subsequent time series modeling. S3-4, Timing Modeling: The high-dimensional feature vector B is input into the bidirectional long short-term memory network LSTM according to the order of the heartbeat sequence; LSTM captures the temporal dependency and rhythm changes between ECG events and outputs the hidden state sequence F_tem; S3-5, attention strengthening: A complex multi-scale attention mechanism and feature adaptive attention mechanism are embedded in LSTM, and a weighting strategy is applied to the time segments in the hidden state sequence F_tem to highlight the segments that have a significant impact on the classification results; the weighted time series representation is recorded as a high-dimensional feature vector C; S3-6, Dimensionality reduction mapping and classification: The high-dimensional feature vector C is input into the fully connected dimensionality reduction mapping layer, and then the Softmax activation function is used to output the probability ŷ∈[0, 1] of atrial fibrillation AF and normal heart rate NSR; when ŷ ≥ threshold, it is judged as atrial fibrillation, otherwise it is judged as normal rhythm. Example

[0023] like Figure 4 As shown in Figure 1, the deep learning model architecture includes an input layer, a CNN-Pooling feature extraction layer, an LSTM-Attention time series modeling layer, and a fully connected layer. The input layer is used in step S2, signal preprocessing and feature fusion. The CNN-Pooling feature extraction layer is used in steps S3-1 to S3-3 to extract spatial features from the preprocessed multi-lead ECG matrix. The LSTM-Attention time series modeling layer is used in steps S3-4 to S3-5 to perform bidirectional LSTM encoding on the spatial feature sequence and highlight the key time segments in the RR interval sequence through the attention mechanism. The fully connected layer is used in step S3-6 to output the predicted probability of atrial fibrillation / normal heart rate. The structure and workflow of each component are as follows: Input layer: It is used to preprocess and fuse features of input data. After dynamic cropping, noise reduction (such as Gaussian blurring) and binarization of multi-lead ECG data, the preprocessing signal S_pre is extracted. The RR interval sequence S_rr is obtained using the R wave detection algorithm, and the RR interval sequence S_rr is normalized and interpolated to form time series data of uniform length suitable for convolution operations and time series modeling. Based on the time series data of uniform length, various features can be further extracted, including HRV statistics, TDA features (such as persistence entropy and Betti number curve) and phase space reconstruction features.

[0024] CNN-Pooling feature extraction layer: Convolutional Neural Network (CNN) Construction: A multi-scale filter scheme is employed. Specifically, convolution kernel sizes of multiple scales can be set based on the ECG data sampling rate fs, such as fs rows, 2×fs rows, and 5×fs rows. This allows for capturing ECG signal features at different time scales and frequency domains. For each filter scale, multiple convolution kernels (64 to 128) can be configured in parallel to obtain rich feature representations from different frequency domains and time scales. Through multi-scale convolution processing, the model can obtain a variety of feature maps, thereby improving its perception of complex ECG signals.

[0025] Spatial feature extraction: The preprocessed signal S_pre is used to extract the spatial feature vector F_spa in the convolution and pooling structure of the convolutional neural network (CNN). Among them, a nonlinear activation function can be connected after each layer of convolution operation to introduce nonlinear expression capabilities. The activation function in this embodiment can use PReLU (Parametric Rectified Linear Unit) as an improved version of the ReLU activation function; compared with traditional sigmoid or tanh, PReLU has the advantages of fast convergence, not easy to dissipate gradients, and improved network sparsity; compared with the standard ReLU, PReLU has a learnable slope parameter a in the negative semi-axis direction, so that the network can automatically optimize the activation function during training, making feature extraction more flexible and efficient; Pooling Dimensionality Reduction: After each convolution layer, global or layered pooling can be set to achieve feature dimensionality reduction, reducing the spatial feature vector F_spa to a one-dimensional feature vector, compressing the number of parameters and reducing the risk of overfitting. Through the pooling operation, the most significant eigenvalues ​​can be selected from the convolution result, thereby compressing the amount of data while retaining key information. Multi-layer convolution and pooling can be stacked in series, that is, the pooling result of the previous layer is used as the input of the next convolution layer, and the multi-layer superposition realizes the gradual extraction and integration of low-level features to high-level features. Multimodal feature fusion: After obtaining the output feature map after multi-layer convolution and pooling operations, the feature map is fused with the RR interval sequence features, TDA features, HRV features and phase space reconstruction features to form a high-dimensional feature vector B.

[0026] LSTM-Attention timing modeling layer: Time Series Modeling: The high-dimensional feature vector B is input into the LSTM-Attention layer for time series modeling. The high-dimensional feature vector B is fed into the bidirectional LSTM in the order of heartbeats. Through its memory cells and gating mechanism, the bidirectional LSTM effectively captures the long-term and short-term temporal dependencies of the ECG signal and outputs a latent state sequence F_tem. This is particularly sensitive to non-periodic and complex atrial fibrillation signals.

[0027] Attention enhancement: A complex multi-scale attention mechanism and a feature-adaptive attention mechanism are embedded in the LSTM to weight the features of each time step of the latent state sequence F_tem, highlighting the RR rhythm segments that have a significant impact on atrial fibrillation discrimination, and obtaining an enhanced high-dimensional feature vector C, thereby improving the classification performance and the model's attention to key features.

[0028] The high-dimensional feature vector C is input into a fully connected layer, which includes a multi-layer perceptron architecture. This layer further refines and selects the most discriminative features through nonlinear mapping. The final output layer uses a softmax or sigmoid function to normalize the classification probabilities, presenting the atrial fibrillation prediction results as probability values ​​to downstream applications. By setting an appropriate threshold for the probability values, accurate differentiation between atrial fibrillation (AF) and normal heart rhythm (NSR) can be achieved.

[0029] In this embodiment, building on the local and multi-scale features extracted by CNN, LSTM-Attention is combined to model long-range dependencies in time series data. This allows the model to capture abnormal characteristics of individual heartbeat intervals at the microscale while also capturing the temporal dynamics of the overall ECG signal at the macroscale. Through the synergistic effect of this multi-scale deep convolution and time series modeling, the present invention achieves excellent performance in both single-beat and multiple-beat rhythm feature recognition, providing strong technical support for early and complex atrial fibrillation detection.

[0030] Finally, it should be noted that the above is only a preferred embodiment of the present invention and is not intended to limit the present invention. Although the present invention has been described in detail with reference to the aforementioned embodiments, those skilled in the art can still modify the technical solutions described in the aforementioned embodiments or make equivalent substitutions for some of the technical features therein. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.

Claims

1. A method for predicting atrial fibrillation from electrocardiogram based on multimodal deep learning, characterized in that: The steps include: S1. Collect ECG signal data: obtain raw ECG signal data from the clinical equipment database; S2. Signal preprocessing and feature fusion: S2-1: Preprocess the original ECG signal data and extract the preprocessing signal S_pre; S2-2: Perform R wave peak detection and RR interval sequence calculation on the preprocessed signal S_pre to obtain the RR interval sequence S_rr, perform normalization and interpolation filling on the RR interval sequence S_rr to obtain time series data of uniform length; S2-3: Using a complex multi-scale attention mechanism and a feature-adaptive attention mechanism, the time series data obtained in step S2-2 is weighted and fused to generate multimodal features; The complex multi-scale attention mechanism is used to perform feature weighting and fusion on time series data. The complex multi-scale attention mechanism is used to perform multi-scale weighting on the hidden representation of time series data, capture the feature correlation at different scales, and optimize the expressiveness of features. The calculation formula of the complex multi-scale attention mechanism is as follows: in, represents the attention weight matrix, Indicates the The trainable scaling factor of each scale, represents a value vector, Indicates the scale number, Represents element-wise multiplication; The feature-adaptive attention mechanism is used to further perform dynamic feature weighting and fusion on the time series data processed by the complex multi-scale attention mechanism. Through the feature-adaptive attention mechanism, trainable attention weights are used to dynamically weight features, highlight key features and suppress redundant features. The calculation formula of the feature-adaptive attention mechanism is as follows: where represents the feature vector, represents the trainable attention weight, Represents element-wise multiplication; S3. Model training and atrial fibrillation prediction: Multimodal features are used to train a deep learning model based on a combination of convolutional neural networks and long short-term memory networks. ECG signals are input into the trained model to predict atrial fibrillation and output the classification results.

2. The electrocardiogram atrial fibrillation prediction method based on multimodal deep learning according to claim 1, characterized in that: In step S2-1, preprocessing the original ECG signal data includes the following steps: a. Improve the data quality of the collected raw ECG signal data through image processing technology, perform image cropping on the raw ECG signal data, dynamically adjust the cropping area according to the lead type of the raw ECG signal data, and accurately extract the signal information of each lead; b. Perform Gaussian blur processing and binarization processing on the signal extracted in step a to extract the preprocessed signal S_pre with a clear signal contour.

3. The electrocardiogram atrial fibrillation prediction method based on multimodal deep learning according to claim 2, characterized in that: The step S2-2 specifically includes the following steps: c. Using an improved dynamic threshold algorithm on the preprocessed signal S_pre, calculate the dV(t) / dt change rate and locate the R peak to obtain the timestamp sequence T_R; d. Based on the timestamp sequence T_R, the RR interval sequence S_rr is formed according to the adjacent time difference Δt_k = T_R(k+1)-T_R(k), and normalized and interpolated to output time series data of uniform length for multimodal feature extraction.

4. The electrocardiogram atrial fibrillation prediction method based on multimodal deep learning according to claim 1, characterized in that: In step S2-3, the multimodal features include heart rate variability (HRV) features, topological data analysis (TDA) features, and phase space reconstruction features.

5. The electrocardiogram atrial fibrillation prediction method based on multimodal deep learning according to claim 4, characterized in that: The heart rate variability HRV features include RMSSD, pNN50, SDNN, average heart rate and HRV triangular index; the calculation formulas of the RMSSD, the pNN50 and the SDNN are as follows: in, It is RR intervals, is the average value of the RR interval.

6. The electrocardiogram atrial fibrillation prediction method based on multimodal deep learning according to claim 4, characterized in that: The topological data analysis (TDA) features include persistence entropy, Betti curve and persistence landscape extracted using a persistence homology algorithm.

7. The electrocardiogram atrial fibrillation prediction method based on multimodal deep learning according to claim 4, characterized in that: In step S3, the specific steps of model training and atrial fibrillation prediction are as follows: S3-1. Spatial feature extraction: The preprocessed signal S_pre is converted into a two-dimensional electrocardiogram matrix and input into the convolutional neural network CNN. CNN extracts spatial information such as local waveform morphology and cross-lead correlation through multi-layer convolution-pooling operations to obtain a spatial feature vector, which is expanded and recorded as F_spa. S3-2, Pooling Dimensionality Reduction: Perform global or hierarchical pooling on F_spa to reduce its dimension to one-dimensional spatial features, reducing the amount of computation and the risk of overfitting; after pooling, the feature order still remains one-to-one with the original time series; S3-3. Multimodal feature fusion: The one-dimensional spatial features expanded by pooling are concatenated with the RR interval sequence features, HRV features, TDA features, and phase space reconstruction features calculated in step S2 in the feature channel dimension to form a unified high-dimensional feature vector B. This vector contains morphological, rhythmic, and topological information, providing comprehensive semantics for subsequent time series modeling. S3-4, Timing Modeling: The high-dimensional feature vector B is input into the bidirectional long short-term memory network LSTM according to the order of the heartbeat sequence; LSTM captures the temporal dependency and rhythm changes between ECG events and outputs the hidden state sequence F_tem; S3-5, attention strengthening: A complex multi-scale attention mechanism and feature adaptive attention mechanism are embedded in LSTM, and a weighting strategy is applied to the time segments in the hidden state sequence F_tem to highlight the segments that have a significant impact on the classification results; the weighted time series representation is recorded as a high-dimensional feature vector C; S3-6, Dimensionality reduction mapping and classification: The high-dimensional feature vector C is input into the fully connected dimensionality reduction mapping layer, and then the Softmax activation function is used to output the probability ŷ∈[0, 1] of atrial fibrillation AF and normal heart rate NSR; when ŷ ≥ threshold, it is judged as atrial fibrillation, otherwise it is judged as normal rhythm.

Images