Heart disease prediction method fusing time sequence decomposition and multi-head attention mechanism
Through the heart disease prediction method that integrates timing decomposition and multi-head attention mechanism, the problem of inefficient treatment of the existing central heart disease prediction method is solved, and more efficient heart disease prediction is achieved, and accuracy and model adaptability are improved.
Patent Information
- Application Number
- CN202510325623.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-03-19
- Publication Date
- 2025-07-25
AI Technical Summary
Existing heart disease prediction methods are difficult to efficiently capture dynamic and long-term dependencies in medical data such as electrocardiogram, resulting in inefficient data analysis and processing.
A heart disease prediction method that combines time-series decomposition and multi-head attention mechanism is adopted. By pre-processing the heart disease data set, time-series decomposition, trend and periodic components are extracted, and combined with convolutional operations and multi-head self-attention mechanism, the TD-CS-Transformer algorithm is constructed for prediction.
It significantly improves the accuracy of heart disease prediction, enhances the sensitivity to key features and the robustness of abnormal detection, reduces data complexity, and improves the adaptability and generalization capabilities of the model.
Smart Images
Figure CN120376124A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of medical data analysis, and specifically, to a heart disease prediction method that integrates time series decomposition and multi-head attention mechanism. Background Art
[0002] Heart disease is one of the main diseases causing death and disability globally. The continuous increase in its incidence and mortality rate has made it a focus of attention in the field of public health. Effectively predicting the risk of heart disease is of great significance for the early intervention and treatment of patients.
[0003] Medical data contains rich dynamic and static information, such as electrocardiogram (ECG) signals, heart rate variability, blood pressure, and static characteristics such as age and gender. A large number of time-dependent patterns and potential features are contained in these data. However, due to their complexity and diversity, existing statistical methods and single deep learning models often have difficulty efficiently capturing the key dynamic and long-term dependence relationships in the data, resulting in low efficiency in data analysis and processing. Summary of the Invention
[0004] The purpose of the present invention is to overcome the problems of insufficient data processing ability and low efficiency in the existing heart disease prediction methods, and to provide a heart disease prediction method that integrates time series decomposition and multi-head attention mechanism. This heart disease prediction method that integrates time series decomposition and multi-head attention mechanism greatly improves the accuracy of heart disease prediction.
[0005] To achieve the above purpose, the present invention provides a heart disease prediction method that integrates time series decomposition and multi-head attention mechanism, and the method includes:
[0006] Preprocess the open-source heart disease dataset to obtain the training set D of the algorithm, and the total number of patients in the dataset is N;
[0007] Divide the dataset into a training set, a validation set, and a test set for model training, parameter tuning, and performance evaluation;
[0008] Extract the trend part and periodic components from the patient data through time series decomposition of the patient data, and calculate the residuals;
[0009] Combine the convolutional operation and the multi-head self-attention mechanism to form the TD-CS-Transformer algorithm;
[0010] Predict the heart disease risk according to the TD-CS-Transformer algorithm.
[0011] Preferably, the preprocessing of the open-source heart disease dataset includes:
[0012] Perform personal privacy-related processing on the open-source heart disease dataset;
[0013] Delete the data of neonatal and pediatric patients in the dataset;
[0014] Divide the patient data into dynamic time series and static features;
[0015] Adopt the multiple interpolation method to handle missing values, and use the weighted Euclidean distance to interpolate and fill the dynamic time series according to formula (1).
[0016]
[0017] where d i is the Euclidean distance from the missing value point to the known data point i, x and y are the position coordinates of the missing value point, x i is the time index, y i is the corresponding signal value;
[0018] Normalize the dataset according to formula (2), and uniformly normalize the dynamic and static data to [0, 1].
[0019]
[0020] where ∈ is a constant to prevent the denominator from being zero, X t is the normalized data point, between [0, 1], T is the time series, and x i is the value of the original data point.
[0021] Preferably, the dynamic time series includes electrocardiogram (ECG) signals, heart rate, and blood pressure, and the static features include age, gender, BMI, and medical history.
[0022] Preferably, the time series decomposition includes:
[0023] According to formula (3), use moving average filtering to calculate the trend component and smooth the sequence data.
[0024]
[0025] where d is the window, adjusted according to the sampling frequency of the electrocardiogram (ECG) signal and the clinical characteristics of heart disease; X t is the value of the ECG signal at the t-th time step, q is half the size of the sliding window, and the size of the sliding window is adjusted according to the periodic change of the ECG signal;
[0026] Extract the periodic features in the ECG signal according to formula (4) and formula (5).
[0027]
[0028] D t = X t - T t , (5)
[0029] where D t is the residual part after removing the trend, is the average value within the period, and S i is the seasonal component of the i-th period;
[0030] According to formula (6), the residual part of the ECG signal is used as an important feature for predicting heart disease abnormalities,
[0031] X s = T t + S t + N t , (6)
[0032] where N t is the residual part, reflecting short-term fluctuations and abnormalities.
[0033] Preferably, the TD-CS-Transformer algorithm that combines convolutional operation and multi-head self-attention mechanism includes:
[0034] Extract local features according to formula (7),
[0035]
[0036] where k is the convolutional kernel size, which is optimized according to the waveform and frequency characteristics of the ECG signal, and ω i is the weight;
[0037] For long time series, calculate the attention weights of key points according to formula (8),
[0038]
[0039] According to formula (9), adopt the multi-head attention mechanism, and each head learns the attention weights in different feature spaces,
[0040] head i = Attention(QWi i Q , KW i K , VW i V ), (9)
[0041] where Q, K, and V are the query, key, and value matrices respectively, and W i Q , W i K , Wi V Represents the weight matrix for each head;
[0042] After concatenating the outputs of all heads according to formula (10), project through W O onto the final feature space to obtain the final projection matrix,
[0043] MultiHead(q, K, V) = Concat(head1,..., head h )W O , (10).
[0044] Preferably, the convolutional kernel size k is optimized according to the waveform and frequency characteristics of the ECG signal, including:
[0045] For rapidly changing ECG signals, the convolutional kernel k is reduced;
[0046] For slower ECG signal fluctuations, the convolutional kernel k is increased.
[0047] Preferably, predicting the risk of heart disease according to the TD-CS-Transformer algorithm includes:
[0048] According to formula (11), use the Softmax activation function to output the class probability and predict the occurrence of heart disease,
[0049] P(y = 1|X) = Softmax(W T h + b), (11)
[0050] According to formula (12), use the linear activation function to predict heart disease-related indicators,
[0051]
[0052] According to formula (13), use the cross-entropy method to calculate the prediction loss,
[0053]
[0054] According to formula (14), use the mean squared error as the loss function,
[0055]
[0056] According to the above technical solution, first, preprocess the open-source heart disease dataset to obtain the training set D of the algorithm; then divide the dataset into a training set, a validation set, and a test set for model training, parameter tuning, and performance evaluation; next, perform time series decomposition on the patient data, extract the trend part and periodic components from the patient data, and calculate the residuals; then, combine the convolution operation and the multi-head self-attention mechanism to form the TD-CS-Transformer algorithm; finally, predict the heart disease risk according to the TD-CS-Transformer algorithm. In this way, the processed data simplifies the signal structure through time series decomposition, combines the convolution operation with the multi-head attention mechanism to process complex medical time series data, and overcomes the defects of the existing heart disease prediction in terms of time information. BRIEF DESCRIPTION OF THE DRAWINGS
[0057] Figure 1 is a flowchart of a heart disease prediction method that integrates time series decomposition and multi-head attention mechanism according to the present invention;
[0058] Figure 2 is an architecture diagram of a heart disease prediction method that integrates time series decomposition and multi-head attention mechanism according to the present invention;
[0059] Figure 3 is a schematic structural diagram of a fusion module in a heart disease prediction method that integrates time series decomposition and multi-head attention mechanism according to the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS
[0060] The following will describe in detail the specific embodiments of the present invention with reference to the accompanying drawings. It should be understood that the specific embodiments described herein are only for the purpose of illustrating and explaining the present invention, and are not intended to limit the present invention.
[0061] See Figures 1 to 3 , the present invention provides a heart disease prediction method that integrates time series decomposition and multi-head attention mechanism, and the method includes:
[0062] S1. Preprocess the open-source heart disease dataset to obtain the training set D of the algorithm, and the total number of patients in the dataset is N;
[0063] S2. Divide the dataset into a training set, a validation set, and a test set for model training, parameter tuning, and performance evaluation;
[0064] S3. Perform time series decomposition on the patient data, extract the trend part and periodic components from the patient data, and calculate the residuals;
[0065] S4. Combine the convolution operation and the multi-head self-attention mechanism to form the TD-CS-Transformer algorithm;
[0066] S5. Predict the risk of heart disease according to the TD-CS-Transformer algorithm.
[0067] Through the above technical solution, the processed data simplifies the signal structure through time series decomposition, combines the convolution operation with the multi-head attention mechanism, processes complex medical time series data, and overcomes the defects of the existing heart disease prediction time information.
[0068] Specifically, the above step S1 includes:
[0069] S1.1. Perform personal privacy-related processing on the open-source heart disease dataset. In one implementation, the patient's name can be hidden, and only a number is used to represent the patient's identity;
[0070] S1.2. Since the physiology of pediatric intensive care patients is significantly different from that of adults, it is necessary to delete the data of neonatal and pediatric patients in the dataset, specifically neonates and pediatric patients under 18 years old hospitalized in the intensive care unit;
[0071] S1.3. Divide the patient data into dynamic time series and static features. Among them, the dynamic time series includes electrocardiogram (ECG) signals, heart rate, blood pressure, etc., and the static features include age, gender, BMI, medical history, etc.
[0072] S1.4. Use the multiple interpolation method to process missing values, and fill in the interpolation of the dynamic time series using the weighted Euclidean distance according to formula (1).
[0073]
[0074] where d i is the Euclidean distance from the missing value point to the known data point i, x and y are the position coordinates of the missing value point, x i is the time index, and y i is the corresponding signal value;
[0075] S1.5. Normalize the dataset according to formula (2), and unify the dynamic and static data to be normalized to [0, 1].
[0076]
[0077] where ∈ is a constant to prevent the denominator from being zero, X t is the normalized data point, between [0, 1], T is the time series, and x i is the value of the original data point (such as heart rate, blood pressure, or ECG signal).
[0078] The above step S3 includes:
[0079] S3.1. Calculate the trend component using moving average filtering according to formula (3) to smooth the sequence data,
[0080]
[0081] where d is the window, adjusted according to the sampling frequency of the electrocardiogram (ECG) signal and the clinical characteristics of heart diseases; X t is the value of the ECG signal at the t-th time step, q is half the size of the sliding window and the size of the sliding window is adjusted according to the periodic changes of the ECG signal;
[0082] S3.2. Extract the periodic features in the ECG signal according to formula (4) and formula (5), for example, diurnal variations, heart rate fluctuations after exercise:
[0083]
[0084] D t = X t - T t , (5)
[0085] where D t is the residual part after removing the trend, is the average value within the period, S i is the seasonal component of the i-th period, and this seasonal component can be associated with the periodic changes of heart attacks.
[0086] S3.3. Take the residual part of the ECG signal as an important feature for predicting heart disease abnormalities according to formula (6),
[0087] X s = T t + S t + N t , (6)
[0088] where N t is the residual part, reflecting short-term fluctuations and abnormalities, such as arrhythmia, cardiac arrest, etc.
[0089] The above step S4 includes:
[0090] S4.1. Extract local features according to formula (7),
[0091]
[0092] where k is the size of the convolution kernel, optimized according to the waveform and frequency characteristics of the ECG signal, ω i is the weight; among them, for the rapidly changing ECG signal, the size of the convolution kernel should be smaller; for the slower fluctuations, the convolution kernel can be slightly increased.
[0093] S4.2. For long time series, calculate the attention weights of key points according to formula (8).
[0094]
[0095] S4.3. According to formula (9), adopt the multi-head attention mechanism, and each head learns the attention weights of different feature spaces.
[0096] head i = Attention(QWi i Q , KW i K , VW i V ), (9)
[0097] where Q, K, and V are the query, key, and value matrices respectively, and W i Q , W i K , W i V represent the weight matrices of each head;
[0098] S4.4. After concatenating the outputs of all heads according to formula (10), project them into the final feature space through W O to obtain the final projection matrix.
[0099] MultiHead(q, K, V) = Concat(head1,..., head h )W O , (10).
[0100] The above step S5 includes:
[0101] S5.1. According to formula (11), use the Softmax activation function to output the class probability and predict the occurrence of heart disease.
[0102] P(y = 1|X) = Softmax(W T h + b), (11)
[0103] S5.2. According to formula (12), use the linear activation function to predict heart disease-related indicators.
[0104]
[0105] S5.3. According to formula (13), adopt the cross-entropy method to calculate the prediction loss.
[0106]
[0107] S5.4. According to formula (14), the mean square error is used as the loss function.
[0108]
[0109] In addition, in a specific implementation, the above model can also be subjected to 10-fold cross-validation, and indicators such as accuracy, precision, recall, F1-Score, and AUC value are calculated through the confusion matrix to comprehensively evaluate the model performance. The confusion matrix is as follows:
[0110]
[0111] The formula for calculating accuracy is:
[0112] The formula for calculating precision is:
[0113] The formula for calculating recall is:
[0114] The formula for calculating F1-Score is:
[0115] In summary, time series decomposition technology is a powerful data preprocessing method that can decompose complex dynamic signals into trend, periodic, and residual components. This method can not only enhance the structural characteristics of the data but also significantly reduce the noise of the sequence data, providing clearer features for subsequent modeling. Time series data in heart disease prediction, such as ECG signals and heart rate fluctuations, usually contain significant short-term fluctuations and long-term trends. Using time series decomposition can better extract potential health signals. At the same time, the periodic features (such as heartbeat rhythm) and abnormal fluctuations (such as arrhythmia) in ECG signals are often important prediction indicators for heart disease, while the residual part may reveal potential risks of emergencies or abnormal changes. Through time series decomposition, not only can the sensitivity of the model to key features be improved, but also its robustness to anomaly detection can be enhanced.
[0116] To further improve the model's ability to process complex medical data, the multi-head convolutional sparse self-attention mechanism is introduced into the field of heart disease prediction. This method combines the advantages of convolutional neural networks (CNNs) and self-attention mechanisms. Among them, convolutional operations are good at extracting local features in time series, such as QRS waveforms and R-wave peaks in electrocardiograms, while self-attention mechanisms can effectively capture long-term dependencies and global correlations between different features. Sparse attention significantly reduces the computational complexity by only focusing on key data points, making the model more efficient in processing long sequence data. The parallel computing characteristics of the multi-head attention mechanism enable it to simultaneously focus on multiple feature patterns, such as local anomalies and long-term trends in dynamic signals, as well as the interaction between dynamic and static features, thus comprehensively improving the model's prediction ability and accuracy. This method provides a new technical path for heart disease prediction, which can not only achieve more efficient data analysis but also provide strong technical support for precision medicine and personalized treatment.
[0117] In the present invention, the heart disease prediction method that combines time series decomposition and multi-head attention mechanism optimizes the feature extraction of each component through time series decomposition, fully mines the potential patterns in the time series, reduces the complexity of the data, and improves the prediction accuracy. In addition, the multi-head attention mechanism is introduced to simultaneously learn local short-term anomalies and global long-term trends. By combining convolutional operations with the multi-head self-attention mechanism, the model is allowed to parallelly capture multi-level information from different inputs (such as dynamic time series and static features), improving the model's adaptability and generalization ability, and also having strong robustness to outliers and noise. In addition, the present invention also combines convolutional neural networks and self-attention mechanisms, which can fully mine the complex correlations between various features.
[0118] The preferred embodiments of the present invention have been described in detail above in conjunction with the accompanying drawings. However, the present invention is not limited thereto. Within the scope of the technical concept of the present invention, various simple modifications can be made to the technical solutions of the present invention, including any suitable combination of each specific technical feature. To avoid unnecessary repetition, the present invention does not separately describe various possible combination methods. But these simple modifications and combinations should also be regarded as the content disclosed by the present invention and fall within the protection scope of the present invention.
Claims
1. A heart disease prediction method integrating time series decomposition and multi-head attention mechanism, characterized in that, The method includes: Preprocessing the open-source heart disease dataset to obtain the training set D of the algorithm, and the total number of patients in the dataset is N; Dividing the dataset into a training set, a validation set, and a test set for model training, parameter tuning, and performance evaluation; Through time series decomposition of patient data, extracting the trend part and periodic components from the patient data, and calculating the residuals; Combining convolutional operations and the multi-head self-attention mechanism to form the TD-CS-Transformer algorithm; Predicting the risk of heart disease according to the TD-CS-Transformer algorithm.
2. The heart disease prediction method integrating temporal sequence decomposition and multi-head attention mechanism according to claim 1, characterized in that The preprocessing of the open-source heart disease dataset described above includes: Performing processing related to personal privacy on the open-source heart disease dataset; Deleting the data of neonatal and pediatric patients in the dataset; Dividing the patient data into dynamic time series and static features; Using the multiple interpolation method to process missing values, and filling in the interpolation of the dynamic time series using the weighted Euclidean distance according to formula (1); Among them, d i is the Euclidean distance from the missing value point to the known data point i, x and y are the position coordinates of the missing value point, x i is the time index, and y i is the corresponding signal value; Normalizing the dataset according to formula (2), and uniformly normalizing the dynamic and static data to [0, 1]; where ∈ is a constant to prevent the denominator from being zero, and X t is the normalized data point, within [0, 1], T is the time series, and x i is the value of the original data point.
3. The heart disease prediction method integrating temporal sequence decomposition and multi-head attention mechanism according to claim 2, wherein The dynamic time series includes electrocardiogram (ECG) signals, heart rate, and blood pressure, and the static features include age, gender, BMI, and medical history.
4. The heart disease prediction method integrating temporal decomposition and multi-head attention mechanism according to claim 1, characterized in that The time series decomposition described above includes: According to formula (3), using moving average filtering to calculate the trend component and smooth the sequence data; where d is a window, which is adjusted according to the sampling frequency of the electrocardiogram (ECG) signal and the clinical characteristics of heart diseases; X t is the value of the ECG signal at the t-th time step, q is half the size of the sliding window, and the size of the sliding window is adjusted according to the periodic variation of the ECG signal; Extracting the periodic features in the ECG signal according to formula (4) and formula (5); Among them, D t is the residual part after removing the trend, is the average value within the period, and S i is the seasonal component of the i-th period; Taking the residual part of the ECG signal as an important feature for predicting heart disease abnormalities according to formula (6); X s = T t + S t + N t , (6) where N t is the residual part, reflecting short-term fluctuations and anomalies.
5. The heart disease prediction method integrating temporal sequence decomposition and multi-head attention mechanism according to claim 1, characterized in that The combination of convolutional operations and the multi-head self-attention mechanism to form the TD-CS-Transformer algorithm includes: Extracting local features according to formula (7); where k is the convolution kernel size, which is optimized according to the waveform and frequency characteristics of the ECG signal, and ω i is the weight; For long time series, calculating the attention weights of key points according to formula (8); According to formula (9), adopting the multi-head attention mechanism, and each head learns the attention weights in different feature spaces; head i = Attention(QW i Q ,KW i K ,VW i V ), (9) Among them, Q, K, and V are the query, key, and value matrices respectively, and W i Q , W i K , w i V represent the weight matrix for each head; After concatenating the outputs of all heads according to formula (10), project them through W O onto the final feature space to obtain the final projection matrix MultiHead(Q,K,V)=Concat(head1,...,head h )W O , (10).
6. The heart disease prediction method integrating temporal decomposition and multi-head attention mechanism according to claim 5, characterized in that The optimization of the convolutional kernel size k according to the waveform and frequency characteristics of the ECG signal includes: For rapidly changing ECG signals, the convolutional kernel k is reduced; For slower ECG signal fluctuations, the convolutional kernel k is increased.
7. The heart disease prediction method integrating temporal decomposition and multi-head attention mechanism according to claim 1, characterized in that The prediction of the risk of heart disease according to the TD-CS-Transformer algorithm includes: According to formula (11), using the Softmax activation function to output the class probabilities and predict the occurrence of heart disease; P(y = 1|X) = Softmax(W T h + b), (11) According to formula (12), using the linear activation function to predict heart disease-related indicators; According to formula (13), adopting the cross-entropy method to calculate the prediction loss; According to formula (14), using the mean squared error as the loss function.