A 12-lead ECG reconstruction method based on attention-based generative adversarial networks

Abstract

This invention discloses a 12-lead electrocardiogram (ECG) reconstruction method based on a generative adversarial network (GAN) with an attention mechanism. The method preprocesses 12-lead ECGs of various diseases through filtering, time segmentation, and data augmentation to obtain ECG segments. The I-lead signal is extracted, copied, and zero-filled before being input into the network. Using the corresponding 12-lead ECG as the ground truth label, the network is trained to learn the correspondence between the two, achieving a game balance between the generator and the discriminator, thus realizing the reconstruction from a single-lead ECG to a 12-lead ECG. The generative adversarial network (GAN) model used in this invention incorporates an attention mechanism. Applying this mechanism to the generator allows for better focus on salient regions, highlighting the salient features of specific local areas, and improving the model's sensitivity and prediction accuracy.

Description

Technical Field

[0001] This invention belongs to the field of electrocardiogram signal reconstruction technology, specifically relating to a 12-lead electrocardiogram reconstruction method based on attention mechanism generative adversarial networks. Background Technology

[0002] An electrocardiogram (ECG) is a common and non-invasive method used in clinical practice to diagnose cardiovascular diseases. It is a voltage graph that changes over time, reflecting the electrical activity of myocardial depolarization and repolarization during each heartbeat. An ECG at a normal rhythm consists of a series of waveforms, including the P wave, QRS complex, and T wave, and also includes the PR, ST, and QT intervals. ECG recordings reflect the electrical activity of the heart and can be used to detect and diagnose common heart diseases such as atrial fibrillation (AF) or myocardial infarction (MI). Currently, the most common clinical ECG examination is the 12-lead ECG, which obtains ECG data by placing electrodes in the limbs and chest to analyze cardiovascular disease conditions. A 12-lead ECG includes three standard limb leads (I, II, III), three enhanced limb leads (aVR, aVL, aVF), and six precordial leads (V1, V2, V3, V4, V5, V6).

[0003] However, the 12-lead ECG measurement method requires placing multiple electrodes on the patient's limbs and chest, making real-time ECG monitoring difficult in daily life. With technological advancements, wearable devices have provided a portable and real-time method for measuring ECG signals. For example, smartwatches detect potential changes caused by cardiac activity between the left and right arms to obtain ECG waveforms. However, this only records potential changes in one direction, yielding only single-lead (lead I) data, which is significantly less information than the standard 12-lead ECG used clinically. This limits its diagnostic value for cardiac diseases. Therefore, reconstructing ECG signals from single-lead to multi-lead models has significant clinical implications.

[0004] Currently, the literature [Atoui, Hussein, Jocelyne Fayn, and Paul Rubel. "A novel neural-network model for deriving standard 12-lead ECGs from serial three-lead ECGs: application to self-care." IEEE transactions on information technology in biomedicine 14.3(2010):883-890] uses a general and specific nonlinear reconstruction method based on artificial neural networks (ANNs) to recover 12-lead ECGs from orthogonal three leads (I, II, V2). However, acquiring three-lead ECG signals requires specialized medical equipment to set electrode patches, which presents certain operational difficulties for daily monitoring compared to acquiring single-lead signals.

[0005] Generative models are a type of model in the field of deep learning. They learn the distribution patterns of training data to generate new data that are similar to but not exactly the same as real data. Generative Adversarial Networks (GANs) consist of two networks: a generator and a discriminator. The generator is responsible for generating fake data, while the discriminator is responsible for distinguishing between generated data and real data. During the training iteration process, the two networks continuously compete against each other until they reach an equilibrium state. The emergence of Generative Adversarial Networks (GANs) has met the research needs of many fields and has been widely applied in image vision, speech processing, and other areas. For example, the literature [He, Yan, et al. "Efficient learning of healthcare data from IoT devices by edgeconvolution neural networks." Applied Sciences 10.24(2020):8934] uses a deep convolutional generative adversarial network (DCGAN) to generate additional ECG data from the MIT-BIH arrhythmia database, improving the accuracy of remote health monitoring and edge computing. However, the generated data is only used for data augmentation to train the prediction model and is not combined with commonly used wearable devices that measure single-lead ECGs, thus failing to realize practical application scenarios. Summary of the Invention

[0006] In view of the above, the present invention provides a 12-lead electrocardiogram reconstruction method based on attention mechanism generative adversarial networks, which can realize the reconstruction of electrocardiogram signals from single lead to 12 lead, overcoming the limitations of modern electrocardiogram measurement methods.

[0007] A method for reconstructing a 12-lead electrocardiogram based on a generative adversarial network with an attention mechanism includes the following steps:

[0008] (1) Obtain 12-lead electrocardiogram (ECG) signals from patients including those with normal heart rate, myocardial infarction, and various heart diseases, and divide these ECG signals into training set and test set after preprocessing;

[0009] (2) Construct a 12-lead electrocardiogram reconstruction model based on generative adversarial network (GAN), which includes a discriminator D and two generators G. I and G L The system consists of inputting single-lead ECG signals into G... I and G L In the middle, G I Used to reconstruct and generate the corresponding 12-lead ECG signal, G L This is equivalent to an autoencoder, which encodes the latent space vector z. l As G I The encoder outputs the latent space vector z i The tag makes G I The decoder is better able to learn the important characteristics of single-lead ECG signals; the discriminator D is used to analyze G... I The generated 12-lead ECG signal and the original 12-lead ECG signal are compared to determine whether they are genuine or fake.

[0010] (3) The above model is trained using the training set, that is, the 12-lead ECG signal of the training set is used as a label and single-lead ECG signal is extracted from it as the model input.

[0011] (4) Extract single-lead ECG signals from the 12-lead ECG signals in the test set and input them into the generator G of the trained model. I In this way, the corresponding 12-lead ECG signal can be reconstructed and output.

[0012] Furthermore, in step (1), the obtained 12-lead ECG signal is first filtered to eliminate low-frequency drift and interference in the ECG signal, and then the ECG signal is divided into multiple signal segments of fixed time length. Several additional segments are generated for each group of ECG signals for data augmentation. The size of each signal segment is m×y, where m is the time length of the segment and y = 12 is the number of leads.

[0013] Furthermore, the generator G IThe encoder consists of five cascaded encoding modules, each of which comprises a convolutional layer, a batch normalization layer, and a Leaky ReLU activation function layer connected in sequence; G I The decoder is the inverse structure of the encoder, consisting of five cascaded decoding modules. Except for the last decoding module, the remaining decoding modules are composed of attention mechanism layers, deconvolution layers, batch normalization layers, and ReLU activation function layers connected in sequence. Starting from the second decoding module, the output of the previous decoding module is concatenated with the output of the corresponding encoding module and used as the input of the current decoding module. The last decoding module contains only deconvolution layers.

[0014] Furthermore, the attention mechanism layer modifies the input with the linear transformation matrix W respectively. Q W K and W V Multiplying them yields the query vector matrix Q, the key vector matrix K, and the value vector matrix V. The output result is then calculated using the following formula.

[0015]

[0016] Where: Attention(Q,K,V) is the output of the attention mechanism layer, and d is the number of columns in the query vector matrix Q. T This indicates transpose, and softmax() represents the normalized exponential function.

[0017] Furthermore, the generator G L The structure is mostly the same as that of the generator G I The structures are similar, the difference lies in G. L The decoding module does not contain an attention mechanism layer and is not concatenated with the output of the encoding module as input.

[0018] Furthermore, the discriminator employs a 1D U-Net network, the structure of which is similar to that of the generator G. L They have the same structure.

[0019] Further, in step (3), the ECG signal of any lead is extracted from the 12-lead ECG signal of the training set, copied 12 times, and two extra rows are added above and below for zero padding (a total of 16 rows). Then, the data is combined in batches and input into the model for training. The data dimension of the input model is b×m×n×c, where b is the number of signal segments in a batch, c is the number of channels, and n=16.

[0020] Furthermore, the objective function expression used in step (3) during model training is as follows:

[0021]

[0022] in: This represents the finally trained generator G. I , To generate the loss function for a Generative Adversarial Network (GAN), For generator G I loss function, For generator G L The loss function (which is only used to train the generator G) L (Not included in the overall optimization objective function) Let be the loss function for the latent space vector, and λ and α be hyperparameters. Let ||||1 denote the expectation function, and e denote the L1 norm. i For the single-lead ECG signal of the input model, e o For a true 12-lead ECG signal, D(e) o ) indicates that the discriminator is for e o The discrimination result, G I (e i ) represents the generator G I For e i The reconstructed 12-lead ECG signal, D(G) I (e i )) indicates that the discriminator is for G I (e i The discrimination result of G) L (e i ) represents the generator G L For e i The reconstructed single-lead electrocardiogram signal.

[0023] A computer device includes a memory and a processor, wherein the memory stores a computer program and the processor executes the computer program to implement the above-described 12-lead electrocardiogram reconstruction method.

[0024] A computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described 12-lead electrocardiogram reconstruction method.

[0025] This invention preprocesses 12-lead electrocardiograms of various diseases by filtering, time segmentation, and data augmentation to obtain electrocardiogram segments. The I-lead signal is extracted, copied, and zero-filled and input into the network. The corresponding 12-lead electrocardiogram is used as the ground truth label to train the network to learn the correspondence between the two, achieving a game balance between the generator and the discriminator, and realizing the recovery from single-lead electrocardiograms to 12-lead electrocardiograms.

[0026] This invention incorporates an attention mechanism into the Generative Adversarial Network (GAN) model. Applying this mechanism to the generator effectively focuses on salient regions, highlighting the significant features of specific local areas and improving the model's sensitivity and prediction accuracy. Experiments using electrocardiograms from different diseases demonstrated the superiority of this invention across various evaluation metrics. Compared to other deep learning methods, this invention achieves closer accuracy and more precise reconstruction results in the experiments. Attached Figure Description

[0027] Figure 1 This is a schematic diagram of the 12-lead electrocardiogram reconstruction model based on generative adversarial networks of the present invention.

[0028] Figure 2 This is a schematic diagram of the attention mechanism.

[0029] Figure 3 This is a schematic diagram of the dataset preprocessing process according to an embodiment of the present invention.

[0030] Figure 4 This is a pie chart of various diseases in the training dataset of this invention.

[0031] Figure 5 The results of electrocardiogram reconstruction for patients with myocardial infarction are presented using various networks.

[0032] Figure 6 The results of electrocardiogram reconstruction for patients with ventricular hypertrophy using various networks.

[0033] Figure 7 The results of electrocardiogram reconstruction for people with normal heart rates using various networks.

[0034] Figure 8 The results of electrocardiogram reconstruction for patients with myocardial infarction are presented using various networks. Detailed Implementation

[0035] To describe the present invention in more detail, the technical solution of the present invention will be described in detail below with reference to the accompanying drawings and specific embodiments.

[0036] This invention relates to a 12-lead electrocardiogram reconstruction method based on attention-based generative adversarial networks, comprising the following steps:

[0037] (1) Prepare data.

[0038] The publicly available ECG dataset PTB-XL was selected, which includes 12-lead ECGs from nearly 20,000 patients, covering normal heart rate, myocardial infarction, and ST-segment T-wave changes. These 10-second ECG data were first filtered to eliminate low-frequency drift and network interference, and then divided into training and testing sets proportionally. Subsequently, the signals in both the training and testing subsets were further divided into 1.28-second segments. Finally, 10 additional segments were generated for each signal in the training subset for data augmentation, resulting in a training subset of size s×m×y, where s = 214624 represents a signal segment, m = 128 represents the segment length, and y = 12 represents the lead number. The testing subset contained 22095 segments.

[0039] (2) Construct a 12-lead electrocardiogram reconstruction model based on generative adversarial model (GAN), such as Figure 1 As shown.

[0040] single-lead signal e i Before inputting the network model, the data must first be extracted from the training set and copied and zero-padding is performed. The data dimension of the input network is b×m×n×c, where b is the batch number, m=128 is the signal length, n=16 is the result after copying and zero-padding, and c=1 is the number of channels. The vector matrix is ​​input into the encoder-decoder structure of the generator G. The output matrix of the encoder-decoder structure of the generator G has the same dimension as the input matrix. Then, the 12 leads generated by the generator G and the original 12 leads are input to the discriminator D for discrimination.

[0041] The network model consists of a generator G and a discriminator D, where the generator G includes a 12-lead generator G. I and tag generator G L single-lead signal e i The inputs are fed into two generators, generator G. I The corresponding 12-lead signal is generated, and the tag generator G... L It is equivalent to an autoencoder, whose encoder obtains the latent space vector z l As G I The latent space vector z obtained by the encoder i The tag makes G I The decoder is better able to learn the important characteristics of the input single lead. The discriminator D for G... I The generated 12-lead ECG and the original 12-lead ECG are compared and distinguished. This process is repeated to train the generator G and the discriminator D.

[0042] 12-lead generator G in the network model IThe encoder consists of five coding blocks, each of which includes convolution, batch normalization, and Leaky ReLU layers. The decoder is the inverse structure of the encoder, consisting of five decoding blocks, each of which includes attention, a concatenation layer, deconvolution, batch normalization, and ReLU. The first decoding block does not contain a concatenation layer, and the last decoding block contains only a concatenation layer and deconvolution. Starting from the second decoding block, the concatenation layer concatenates the output of the previous decoding block with the output of the corresponding coding block and uses it as the input of the next decoding block.

[0043] 12-lead generator G in the network model I The key to the attention mechanism in the decoder is learning a weight distribution and then applying it to the features; for example... Figure 2 As shown, it first calculates the similarity of the query for each key. From a signal processing perspective, the inner product represents the similarity between two signals. The distance between the query and keys is calculated using the inner product to obtain the weight coefficients.

[0044]

[0045] Where d is the number of columns in matrices Q and K.

[0046] Then, the weights are normalized to obtain directly usable weights; the softmax operation can organize the original calculated scores into a probability distribution where the sum of the weights of all elements is 1, and the calculation is performed on each row of the matrix:

[0047]

[0048] Finally, by summing the values ​​based on their weights, the attention value for the query can be calculated.

[0049]

[0050] Tag Generator G L The structure is mostly related to the 12-lead generator G. I Similar to, but different in that the decoding block does not contain attention or concatenation layers. The discriminator D is a 1D U-Net, and the label generator G... L Having the same layer structure, the 12-lead generator G I The output and the true value of the 12-lead ECG are used as inputs for discrimination.

[0051] (3) The single-lead data e in the training set i Enter network, e oUsing real 12-lead data, i.e., the network's labels, the network model is trained to generate 12-lead ECGs from single-lead ECGs. The training process is as follows:

[0052] 3.1 Set the learning rate, batch size, and maximum number of iterations to initialize the network parameters;

[0053] 3.2 The single-lead data e in the training set i Input network, calculate generator G I The loss between the generated 12 leads and the label truth value, and the generator G I Latent space variable z i With the truth value of the label, i.e., the generator G L Latent space variable z l Loss, Generator G I Adversarial loss function against discriminator D:

[0054]

[0055] Where: e i For the input single-lead data, e o This is real 12-lead data. In addition to the losses inherent in the GAN network itself, a 12-lead generator G was also introduced. I loss and the loss of the latent space vector in Used only for training the label generator G L It is not included in the final objective function; the hyperparameters λ and α are set to 50 and 1, respectively.

[0056] 3.3 The network parameters are updated using gradient descent. Training is completed when the optimization objective converges or the maximum number of training iterations is reached.

[0057] (4) The single-lead data of the test set e i The generated 12-lead electrocardiogram is obtained by inputting the trained generator. In this embodiment, we select the electrocardiogram signal of standard limb lead I for reconstruction.

[0058] The effectiveness of this invention is verified through simulation experiments. The experiment uses the publicly available dataset PTB-XL, and the data preprocessing process is as follows: Figure 3 As shown, the training and test sets are divided in an 8:2 ratio. The training dataset is structured as follows: Figure 4 As shown, each data set has a duration of 1.28 seconds, and the single-lead data e from the training set is used. i Enter network, e oThe network was trained using real 12-lead data as labels. During training, the learning rate was set to 0.0001, the batch size was set to 32, and the maximum number of iterations was set to 10. The network of this invention was then compared with CycleGAN.

[0059] Table 1 shows the quantitative analysis results of the 12-lead dataset. Root mean square error (RMSE) is a commonly used measure of the difference between measured values. The specific formula is as follows:

[0060]

[0061] Mean Absolute Error (MAE) describes the error between the predicted value and the actual value, measuring the absolute deviation between the predicted result and the actual result. The specific formula is as follows:

[0062]

[0063] Where: s represents the electrocardiogram segment, and M is the sample length.

[0064] The table shows the results of the other 11 leads generated from top to bottom, and from left to right, the results of CycleGAN, EKGAN, and EKGAN with attention mechanism. The table shows that the average evaluation index of the present invention is higher than that of the other two methods, which proves the superiority of the present invention.

[0065] Table 1

[0066]

[0067]

[0068] Figure 5 The images show the ECG reconstruction results of various networks for patients with myocardial infarction. The black, red, and blue waveforms correspond to the ground truth image, the ECG reconstruction images of EKGAN (with attention mechanism removed), and CycleGAN, respectively. The images show the reconstruction results of limb leads (II, III, aVR, aVL, aVF) and precordial leads (V1, V2, V3, V4, V5, V6), respectively. It can be seen that the reconstruction results of CycleGAN have a larger error, while the reconstruction results of EKGAN are closer to the ground truth image.

[0069] Figure 6The image shows the ECG reconstruction results of various networks for patients with ventricular hypertrophy. The black, red, and blue waveforms correspond to the ground truth image, the ECG reconstruction images of EKGAN (with attention mechanism removed), and CycleGAN, respectively. The images show the reconstruction results of limb leads (II, III, aVR, aVL, aVF) and precordial leads (V1, V2, V3, V4, V5, V6), respectively. It can be seen that the reconstruction results of CycleGAN have a larger error, while the reconstruction results of EKGAN are closer to the ground truth image.

[0070] Figure 7 The images show the ECG reconstruction results of various networks for people with normal heart rates. The black, red, and blue waveforms correspond to the ground truth image, the ECG reconstruction images of the network of this invention, and the ECG reconstruction images of EKGAN (with attention mechanism removed). The images show the reconstruction results of limb leads (II, III, aVR, aVL, aVF) and precordial leads (V1, V2, V3, V4, V5, V6) in that order. It can be seen that the reconstruction results of the network of this invention are closer to the ground truth, which demonstrates the superiority of this invention.

[0071] Figure 8 The images show the ECG reconstruction results of various networks for patients with myocardial infarction. The black, red, and blue waveforms correspond to the ground truth image, the ECG reconstruction images of the network of this invention, and the ECG reconstruction images of EKGAN (with attention removed). The images show the reconstruction results of limb leads (II, III, aVR, aVL, aVF) and precordial leads (V1, V2, V3, V4, V5, V6) in that order. It can be seen that the reconstruction results of this invention are closer to the ground truth, which demonstrates the superiority of this invention.

[0072] Experimental results show that the 12-lead electrocardiogram reconstruction method based on attention generative adversarial network (GAN) of this invention is effective. Applying the attention mechanism to the generator of GAN network can effectively focus on salient regions, highlight the salient features of specific local regions, and improve the sensitivity and prediction accuracy of the model.

[0073] The above description of the embodiments is provided to enable those skilled in the art to understand and apply the present invention. It will be apparent to those skilled in the art that various modifications can be made to the above embodiments, and the general principles described herein can be applied to other embodiments without inventive effort. Therefore, the present invention is not limited to the above embodiments, and any improvements and modifications made to the present invention by those skilled in the art based on the disclosure thereof should be within the scope of protection of the present invention.

Claims

1. A method for 12-lead electrocardiogram reconstruction based on attention mechanism generative adversarial networks, comprising the following steps: (1) Obtain 12-lead electrocardiogram (ECG) signals from patients including those with normal heart rate, myocardial infarction, and various heart diseases, and divide these ECG signals into training set and test set after preprocessing; (2) Construct a 12-lead electrocardiogram reconstruction model based on generative adversarial network (GAN), which includes a discriminator D and two generators G. I and G L The system consists of inputting single-lead ECG signals into G... I and G L In the middle, G I Used to reconstruct and generate the corresponding 12-lead ECG signal, G L This is equivalent to an autoencoder, which encodes the latent space vector z. l As G I The encoder outputs the latent space vector z i The tag makes G I The decoder is better able to learn the important characteristics of single-lead ECG signals; the discriminator D is used to analyze G... I The generated 12-lead ECG signal and the original 12-lead ECG signal are compared to determine whether they are genuine or fake. The generator G I The encoder consists of five cascaded encoding modules, each of which comprises a convolutional layer, a batch normalization layer, and a Leaky ReLU activation function layer connected in sequence; G I The decoder is the inverse structure of the encoder, which consists of five cascaded decoding modules. Except for the last decoding module, the other decoding modules are composed of attention mechanism layer, deconvolution layer, batch normalization layer and ReLU activation function layer connected in sequence. Starting from the second decoding module, the output of the previous decoding module is concatenated with the output of the corresponding encoding module and used as the input of the current decoding module. The last decoding module only contains deconvolution layer. The generator G L The structure is mostly the same as that of the generator G I The structures are similar, the difference lies in G. L The decoding module does not contain an attention mechanism layer and is not concatenated with the output of the encoding module as input; the discriminator uses a 1D U-Net network, the structure of which is the same as the generator G. L Same structure; (3) The above model is trained using the training set. Specifically, the 12-lead ECG signal in the training set is used as the label, and single-lead ECG signals are extracted from it as the model input. Specifically, the ECG signal of any lead is extracted from the 12-lead ECG signal in the training set, copied 12 times, and two extra rows are added above and below for zero padding. Then, the data is batch-combined and input into the model for training. The data dimension of the input model is b×m×n×c, where b is the number of signal segments in a batch, m is the time length of the segment, c is the number of channels, and n=16. The optimization objective function expression used in the model training process is as follows: in: This represents the finally trained generator G. I , To generate the loss function for a Generative Adversarial Network (GAN), For generator G I loss function, For generator G L loss function, Let be the loss function for the latent space vector, and λ and α be hyperparameters. Let ||||1 denote the expectation function, and e denote the L1 norm. i For the single-lead ECG signal of the input model, e o For a true 12-lead ECG signal, D(e) o ) indicates that the discriminator is for e o The discrimination result, G I (e i ) represents the generator G I For e i The reconstructed 12-lead ECG signal, D(G) I (e i )) indicates that the discriminator is for G I (e i The discrimination result of G) L (e i ) represents the generator G L For e i The reconstructed single-lead ECG signal; (4) Extract single-lead ECG signals from the 12-lead ECG signals in the test set and input them into the generator G of the trained model. I In this way, the corresponding 12-lead ECG signal can be reconstructed and output.

2. The 12-lead electrocardiogram reconstruction method according to claim 1, characterized in that: In step (1), the obtained 12-lead ECG signal is first filtered to eliminate low-frequency drift and interference in the ECG signal. Then, the ECG signal is divided into multiple signal segments of fixed time length, and several additional segments are generated for each group of ECG signals for data augmentation. The size of each signal segment is m×y, where m is the time length of the segment and y=12 is the number of leads.

3. The 12-lead electrocardiogram reconstruction method according to claim 1, characterized in that: The attention mechanism layer will respectively connect the input to the linear transformation matrix W Q W K and W V Multiplying them yields the query vector matrix Q, the key vector matrix K, and the value vector matrix V. The output result is then calculated using the following formula. Where: Attention(Q, K, V) is the output of the attention mechanism layer, and d is the number of columns in the query vector matrix Q. T This indicates transpose, and softmax() represents the normalized exponential function.

4. A computer device, comprising a memory and a processor, characterized in that: The memory contains a computer program, and the processor executes the computer program to implement the 12-lead electrocardiogram reconstruction method as described in any one of claims 1 to 3.

5. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by the processor, it implements the 12-lead electrocardiogram reconstruction method as described in any one of claims 1 to 3.

Images