Apparatus for generating electrocardiogram reconstruction model, method for generating electrocardiogram reconstruction model, and method for reconstructing electrocardiogram

Abstract

The present invention relates to an apparatus for generating a generative adversarial network (GAN)-based electrocardiogram reconstruction model, the apparatus comprising: an input unit for receiving a single-lead electrocardiogram; a generator for reconstructing a 12-lead electrocardiogram on the basis of the single-lead electrocardiogram; and a discriminator that uses one of a first input value, which is a combination of the 12-lead electrocardiogram and the single-lead electrocardiogram, or a second input value, which is a combination of an actual 12-lead electrocardiogram (ground-truth) and the single-lead electrocardiogram, as an input value, and determines whether the input value is the reconstructed 12-lead electrocardiogram or the actual 12-lead electrocardiogram. According to the present invention, a multi-lead electrocardiogram that more accurately reflects the characteristics of an actual electrocardiogram can be reconstructed from a single lead electrocardiogram.

Description

Electrocardiogram restoration model generation device, electrocardiogram restoration model generation method, and electrocardiogram restoration method

[0001] The disclosed invention relates to an electrocardiogram restoration model generation device, an electrocardiogram restoration model generation method, and an electrocardiogram restoration method based on a generative adversarial neural network capable of restoring a multi-lead electrocardiogram from a single-lead electrocardiogram.

[0002] And this invention is the result of research supported by the National Research Foundation of Korea (Project No.: 2022RIS-005, Local Government-University Cooperation-Based Regional Innovation Project, Project Identification Number: 1345370814, Contribution Rate: 50%) with funding from the government (Ministry of Education) in 2024, and the National IT Industry Promotion Agency (NIPA) funded by the government (Ministry of Science and ICT) (No. RS-2022-00155966, Artificial Intelligence Convergence Innovation Talent Development (Ewha Womans University), National Project Identification Number: 1711179344, Contribution Rate: 30%), (IITP-2024-RS-2023-00260267, Regional Intelligence Innovation Talent Development (Kangwon National University), Project Identification Number: 1711198468, Contribution Rate: 20%)).

[0003] An electrocardiogram is a graphic representation of the heart's electrical activity during a cardiac cycle. An electrocardiogram is an essential test for diagnosing heart abnormalities and irregular heartbeats, such as cardiac arrhythmias. It detects the heart's electrical activity by attaching electrodes to the skin. A representative electrocardiogram test method is the standard 12-lead electrocardiogram, which utilizes six limb leads (Leads I, II, III, aVR, aVL, aVF) in which electrode cables are attached to the extremities, and six chest leads (V1, V2, V3, V4, V5, V6) in which electrode cables are attached to specific locations on the chest.

[0004] Figure 1 illustrates a typical electrocardiogram (ECG). As shown in Figure 1, each lead of an ECG appears similar, but each contains unique cardiac information. Typically, different leads are monitored for each heart condition to diagnose it. The cardiac information obtained from a single lead is inherently limited. Therefore, when diagnosing heart disease using an ECG, it's crucial to systematically observe various leads to obtain a comprehensive assessment.

[0005] Recently, electrocardiogram (ECG) monitoring has become possible through smart devices such as smartwatches like the Galaxy Watch and Apple Watch, as well as portable electrocardiographs. However, most of these smart devices can only measure single-lead ECGs. While simple atrial fibrillation can be diagnosed with a single lead, other heart conditions are difficult to measure.

[0006] Recently, a growing number of studies have been using electrocardiograms (ECGs) to accurately diagnose heart disease and predict related conditions. While measuring ECGs using smart devices offers the advantages of being relatively easy and convenient, and allowing for long-term recording, as mentioned above, most ECGs measured using smart devices are single-lead ECGs. Therefore, diagnostic techniques utilizing 12-lead ECGs, which have been actively researched to date, cannot be applied.

[0007] In the case of conventional technologies, there is a technology that restores a missing electrocardiogram using an artificial intelligence model based on a long-term memory neural network, such as Korean Patent No. 10-2570598 (announcement date: August 28, 2023), or generates the remaining 12-n electrocardiograms from information from n electrodes based on an adversarial neural network algorithm, such as Korean Patent No. 10-2412974 (announcement date: June 24, 2022). However, these conventional technologies only aim to generate or restore an electrocardiogram well, and do not perform detailed comparisons at the pixel or time level. Furthermore, in order to diagnose a disease using an electrocardiogram, it is necessary to confirm that the restored electrocardiogram well reflects the characteristics of the actual electrocardiogram. In other words, a technology is required that does not simply restore the electrocardiogram, but ensures that the restored electrocardiogram well reflects the characteristics of the existing actual electrocardiogram.

[0008] For the above reasons, one aspect of the disclosed invention is to provide an electrocardiogram restoration model generation device, an electrocardiogram restoration model generation method, and an electrocardiogram restoration method based on a generative adversarial neural network capable of restoring multiple electrocardiograms from a single electrocardiogram so as to accurately reflect the characteristics of an actual electrocardiogram.

[0009] According to one aspect of the disclosed invention, an electrocardiogram reconstruction model generation device for generating an electrocardiogram reconstruction model based on a generative adversarial network (GAN) may include: an input unit for receiving a single-lead electrocardiogram; a generator for reconstructing a 12-lead electrocardiogram based on the single-lead electrocardiogram; and a discriminator for using either a first input value obtained by combining the 12-lead electrocardiogram and the single-lead electrocardiogram or a second input value obtained by combining an actual 12-lead electrocardiogram (ground-truth) and the single-lead electrocardiogram as an input value to determine whether the input value is the reconstructed 12-lead electrocardiogram or the actual 12-lead electrocardiogram.

[0010] The generator may include a label generator that receives the single-lead electrocardiogram as input and outputs a second latent vector for outputting a single-lead electrocardiogram identical to the single-lead electrocardiogram based on a U-Net-based encoder-decoder generation model that does not apply skip connections; and an inference generator that performs preprocessing on the single-lead electrocardiogram based on a U-Net-based encoder-decoder generation model and reconstructs and outputs the 12-lead electrocardiogram based on the first latent vector and the second latent vector of the preprocessed single-lead electrocardiogram.

[0011] The above inference generator can perform learning based on an adversarial loss function for the first input value and the second input value.

[0012] The above inference generator can perform learning based on an L1 loss function that reflects the Manhattan distance between the 12-lead electrocardiogram and the actual 12-lead electrocardiogram.

[0013] The above inference generator can perform learning based on a latent vector loss function for the first latent vector and the second latent vector.

[0014] The above inference generator can perform learning based on a total loss function that assigns weights to the adversarial loss function, the L1 loss function, and the latent vector loss function, respectively.

[0015] The above discriminator may be an encoder-decoder generation model based on a 1D U-Net.

[0016] According to one aspect of the disclosed invention, a method for generating an electrocardiogram restoration model based on a generative adversarial network (GAN) may include the steps of: receiving a single-lead electrocardiogram as input; restoring a 12-lead electrocardiogram based on the single-lead electrocardiogram; receiving as input either a first input value obtained by summing the 12-lead electrocardiogram and the single-lead electrocardiogram or a second input value obtained by summing an actual 12-lead electrocardiogram (ground-truth) and the single-lead electrocardiogram; and determining whether the input value is the restored 12-lead electrocardiogram or the actual 12-lead electrocardiogram.

[0017] The step of restoring a 12-lead electrocardiogram based on the single-lead electrocardiogram may include: performing preprocessing on the single-lead electrocardiogram based on a U-Net-based encoder-decoder generation model; receiving the single-lead electrocardiogram as input based on a U-Net-based encoder-decoder generation model that does not apply skip connections and outputting a second latent vector for outputting a single-lead electrocardiogram identical to the single-lead electrocardiogram; and restoring and outputting the 12-lead electrocardiogram based on the first latent vector of the preprocessed single-lead electrocardiogram and the second latent vector.

[0018] The method for generating an electrocardiogram restoration model according to an embodiment of the present invention may further include a step of performing learning based on an adversarial loss function for the first input value and the second input value.

[0019] A method for generating an electrocardiogram restoration model according to an embodiment of the present invention may further include a step of performing learning based on an L1 loss function that reflects the Manhattan distance between the 12-lead electrocardiogram and the actual 12-lead electrocardiogram.

[0020] The method for generating an electrocardiogram restoration model according to an embodiment of the present invention may further include a step of performing learning based on a latent vector loss function for the first latent vector and the second latent vector.

[0021] The method for generating an electrocardiogram restoration model according to an embodiment of the present invention may further include a step of performing learning based on a total loss function in which weights are respectively assigned to the adversarial loss function, the L1 loss function, and the latent vector loss function.

[0022] According to one aspect of the disclosed invention, a method for restoring an electrocardiogram using a generative adversarial network (GAN)-based electrocardiogram restoration model may include the steps of: receiving a single-lead electrocardiogram; preprocessing the single-lead electrocardiogram; restoring a 12-lead electrocardiogram by inputting the single-lead electrocardiogram into a pre-trained generative adversarial network-based electrocardiogram restoration model; and outputting the restored 12-lead electrocardiogram.

[0023] According to one aspect of the disclosed invention, an electrocardiogram restoration model generation device, an electrocardiogram restoration model generation method, and an electrocardiogram restoration method based on a generative adversarial network capable of restoring a multi-lead electrocardiogram that more accurately reflects the characteristics of an actual electrocardiogram from a single-lead electrocardiogram can be provided.

[0024] According to one aspect of the disclosed invention, reconstructing a multi-lead electrocardiogram from only a single-lead electrocardiogram can save medical professionals money and time in diagnosing heart disease.

[0025] According to one aspect of the disclosed invention, in addition to electrocardiograms, when specific multiple signals exist, it is possible to restore a missing signal or restore a signal that is difficult to measure from another signal, so that it can be applied to various fields such as various brain wave or cerebral blood flow measurements.

[0026] Figure 1 is a graph showing a typical 12-lead electrocardiogram image.

[0027] Fig. 2 is a drawing schematically showing the configuration of an electrocardiogram restoration model generation device according to one embodiment.

[0028] Figure 3 schematically illustrates an electrocardiogram restoration model according to one embodiment.

[0029] Fig. 4 is a flowchart showing a method for generating an electrocardiogram restoration model according to one embodiment.

[0030] FIG. 5 is a flowchart showing in more detail the steps of restoring a 12-lead electrocardiogram among the methods for generating an electrocardiogram restoration model according to one embodiment.

[0031] Figure 6 is a diagram comparing the restoration performance of an electrocardiogram restoration model according to one embodiment and existing models.

[0032] Figure 7 is a diagram showing the performance difference according to the presence or absence of a discriminator and a label generator in an electrocardiogram restoration model according to one embodiment.

[0033] FIG. 8 is a diagram comparing the prediction performance for three diseases of a 12-lead electrocardiogram recovered by an electrocardiogram recovery model according to one embodiment, a 12-lead electrocardiogram recovered by an existing model, and an actual 12-lead electrocardiogram.

[0034] Figure 9 is a diagram comparing the restoration performance of an electrocardiogram restoration model according to one embodiment and existing models.

[0035] Throughout the specification, the same reference numerals denote the same components. This specification does not describe all elements of the embodiments, and any content that is general in the technical field to which the disclosed invention belongs or that overlaps between the embodiments is omitted. The terms 'part, module, element, block' used in the specification may be implemented in software or hardware, and depending on the embodiments, multiple 'parts, modules, elements, blocks' may be implemented as a single component, or a single 'part, module, element, block' may include multiple components.

[0036] Throughout the specification, when a part is said to be 'connected' to another part, this includes not only direct connection but also indirect connection, and indirect connection includes connection via a wireless communication network.

[0037] Additionally, when a part is said to 'include' a component, this does not mean that it excludes other components, but rather that it may include other components, unless otherwise specifically stated.

[0038] Throughout the specification, when we say that an element is located 'on' another element, this includes not only cases where the element is in contact with the other element, but also cases where another element exists between the two elements.

[0039] The terms first, second, etc. are used to distinguish one component from another, and the components are not limited by the aforementioned terms.

[0040] Singular expressions include plural expressions unless the context clearly indicates otherwise.

[0041] The identification codes for each step are used for convenience of explanation and do not describe the order of each step. Each step may be performed in a different order than specified unless the context clearly indicates a specific order.

[0042] The operating principle and embodiments of the disclosed invention are described below with reference to the attached drawings.

[0043] FIG. 2 is a drawing schematically showing the configuration of an electrocardiogram restoration model generation device (100) according to one embodiment.

[0044] Referring to FIG. 2, an electrocardiogram restoration model generation device (100) according to an embodiment of the present invention may include an input unit (110), a generator (120), and a discriminator (130).

[0045] The input unit (110) can receive a single-lead electrocardiogram of a specific patient. The input unit (110) can preprocess the single-lead electrocardiogram. In addition, the input unit (110) can provide the single-lead electrocardiogram or the preprocessed single-lead electrocardiogram to the generator (120).

[0046] In one embodiment, the input unit (110) may additionally receive an actual 12-lead electrocardiogram (ground-truth) of the specific patient. The input unit (110) may provide the actual 12-lead electrocardiogram to the discriminator (130).

[0047] The generator (120) can restore a 12-lead electrocardiogram based on a single-lead electrocardiogram provided from the input unit (110).

[0048] For this purpose, the generator (120) may include an inference generator (121) and a label generator (122).

[0049] The inference generator (121) performs preprocessing on a single-lead electrocardiogram based on a U-Net-based encoder-decoder generation model, and can restore and output a 12-lead electrocardiogram based on a first latent vector of the preprocessed single-lead electrocardiogram and a second latent vector provided from a label generator (122) to be described later.

[0050] The label generator (122) can input the single-lead electrocardiogram based on an encoder-decoder generation model based on a U-Net that does not apply skip connections and output a second latent vector for outputting a single-lead electrocardiogram identical to the single-lead electrocardiogram.

[0051] The discriminator (130) can use either a first input value that is a sum of a restored 12-lead electrocardiogram and a single-lead electrocardiogram or a second input value that is a sum of an actual 12-lead electrocardiogram and a single-lead electrocardiogram as an input value to determine whether the input value is a restored 12-lead electrocardiogram or an actual 12-lead electrocardiogram.

[0052] According to one embodiment, the discriminator (130) may be a 1D U-Net based encoder-decoder generation model.

[0053] According to one embodiment, the inference generator (121) can perform learning based on an adversarial loss function for the first input value and the second input value.

[0054] According to one embodiment, the inference generator (121) can perform learning based on an L1 loss function that reflects the Manhattan distance between the reconstructed 12-lead electrocardiogram and the actual 12-lead electrocardiogram.

[0055] According to one embodiment, the inference generator (121) can perform learning based on a latent vector loss function for the first latent vector and the second latent vector.

[0056] According to one embodiment, the inference generator (121) can perform learning based on a full loss function that assigns weights to the adversarial loss function, the L1 loss function, and the latent vector loss function, respectively.

[0057] Figure 3 schematically illustrates an electrocardiogram restoration model (EKGAN) according to one embodiment.

[0058] The process of reconstructing a single-lead ECG into a multi-lead ECG can be performed based on a Generative Adversarial Network (GAN). However, conventional GAN ​​models can have problems in accurately reflecting the characteristics of actual (diseased) ECGs. To overcome these limitations when reconstructing a single-lead ECG into a multi-lead ECG using existing GAN models, the present invention discloses an ECG reconstruction model (EKGAN).

[0059] Referring to Figure 3, the electrocardiogram reconstruction model (EKGAN) is an inference generator (G I , 121), label generator (G L , 122) may include a generator (Generator, 120) and a discriminator (D, 130). In addition to this, the electrocardiogram restoration model (EKGAN) may further include a preprocessing unit (not shown).

[0060] The generator (120) plays a role in restoring a 12-lead electrocardiogram when a single-lead electrocardiogram is input. The generator (120) is composed of two sub-generators, namely an inference generator (G I ) and label generator (G L , 122), each of which performs a different role and can organically progress learning.

[0061] Inference Generator (G I, 121) is based on a U-Net-based encoder-decoder generative model to receive a single-lead ECG signal as input and restore it to a 12-lead ECG. At this time, preprocessing of the input signal can be performed at the input of the encoder to configure the encoder-decoder-based model. Afterwards, when the output generated at the output end is transmitted to the input of the discriminator, the discriminator can distinguish between real and fake and proceed with learning using this.

[0062] Inference Generator (G I , 121) may include an input stage, a U-Net-based encoder-decoder generation model, and an output stage.

[0063] The input terminal receives a single-lead electrocardiogram as input. The single-lead electrocardiogram has a size of (1, n, 1). Here, the first 1 indicates the number of leads, n indicates the length of the electrocardiogram, and the last 1 indicates the channel size. Here, the length n can be set by the user in various ways, and in the present invention, the value n=512 will be used for explanation.

[0064] To utilize an encoder-decoder-based model, the input and output sizes must be identical, requiring preprocessing of the single-lead ECG input. To achieve this, the single-lead ECG is simply copied 12 times and zero-padded to the front and back, resulting in a size of (16, 512, 1). Each convolution filter is configured in the order of 64, 128, 256, 512, and 1024.

[0065] The U-Net-based encoder-decoder generation model may include a Contracting Path, which extracts semantic information by looking at a gradually wider range of image pixels as the encoder, a Bottleneck, which generates the first latent vector (Zi) as a section transitioning from the contracting path to the expanding path, and an Expanding Path, which combines the extracted semantic information using pixel location information and skip connections as the decoder, and provides additional information to each pixel.

[0066] U-Net is a model that can extract image features using information that can be verified not only in low dimensions but also in high dimensions, and combines this with location information to achieve good performance.

[0067] U-Net is based on providing additional information to the decoder that could be lost when compressing input data into latent vectors.

[0068] However, the latent vector generated in the existing U-Net model has a disadvantage in that it does not check how well the latent vector contains the features of the input data because it is updated only using the loss function between the original data and the generated data. To solve this problem, the present invention uses a label generator (G L , 122) is used to help compress the input data without losing information. A detailed description of this is provided in the label generator (G L , 122) will be described later.

[0069] The output terminal is configured to have the same size of the output as the input of size (16, 512, 1) preprocessed at the input terminal. The output terminal is configured to have the first latent vector (Latent Vector, Z) of the single-lead electrocardiogram data passed from the input terminal.i ) to restore the 12-lead ECG. Since it is a U-Net-based structure, it learns to make the most of the information of a single-lead ECG through skip connections. The ECG restored at the output is composed of Lead I, II, III, aVR, aVL, aVF, V1-V6 in that order, including zero padding, and each convolution filter can be composed of 512, 256, 128, 64, 1 in that order.

[0070] Label Generator (G L , 122) is used to minimize information loss that occurs in the process of compressing input data based on U-Net. For this purpose, a label generator (G L , 122) is an inference generator (G I , 121) is configured as an encoder-decoder based model that takes as input a single-lead ECG identical to the input and generates an identical single-lead ECG output. Label generator (G L , 122) plays a key role in providing a second latent vector (Z) that can best represent the single-lead electrocardiogram received as input. l ) and to do this, it is induced to generate an electrocardiogram identical to the input. That is, the second latent vector (Latent Vector, Z l ) alone can generate a single-lead ECG well, so this is the second latent vector (Latent Vector, Z l ) means that a lot of information from a single-lead electrocardiogram received as input can be compressed.

[0071] Label Generator (G L , 122) The overall structure of the inference generator (G I , 121) but does not apply skip connections to compress information as much as possible. This is a general U-Net-based encoder-decoder generative model and label generator (G L, 122) is differentiated by the label generator (G L , 122) generated from the second latent vector (Latent Vector, Z l ) is an inference generator (G I , 121) is generated from the input data of the first latent vector (Z i ) and the loss function through the inference generator (G I , 121) can improve the information compression capability.

[0072] Each lead of a 12-lead ECG has unique characteristics. Therefore, it is crucial for the discriminator (D, 130) to individually analyze each lead signal at the pixel level.

[0073] Therefore, when using a 2D convolution-based discriminator used in a GAN model for general image generation, the unique characteristics of each lead may be mixed with those of other leads, resulting in poor restoration quality.

[0074] To solve these problems, the discriminator can use an encoder-decoder generative model based on a 1D U-Net. The discriminator can be an inference generator (G I , 121) is used as input by combining the restored 12-lead electrocardiogram or the actual 12-lead electrocardiogram output from the single-lead electrocardiogram used as input, and the input value is learned to determine whether it is a generated electrocardiogram or an original electrocardiogram.

[0075] That is, the discriminator is an inference generator (G I , 121) is used as the input value, either as the first input value which is the sum of the restored 12-lead electrocardiogram output from the input and the single-lead electrocardiogram used as input, or as the second input value which is the sum of the actual 12-lead electrocardiogram and the single-lead electrocardiogram used as input.

[0076] FIG. 4 is a flowchart showing a method for generating an electrocardiogram restoration model according to one embodiment, and FIG. 5 is a flowchart showing more specifically a step for restoring a 12-lead electrocardiogram among the methods for generating an electrocardiogram restoration model according to one embodiment.

[0077] Referring to FIG. 4, a method for generating an electrocardiogram restoration model according to an embodiment of the present invention may include a step of receiving a single-lead electrocardiogram (1100), a step of restoring a 12-lead electrocardiogram based on the single-lead electrocardiogram (1200), a step of receiving as an input either a first input value that is a sum of the 12-lead electrocardiogram and the single-lead electrocardiogram or a second input value that is a sum of an actual 12-lead electrocardiogram (ground-truth) and the single-lead electrocardiogram (1300), and a step of determining whether the input value is a restored 12-lead electrocardiogram or an actual 12-lead electrocardiogram (1400).

[0078] Referring to FIG. 5, a step (1200) of restoring a 12-lead electrocardiogram based on a single-lead electrocardiogram may include a step (1210) of performing preprocessing on the single-lead electrocardiogram based on a U-Net-based encoder-decoder generation model, a step (1220) of receiving the single-lead electrocardiogram as input and outputting a second latent vector for outputting a single-lead electrocardiogram identical to the single-lead electrocardiogram based on a U-Net-based encoder-decoder generation model that does not apply skip connections, and a step (1230) of restoring and outputting the 12-lead electrocardiogram based on the first latent vector of the preprocessed single-lead electrocardiogram and the second latent vector.

[0079] According to one embodiment, the method for generating an electrocardiogram reconstruction model may further include a step (not shown) of performing learning based on an adversarial loss function for the first input value and the second input value.

[0080] According to one embodiment, the method for generating an electrocardiogram reconstruction model may further include a step (not shown) of performing learning based on an L1 loss function that reflects the Manhattan distance between the reconstructed 12-lead electrocardiogram and the actual 12-lead electrocardiogram.

[0081] According to one embodiment, the method for generating an electrocardiogram reconstruction model may further include a step (not shown) of performing learning based on a latent vector loss function for the first latent vector and the second latent vector.

[0082] According to one embodiment, the method for generating an electrocardiogram restoration model may further include a step (not shown) of performing learning based on a full loss function in which weights are applied to the adversarial loss function, the L1 loss function, and the latent vector loss function, respectively.

[0083] An electrocardiogram restoration method (not shown) according to one embodiment of the present invention may include a step of receiving a single-lead electrocardiogram, a step of preprocessing the single-lead electrocardiogram, a step of restoring a 12-lead electrocardiogram by inputting the single-lead electrocardiogram into an electrocardiogram restoration model based on a pre-trained generative adversarial network, and a step of outputting the restored 12-lead electrocardiogram.

[0084] An electrocardiogram restoration method according to one embodiment assumes the use of a pre-trained electrocardiogram restoration model (EKGAN) disclosed in FIGS. 1 to 3. Accordingly, a 12-lead electrocardiogram can be restored based on an input single-lead electrocardiogram using an electrocardiogram restoration model (EKGAN) that already has high performance.

[0085] Meanwhile, in the present invention, the generator (120) and discriminator (130) can use a loss function for learning the electrocardiogram restoration model (EKGAN). The loss functions used at this time are largely divided into three types: an adversarial loss function, an L1 loss function, and a latent vector loss function. Additionally, a full loss function that assigns weights to each of these three functions can be used.

[0086] Before explaining each loss function, the meaning of the symbols used in each loss function is as follows.

[0087] G I : Inference Generator (121)

[0088] G L : Label Generator (122)

[0089] D: Discriminator (130)

[0090] e i : Input data of a single-lead electrocardiogram replicated 12 times and zero-padded

[0091] e o : Ground-truth of a single-lead ECG

[0092] z i , z l : G I Wow G L Latent Vector generated from

[0093] The adversarial loss function is used to generate an inference generator (G I , 121) is used as a basis for learning about the difference between the data restored from the single-lead electrocardiogram received as input and the actual 12-lead electrocardiogram. The formula is as follows, and the result when the actual data is input to the discriminator (D, 130) and the inference generator (G I, 121) and the loss function of the Conditional GAN ​​method for the input data. That is, the discriminator (D, 130) learns whether the input value is generated data or real data by using the adversarial loss function using the ECG used for restoration and the actual ECG, and the ECG used for restoration and the restored ECG.

[0094]

[0095] The L1 loss function can additionally use the L1 loss, which reflects the Manhattan distance between the real and generated ECGs, in addition to the adversarial loss. This is used in the inference generator (G) that performs the actual reconstruction. I , 121) In addition to the inference generator (G I , 121) to help with label generator (G L , 122) can be used in the same way, and the detailed formula is as follows.

[0096]

[0097] The latent vector loss function is a label generator (G L , 122) is generated from the second latent vector (Latent Vector, Z l ) and inference generator (G I , 121) is generated from the first latent vector (Z i ) can be used to calculate the L1 loss between two latent vectors to determine their similarity. The detailed formula is as follows.

[0098]

[0099] The total loss function is the inference generator (G) that uses the aforementioned loss functions. I, 121) is summarized as the loss, as shown in the following formula. λ and α are variables that control relative importance and can be set differently depending on the given purpose, and in the embodiment of the present invention, λ = 50, α = 1.

[0100]

[0101] Referring to FIGS. 6 to 9, the electrocardiogram restoration model (EKGAN) according to an embodiment of the present invention receives a single-lead electrocardiogram, Lead I, as input and outputs a restored 12-lead electrocardiogram. In order to verify the restoration performance of the electrocardiogram restoration model (EKGAN), learning and verification were performed using electrocardiograms collected at Ewha Womans University Mokdong Hospital and Ewha Womans University Seoul Hospital, and comparison was made with the existing GAN-based model. In addition, a label generator (GAN) was also used. L , 122) was also measured for performance to verify its usefulness. RMSE (Root Mean Square Error), MAE (Mean Absolute Error), PRD (Percentage Root Mean Square Difference), MMD (Maximum Mean Discrepancy), and MAEHR (Mean Absolute Error for Heart Rates) were used as performance measurement indices.

[0102] Figure 6 is a diagram comparing the restoration performance of an electrocardiogram restoration model (EKGAN) according to one embodiment and existing models.

[0103] As can be seen in Figure 6, it can be confirmed that it shows better performance in all indicators than existing well-known GAN models such as Pix2Pix, CycleGAN, and CardioGAN.

[0104] Figure 7 shows a discriminator (D, 130) and a label generator (G) of an electrocardiogram restoration model (EKGAN) according to one embodiment. L , 122) is a diagram showing the performance difference depending on the presence or absence of the device.

[0105] Referring to FIG. 7, the EKGAN w / o 1D discriminator (D, 130) shows a case in which the 1D U-Net model-based discriminator (D, 130) is not used in the electrocardiogram restoration model (EKGAN) according to an embodiment of the present invention, but the PatchGAN discriminator used in Pix2Pix is ​​used, and it can be seen that the performance is degraded.

[0106] Also, EKGAN w / o label generator (G L , 122) is a label generator (G) in the electrocardiogram restoration model (EKGAN) according to an embodiment of the present invention. L , 122) is used, and the restoration performance is shown when the label generator (G L , 122) can be confirmed to have lower performance compared to the electrocardiogram restoration model (EKGAN).

[0107] FIG. 8 is a diagram comparing the prediction performance for three diseases of a 12-lead electrocardiogram recovered by an electrocardiogram recovery model (EKGAN) according to one embodiment, a 12-lead electrocardiogram recovered by an existing model, and an actual 12-lead electrocardiogram.

[0108] Referring to Fig. 8, the prediction results of the existing well-known electrocardiogram classification model, the prediction results of the original electrocardiogram, and the prediction results of the restored electrocardiogram are compared, and it can be seen that the disease prediction (classification result) using the electrocardiogram restoration model (EKGAN) of the present invention shows excellent performance. In the experiment, the prediction performance was confirmed for three diseases that can be diagnosed through electrocardiogram. In the case of atrial fibrillation (AF), detection is possible with a single lead, but in the case of LBBB and RBBB (Left / Right Bundle Branch Block), detection is possible only when multiple leads are confirmed. In this way, it was experimentally proven through the results of Fig. 8 that the 12-lead electrocardiogram restored using the electrocardiogram restoration model (EKGAN) according to the embodiment of the present invention can be applied to various downstream tasks.

[0109] FIG. 9 is a diagram comparing the restoration performance of an electrocardiogram restoration model (EKGAN) according to an embodiment of the present invention with existing models. More specifically, FIG. 9 compares a 12-lead electrocardiogram actually restored by the electrocardiogram restoration model (EKGAN) according to an embodiment of the present invention and the models used for performance comparison with parts of the original electrocardiogram (V1, Lead II, V4, V6).

[0110] Referring to FIG. 9, it can be seen that the electrocardiogram restoration model (EKGAN) according to an embodiment of the present invention restores an electrocardiogram that has a form that is significantly more similar to the actual original electrocardiogram than other models. This means that the electrocardiogram restoration model (EKGAN) according to an embodiment of the present invention can accurately capture disease characteristics from a single-lead electrocardiogram and restore a 12-lead electrocardiogram relatively more accurately than other models.

[0111] The above description is merely an illustrative illustration of the technical idea of ​​the present invention, and those skilled in the art will appreciate that various modifications and variations can be made without departing from the essential characteristics of the present invention. Therefore, the embodiments disclosed in the present invention are intended to illustrate, rather than limit, the technical idea of ​​the present invention, and the scope of the technical idea of ​​the present invention is not limited by these embodiments. The scope of protection of the present invention should be interpreted by the following claims, and all technical ideas within a scope equivalent thereto should be interpreted as being included in the scope of the rights of the present invention.

[0112] The present invention can be used in wearable devices and diagnostic applications, electrocardiogram diagnostic equipment, etc., which enable more accurate and rapid diagnosis using 12-lead electrocardiogram restoration technology.

Claims

1. In an electrocardiogram restoration model generation device that generates an electrocardiogram restoration model based on a generative adversarial network (GAN), Input section for receiving a single lead electrocardiogram; A generator for reconstructing a 12-lead electrocardiogram based on the single-lead electrocardiogram; and A discriminator that uses as an input either a first input value that is a sum of the 12-lead electrocardiogram and the single-lead electrocardiogram or a second input value that is a sum of an actual 12-lead electrocardiogram (ground-truth) and the single-lead electrocardiogram, and determines whether the input value is the restored 12-lead electrocardiogram or the actual 12-lead electrocardiogram; An electrocardiogram restoration model generation device including:

2. In paragraph 1, The above generator is, A label generator for receiving the single-lead electrocardiogram and outputting a second latent vector for outputting a single-lead electrocardiogram identical to the single-lead electrocardiogram based on an encoder-decoder generation model based on a U-Net without applying skip connections; and An inference generator that performs preprocessing on the single-lead electrocardiogram based on an encoder-decoder generative model based on U-Net, and restores and outputs the 12-lead electrocardiogram based on the first latent vector and the second latent vector of the preprocessed single-lead electrocardiogram; An electrocardiogram restoration model generation device including:

3. In paragraph 2, The above inference generator is, An electrocardiogram restoration model generation device that performs learning based on an adversarial loss function for the first input value and the second input value.

4. In paragraph 3, The above inference generator is, An electrocardiogram restoration model generation device that performs learning based on an L1 loss function reflecting the Manhattan distance between the above 12-lead electrocardiogram and the actual 12-lead electrocardiogram.

5. In paragraph 4, The above inference generator is, An electrocardiogram restoration model generation device that performs learning based on a latent vector loss function for the first latent vector and the second latent vector.

6. In paragraph 5, The above inference generator is, An electrocardiogram restoration model generation device that performs learning based on a total loss function in which weights are applied to the above adversarial loss function, the L1 loss function, and the latent vector loss function, respectively.

7. In paragraph 1, The above discriminator is, An electrocardiogram restoration model generation device, which is an encoder-decoder generation model based on 1D U-Net.

8. A method for generating an electrocardiogram restoration model based on a generative adversarial network (GAN), Step of receiving a single lead electrocardiogram; A step of restoring a 12-lead electrocardiogram based on the above single-lead electrocardiogram; A step of receiving as an input value either a first input value that is a sum of the 12-lead electrocardiogram and the single-lead electrocardiogram or a second input value that is a sum of the actual 12-lead electrocardiogram (ground-truth) and the single-lead electrocardiogram; and A step of determining whether the above input value is the restored 12-lead electrocardiogram or the actual 12-lead electrocardiogram; A method for generating an electrocardiogram restoration model including:

9. In paragraph 8, The step of restoring a 12-lead electrocardiogram based on the above single-lead electrocardiogram is as follows: A step of performing preprocessing on the single-lead electrocardiogram based on an encoder-decoder generative model based on U-Net; A step of inputting the single-lead electrocardiogram and outputting a second latent vector for outputting a single-lead electrocardiogram identical to the single-lead electrocardiogram based on an encoder-decoder generation model based on a U-Net without applying skip connections; and A step of restoring and outputting the 12-lead electrocardiogram based on the first potential vector and the second potential vector of the preprocessed single-lead electrocardiogram; A method for generating an electrocardiogram restoration model including:

10. In paragraph 9, A step of performing learning based on an adversarial loss function for the first input value and the second input value; A method for generating an electrocardiogram restoration model including more.

11. In paragraph 10, A step of performing learning based on an L1 loss function reflecting the Manhattan distance between the above 12-lead electrocardiogram and the actual 12-lead electrocardiogram; A method for generating an electrocardiogram restoration model including more.

12. In paragraph 11, A step of performing learning based on a latent vector loss function for the first latent vector and the second latent vector; A method for generating an electrocardiogram restoration model including more.

13. In paragraph 12, A step of performing learning based on a total loss function in which weights are respectively assigned to the adversarial loss function, the L1 loss function, and the latent vector loss function; A method for generating an electrocardiogram restoration model including more.

14. A method for restoring an electrocardiogram using an electrocardiogram restoration model based on a generative adversarial network (GAN), Step of receiving a single lead electrocardiogram; A step of preprocessing the single lead electrocardiogram; A step of restoring a 12-lead electrocardiogram by inputting the single-lead electrocardiogram into a pre-trained generative adversarial network-based electrocardiogram restoration model; and A step of outputting the restored 12-lead electrocardiogram; A method for restoring an electrocardiogram, comprising:

Images