Computer-implemented method

Abstract

SOLUTION: A computer-implemented method includes: receiving ECG data describing an electrocardiogram (ECG) of a subject in a period by a system formed of one or more computers; processing the ECG data to generate a predictive input; providing the predictive input to a survival estimation model which has been generated by processing multiple pieces of training data for predicting a survival estimation for the subject; and outputting the prediction of the survival estimation for the subject. The survival estimation is a future survival rate of the subject and estimated ejection rate characteristics are mapped to the future survival ratio of the subject.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] This specification describes computer-based techniques for analyzing physiological electrical data (e.g., electrocardiogram data) for purposes such as estimating the ejection fraction characteristics of a patient.

[0002] [Cross-Reference to Related Applications] This application claims priority to U.S. Patent Application No. 62 / 599,163, filed Dec. 15, 2017, and U.S. Patent Application No. 62 / 569,268, filed Oct. 6, 2017. The disclosures of these prior applications are considered to be part of the disclosure of this application and are hereby incorporated by reference in their entirety.

[0003] [Background] Ejection fraction is a major measure of the health of the hearts of humans and other mammals. Ejection fraction typically indicates the amount of blood ejected from the heart with each pumping. With each pumping cycle (i.e., “heartbeat” or “cardiac cycle”), the myocardium contracts and relaxes to pump blood into the subject's arterial system. When the heart relaxes, the ventricles fill with blood. Then, during the contraction phase of the pumping cycle, a portion of the blood is sent from the ventricles through the aorta into the arterial system. However, not all of the blood that filled the ventricles during relaxation can be pumped out during contraction. The percentage of the blood actually ejected from the ventricles during the pumping cycle is called the ejection fraction. Ejection fraction often relates only to the measurement of the portion of the blood ejected from the left ventricle, but for the purposes of this specification, ejection fraction can alternatively refer to the portion of the blood ejected from only the left ventricle, only the right ventricle, or both ventricles, and can indicate the strength or health of the myocardium.

[0004] Generally, an ejection fraction greater than about 50-55 percent is considered normal for humans. However, some people (often without realizing it) have a low ejection fraction, for example, below the range of 50-55 percent, or an extremely low ejection fraction, for example, 35 percent or less. This is called asymptomatic ventricular dysfunction (ASVD). Low or extremely low ejection fractions are often markers of serious heart complications such as cardiac arrest, sudden death, and various stages of heart failure. When diagnosed with ASVD, there are effective treatments for the treatment of ASVD or the prevention of these complications.

[0005] Ejection fraction has conventionally been measured using echocardiography. During echocardiography, a sonographer uses a sound wave transducer and special techniques to generate images of the heart. Echocardiography is non-invasive, but often requires a specialist to perform and interpret the procedure, and typically requires expensive equipment and infrastructure within the patient's healthcare provider's facility. Ejection fraction has also been measured using magnetic resonance imaging (MRI) techniques, computed tomography (CT) techniques, and nuclear medicine scans. All of these require specially trained personnel and expensive equipment.

[0006] [Summary] This specification discloses systems, methods, devices, and other techniques for estimating a mammalian cardiac ejection fraction from an electrocardiogram (ECG). An electrocardiogram is a measurement of the electrical activity of a mammalian heart. The pumping action of the heart is driven by successive cycles of myocardial electrical polarization and depolarization. This electrical activity can be captured by an electrocardiogram, whereby electrodes are placed on the subject's body surface (e.g., the subject's chest and limbs), and the potential between each pair of electrodes over a period of time is measured. The electrical signals captured by this process form an electrocardiogram. When the electrocardiogram is graphed to show the change in potential between electrodes over time, waveforms or ECG tracings can be seen that show the polarization and depolarization of the heart in each of one or more cardiac cycles. For the purposes of this specification, an electrocardiogram can include any number of leads, from a conventional 12-lead, additional leads, or at least a single lead. Additionally, an ECG can be obtained from adhesive electrodes, conductive electrodes, capacitive electrodes, handheld electrodes, wearable / clothing-type electrodes, subcutaneous electrodes, electrodes attached to an implanted device, or any combination thereof.

[0007] The shape of the ECG waveform is affected by multiple factors. The techniques disclosed in this specification are, in part, premised on the recognition that the ECG of a subject may be affected by the heart ejection rate. For example, in the case of a subject with a low or very low ejection rate compared to another subject with a normal ejection rate, minute deformations may be imparted to one or more portions of the ECG waveform. Regardless of the cause, whether due to atherosclerotic disease, muscle disorder, inflammation, or valve disorder, underlying diseases that affect the heart may impair the pumping ability of the myocardium. Underlying diseases may similarly affect the metabolism of individual muscle cells or their interconnections and may lead to fibrosis deposition or infiltration of inflammatory cells, all of which result in minute electrical changes. These local cardiac electrical changes contribute to the deformations recorded on the surface ECG. Such deformations may not be visible to the naked eye, but nevertheless may be detectable using computer-based models with the techniques disclosed in this specification. Accordingly, this specification describes how to use the subject's ECG as a screening tool to predict the subject's ejection rate characteristics. In many scenarios, ECG-based screening is preferred over measurement of the ejection rate by echocardiography or other means. ECG devices are often more widely available than echocardiography devices and can be performed more quickly without the need for a trained sonographer. Furthermore, consumer and home-use ECG devices (e.g., single-lead ECG patches) are becoming increasingly popular. Using the techniques disclosed in this specification, the number of people selected as having a low ejection rate and the number of screening times can be increased using ECG devices in various settings, whether clinical or home use. In some cases, when the subject's ECG screening indicates a high likelihood of having a low or very low ejection rate, further evaluation of the symptoms can be performed and the symptoms verified by using other measurement means such as echocardiography, MRI, or CT scan.

[0008] The subject matter disclosed in this specification includes computer-implemented methods. The methods can be implemented by a system comprising one or more computers at one or more locations. In some embodiments, the system has one or more processors and one or more computer-readable media encoded with instructions that, when executed by the one or more processors, cause the processors to implement the methods. Some embodiments include only a computer-readable media encoded with instructions that, when executed, cause the methods to be implemented.

[0009] The method can include receiving, by the system, electrocardiogram (ECG) data that describes an ECG of a mammal during a period. The system provides a predicted input derived from the ECG data to an ejection fraction prediction model. The ejection fraction prediction model can be used to process the predicted input to generate an estimated ejection fraction characteristic of the mammal. The system then provides the estimated ejection fraction characteristic of the mammal as an output.

[0010] These and other implementations can optionally further include one or more of the following features.

[0011] The ECG data can include one or more channels, each channel including a subset of the ECG data that describes a respective derivation of the mammal's ECG during the period. The predicted input can characterize each respective derivation of the ECG for each of the one or more channels of the ECG data.

[0012] The ECG data can include a plurality of channels, each channel including a subset of the ECG data that describes a respective one of a plurality of derivations of the mammal's ECG during the period. The predicted input can characterize the plurality of derivations of the ECG for each of the plurality of channels of the ECG data.

[0013] The period during which the ECG data describes the mammal's ECG can span a plurality of cardiac cycles of the mammal.

[0014] The mammal can be a human being.

[0015] The expulsion rate prediction model can be a regression model, such as a logistic regression model.

[0016] The expulsion rate prediction model can be a machine learning model such as a neural network (e.g., a feed-forward neural network, a convolutional neural network, or a recurrent neural network).

[0017] The expulsion rate model can use clinical characteristics (including age, gender, weight, and / or the presence or absence of measurable medical conditions such as hypertension and diabetes) as inputs to further refine its output.

[0018] The expulsion rate model can also generate additional outputs such as the risk of death over a given period (e.g., the risk of death in the next year).

[0019] The estimated expulsion rate characteristic of the mammal can be a value representing an absolute (e.g., intrinsic) estimate of the expulsion rate of the mammal.

[0020] The estimated expulsion rate characteristic of the mammal can indicate an estimated range of the expulsion rate of the mammal.

[0021] Providing the estimated expulsion rate characteristic of the mammal as an output can include providing the estimated expulsion rate characteristic for presentation to the mammal or to a healthcare provider associated with the mammal.

[0022] This method can further include generating a prediction input by determining values for one or more morphological features of the mammal's ECG. The prediction input can indicate values for one or more morphological features of the mammal's ECG.

[0023] The morphological characteristics of the ECG of a mammal can include at least one of T-wave amplitude, P-wave amplitude, P-wave area, T-wave area, left T-wave inclination, right T-wave inclination, left P-wave inclination, right P-wave inclination, T-wave duration, P-wave duration, PR interval, QRS duration, QRS amplitude, QRS area, QRS energy, QRS peak-to-peak ratio, or QT segment length.

[0024] The ejection fraction prediction model can be personalized for a mammal.

[0025] Unless otherwise defined, all technical and scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the technical field to which this invention pertains. Although methods and materials similar or equivalent to those described herein can be used to practice the present invention, the preferred methods and materials are described below. All publications, patent applications, patents, and other references mentioned herein are incorporated by reference in their entirety. In case of conflict, this specification, including definitions, will control. In addition, the materials, methods, and examples are for illustrative purposes only and not intended to be limiting.

[0026] Details of one or more embodiments of the invention are set forth in the accompanying drawings and the description below. Other features, objects, and advantages of the invention will be apparent from the description and drawings, and from the claims.

Brief Description of the Drawings

[0027]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

[0028] Like references and labels in the figures indicate like elements.

[0029] This specification discloses systems, methods, devices, and other techniques for estimating the cardiac ejection rate of a mammal based on the electrocardiogram (ECG) of the mammal. In some implementations, a machine learning model, such as a neural network, is configured to process predictive inputs that characterize the ECG data and output a display of the estimated ejection rate of the mammal. This model can be trained to account for complex combinations of features that are typically not recognizable to humans but are determined (e.g., by an iterative training process) to correlate with specific ejection rate characteristics. Additional details regarding these and other techniques are provided in the following description of FIGS. 1 - 12.

[0030] FIG. 1 is a conceptual diagram of an exemplary system 100 that records and processes ECG data and estimates the ejection rate characteristics of subject 102 using the ECG data. For the purposes of this example, subject 102 is considered to be a human, more specifically a patient of a healthcare provider. However, it should be understood that the description is not limited to this example. In other implementations, subject 102 can be a human not specifically related to a healthcare provider, or can be any other mammal, and an appropriate model is constructed to map the ECG data to estimate the ejection rate for that mammal.

[0031] A set of electrodes 104 is placed on the body surface of patient 102 to enable recording of signals indicative of the electrical activity of patient 102's heart during an ECG procedure. In some implementations, ten leads can be attached to the patient to perform a standard 12-lead ECG recording (for example, placing some electrodes near the heart region of the chest and other electrodes on the patient's limbs). A 12-lead ECG is useful for acquiring multiple channels of ECG data, with the data for each channel representing a respective lead. Each lead is formed by the potential between a pair of electrodes. As a result of changes in the positioning of the electrodes, different angles are formed by different pairs of electrodes for different leads, and each lead provides a different view of the electrical activity of the patient's heart. For example, signals from each lead can be recorded simultaneously over a period (for example, 5, 10, or 15 seconds) to capture information regarding the timing and location of the electrical activity along different radial directions.

[0032] In some implementations, the ECG data can be recorded using a sensor platform or electrode configuration other than or in addition to a 12-lead ECG configuration. For example, a removable ECG patch can be attached to a patient's body surface, and the patch includes two or more electrodes that form one, two, or more leads capable of recording an ECG. In some implementations, patient 102 can manually contact a fixed pair of external electrodes with his or her finger, or patient 102 can wear a watch, wristband, chest band, or other device that fixes two or more electrodes at a fixed position on patient 102 to sense the electrical activity of the patient's heart. Patient 102 or a healthcare provider can use a mobile computing device (e.g., smartphone 106) to configure aspects of an ECG procedure in a clinical or non-clinical setting.

[0033] System 100 further includes a data recorder and control unit 108, an ECG processing system 110, and one or more input / output devices 112a-b. In some implementations, wires extending from each of electrodes 104 are coupled to data recorder and control unit 108 to enable recording of electrical signals sensed by the electrodes. Unit 108 can include, for example, an analog-to-digital (A / D) converter and other analog or digital signal conditioning circuitry such as amplifiers, filters.

[0034] The data recorder and controller unit 108 is communicatively coupled to the ECG processing system 110. The ECG processing system 110 is a system consisting of one or more computers, and these computers can be distributed at one or more physical locations. System 110 may or may not be in the same physical location as patient 102 and the data recorder and control unit 108. System 110 can be coupled to a display screen 112a for presenting information to the user and peripheral devices such as a mouse and keyboard 112 for receiving user input. Further, system 110 can include various components 114-126 that facilitate processing of ECG data to determine estimated ejection rate characteristics, estimated survival rate, and present information regarding such estimates.

[0035] For example, system 110 can include a preprocessor 114 that digitally conditions the ECG signal received from data recorder and control unit 108. The preprocessor 114 can perform noise reduction, anti-aliasing, or other digital techniques to prepare ECG data that describes the patient 102's ECG for further processing.

[0036] System 110 further includes one or more ejection fraction prediction models 118. These models 118 are generally configured to process one or more prediction inputs that characterize the patient's ECG data and generate an estimated ejection fraction characteristic for patient 102 based on the prediction inputs. The ejection fraction characteristic can indicate, for example, an absolute estimated value of the patient's ejection fraction (e.g., a specific value such as 50 percent, 45 percent, 40 percent, 35 percent, 30 percent, or another value), or a classification of the patient's estimated ejection fraction (e.g., a normal ejection fraction greater than 50 percent, a low ejection fraction of 35-50 percent, or a very low ejection fraction below 35 percent).

[0037] As will be described in further detail below, the expulsion rate prediction model 118 can be a regression model, a machine learning model, or both. In some implementations, the model 118 is a feedforward, recurrent, or convolutional neural network, or a capsule network. The neural network model can have fully connected layers and can use an autoencoder network. In some implementations, the system 110 stores and maintains multiple expulsion rate prediction models 118. Each model 118 can correspond to a different set of patient characteristics (e.g., age, weight, gender, or other characteristics). When evaluating the expulsion rate for a new patient 102, the system 110 can select an appropriate model from the models 118 that corresponds to the characteristics of patient 102 that match the characteristics of the model 118. Each model 118 can be trained, for example, specifically with respect to data points from patients having the corresponding characteristics for the model 118. Other types of machine learning or regression models can also be applied, such as support vector machines (SVMs), hidden Markov models (HMMs), and other linear and non-linear systems.

[0038] In some implementations, system 110 maintains one or more survival prediction models 120. The survival prediction model 120 processes ECG data, estimated ejection fraction characteristics, or both, and is configured to generate a prediction of a patient's estimated future survival rate (e.g., 1, 2, 5, and / or 10-year survival rates) indicating the likelihood of surviving beyond the patient's heart condition based on the patient's ejection fraction characteristics. For example, the ejection fraction characteristics model 118 can be used to generate estimated ejection fraction characteristics for a patient. The survival prediction model 120 can then map the estimated ejection fraction characteristics to the patient's future survival rate as determined from empirical data. For example, the estimated 5-year survival rate of a patient with a very low ejection fraction may be lower than that of a patient with a normal ejection fraction. In other implementations, the survival prediction model 120 can process predictive inputs that characterize the patient's ECG without first starting from the patient's estimated ejection fraction and directly derive the estimated survival rate for patient 102 from the predictive inputs. In this way, when determining the estimated survival rate for patient 102, characteristics of the patient's ECG other than or in addition to the ejection fraction can be considered. The survival prediction model 120 can be a regression model, a machine learning model, or both. In some implementations, model 120 is a feedforward, recurrent, or convolutional neural network, or a capsule network. The neural network model can have fully connected layers and can use an autoencoder network. In some implementations, system 110 stores and maintains multiple survival prediction models 120. Each model 120 can correspond to a different set of patient characteristics (e.g., age, weight, gender, or other characteristics). When evaluating the ejection fraction for a new patient 102, system 110 can select an appropriate model from among the models 120 that corresponds to the characteristics of patient 102 that match the characteristics of the model 120. Each model 120 can be trained, for example, specifically with respect to data points from patients having the corresponding characteristics for the model 120.

[0039] The prediction input generator 116 processes the ECG data from the data recorder and control unit 108 or the ECG signal preprocessor 114 to generate a prediction input suitable for processing by the ejection rate prediction model 118 or the survival estimation model 120. For example, the prediction input generator 116 can normalize and vectorize the ECG data from one or more channels (corresponding to one or more leads) into the format expected by the ejection rate prediction model 118. In some implementations, the prediction input includes a time series of values representing the amplitudes of the ECG for one or more leads at each time point at a specified sampling frequency over a period spanning one or more cardiac cycles (e.g., 1, 2, 5, or 10 seconds). In some examples, the prediction input can represent an ECG signal over a relatively short amount of time (e.g., 1, 2, 5, or 10 seconds), while in other examples, the prediction input can represent an ECG signal over a relatively longer time (e.g., 30 seconds, 1 minute, 2 minutes, 5 minutes, 10 minutes, or more). In some implementations, the prediction input generator 116 averages the ECGs from two or more cardiac cycles to generate an average ECG representing the average cardiac cycle of the patient 102. In some implementations, the prediction input generator 116 selects the portion of the ECG corresponding to a single cardiac cycle and characterizes the prediction input with only the selected portion of the ECG. Thus, the selected portion is a representative cardiac cycle of the patient 102.

[0040] In some implementations, rather than processing actual time series data representing the amplitude of the ECG waveform over time, the ejection rate prediction model 118 and the survival estimation model 120 can instead process a prediction input indicative of the value of one or more morphological features of a patient's ECG. Morphological features are parameters that characterize the shape of an ECG waveform or a portion of an ECG waveform, such as a P wave, a QRS complex, or a T wave. Exemplary morphological features that can be identified in the prediction input include T wave amplitude, P wave amplitude, P wave area, T wave area, T wave left inclination, T wave right inclination, P wave left inclination, P wave right inclination, T wave duration, P wave duration, PR interval, QRS duration, QRS amplitude, QRS area, QRS energy, QRS peak-to-peak ratio, QT segment length, or a combination of two or more of these. These multiple features are graphically shown in the exemplary ECG waveform 300 of FIG. 3. In some implementations, the system 110 includes a feature extractor 126, and the feature extractor 126 analyzes the ECG data and determines the value of any applicable morphological feature so as to include a prediction input processed by the ejection rate prediction model 118.

[0041] The system 110 further includes a training subsystem 122 and a notification manager 124. The training subsystem 122 is configured to train the ejection rate prediction model 118, the survival estimation model 120, or both. An exemplary process 600 performed by the training subsystem 122 to train the ejection rate prediction model 118 will be described with respect to FIG. 6. The notification manager 124 is configured to provide an estimated ejection rate characteristic, an estimated survival rate, or both as an output to one or more users. In some implementations, the notification manager 124 provides an estimated ejection rate characteristic for display on the screen 112a. Additionally, one or more notification services can be registered with the computing system 110, and thus the notification manager 124 provides the results of the estimated ejection rate characteristic, the survival rate estimation, or both for each of the registered services for presentation by one or more means (e.g., to a smartphone at a healthcare provider's clinic for visual, audible, and / or tactile presentation).

[0042] FIG. 2 is a flowchart of an exemplary process 200 for estimating the ejection rate characteristics of a subject using ECG data. Process 200 can be implemented by a computing system, such as system 110 of FIG. 1. At step 202, the system acquires ECG data for a patient. The ECG data can include one or more channels, each channel representing an ECG signal for a respective lead of an ECG procedure, and the data from each channel can be time-aligned. The ECG data can span a single cardiac cycle, a portion of a cardiac cycle, or multiple cardiac cycles of the patient. At step 204, a prediction input generator of the system generates one or more prediction inputs from the ECG data and feeds them into an ejection rate prediction model, such as ejection rate prediction model 118. In some implementations, the prediction input represents a time series of values for the ECG waveform. In other implementations, the prediction input represents values of morphological features of the ECG waveform (step 206). At step 208, the system provides the prediction inputs to the ejection rate prediction model for processing. At step 210, the ejection rate prediction model processes the prediction inputs representing the patient's ECG results to generate estimated (predicted) ejection rate characteristics. In some implementations (212), the ejection rate characteristics are absolute estimated values of the patient's ejection rate that identify specific values for the ejection rate. In other implementations (214), the ejection rate characteristics are categories or ranges of ejection rate values (e.g., very low ejection rate, low ejection rate, or normal ejection rate). For example, the ejection rate model can be trained to classify the patient's ejection rate into one of two, three, or more possible ejection rate categories defined by a specified threshold ejection rate value. A binary classification model can classify the patient's ejection rate into two possible categories. The system then, at step 216, stores and / or outputs the estimated ejection rate characteristics, for example, for presentation to the patient or the patient's healthcare provider.

[0043] In some implementations, rather than estimating the patient's current ejection fraction characteristics (similar to the description of other implementations), the ejection fraction prediction model can instead be trained and configured to predict the risk of a patient developing a low or very low ejection fraction at some future point in time (e.g., within the next 1 month, 6 months, 1 year, 2 years, 5 years, or more) by detecting minute features or changes in the patient's ECG that indicate early onset of the disease.

[0044] Figure 3 is a diagram of an exemplary ECG tracing 300 for one heartbeat of a patient. The ECG tracing shows the constituent segments of the heartbeat and various morphological waveform features.

[0045] FIG. 4 is a flow diagram of an exemplary process 400 for selecting and using an appropriate ejection fraction prediction model corresponding to a patient's characteristics. Process 400 can be implemented by a computing system such as system 110 of FIG. 1. In some examples, a global ejection fraction prediction model can be trained and used for a wide range of patients. However, in other examples, a personalized model specific to the patient or a semi-personalized model trained on ECG data and measured ejection fraction characteristics from a population of patients having the same or similar characteristics as the new patient for whom an ejection fraction estimate is desired can be used to determine a more accurate or reliable estimate of the patient's ejection fraction. For example, a plurality of ejection fraction prediction models can be generated, each corresponding to a different set of patient characteristics, such as any indicator of comorbidity, such as age, height, weight, body mass index (BMI), gender, family history, diabetes, hypertension, hyperlipidemia, high-sensitivity C-reactive protein (CRP), smoking, coronary artery disease history, inflammatory disease history, or combinations thereof. At step 402, the system identifies a set of characteristics for the patient for whom an estimated ejection fraction characteristic is to be determined. At step 404, the system selects one model from the ejection fraction prediction models that corresponds to the identified set of characteristics for the patient. For example, in the case of a male patient over 50 years old, a model trained on data from subjects with characteristics similar to the patient can be selected rather than other models trained on data from subjects with other characteristics. At step 406, the system generates an ejection fraction prediction using the selected ejection fraction prediction model corresponding to the patient's characteristics. For example, the system can generate an estimated ejection fraction characteristic by the selected model according to process 200 described with respect to FIG. 2.

[0046] ECG-based estimation of a patient's ejection fraction can be a useful screening procedure, and further evaluation of the patient can be justified based on the results of the ECG-based screening procedure. FIG. 5 is a flowchart of an exemplary process 500 for screening for potentially problematic ejection fraction levels using an ECG-based approach. Process 500 can be automated and can be implemented, for example, by computing system 110 or by a healthcare provider or other appropriate individual. At step 502, the estimated ejection fraction characteristics of the patient are obtained, for example, based on process 200 described with respect to FIG. 2. At step 504, the system determines whether the estimated ejection fraction characteristics, and optionally additional factors, meet one or more screening criteria for guiding a decision as to whether further evaluation of the patient's symptoms is justified. For example, the estimated ejection fraction characteristics can be an absolute value indicating the patient's predicted ejection fraction, and the screening criteria can include a threshold ejection fraction (e.g., 35 percent or 50 percent). If the patient's estimated ejection fraction is below the threshold, subsequent procedures for further evaluation can be performed on the patient. The subsequent procedures can be, for example, echocardiography, MRI, CT scan, nuclear medicine diagnosis, or a combination thereof. In some implementations, if the patient's ejection fraction characteristics are a classification into a range of ECG values that are considered dangerous or not safe, further medical evaluation can be performed on the patient.

[0047] In some embodiments, when the patient's ejection fraction falls below a threshold (e.g., 35 percent or 50 percent), or when the patient's ejection fraction characteristics classify into a range of ECG values that are considered dangerous or not safe, a therapeutic agent can be administered to the patient, or a treatment can be performed on the patient. Such agents or treatments can include beta blockers, angiotensin receptor blockers, statins (for coronary artery disease), implantable defibrillators, cardiac resynchronization devices, and other well-defined treatments (see 2013 ACCF / AHA Guideline for the Management of Heart Failure. Circulation. 2013;128:e240-e327, and 2016 ACC / AHA / HFSA Focused Update on New Pharmacological Therapy for Heart Failure, Circulation. 2016;134:e282-e293). In some embodiments, when the patient's ejection fraction falls below a threshold (e.g., 35 percent or 50 percent), or when the patient's ejection fraction characteristics classify into a range of ECG values that are considered dangerous or not safe, additional diagnostic tests by therapeutic intervention include screening for inflammatory and other systemic conditions, and therapeutic interventions include specific treatments to find coronary artery disease or coronary angiography or other imaging, and such diseases have well-defined treatments.

[0048] FIG. 6 is a flowchart of an exemplary process 600 for training an ejection fraction prediction model such as a neural network-based model. In some implementations, machine learning techniques such as gradient descent (including batch gradient descent or stochastic gradient descent) can be used to train the ejection fraction model. Process 600 can be implemented by a computing system such as training subsystem 122 described with respect to FIG. 1. At step 602, the system obtains a plurality of training data pair sets. Each pair includes an ECG prediction input that characterizes the ECG of a particular patient and a target ejection fraction characteristic for that patient. The target ejection fraction characteristic can be the “true” or measured ejection fraction characteristic determined by an echocardiogram or other available procedure. In some cases, the model is trained on data from many different patients. If the model is semi-personalized, the patients represented in the training set may share a common set of characteristics with each other. At step 604, the training system selects a first training data pair for processing. At step 606, the prediction input from the training data pair is provided to the ejection fraction prediction model. The ejection fraction prediction model can be, for example, a neural network machine learning model having one or more layers of perceptrons with associated weights / parameters for the activation function associated with each perceptron. These weights can be randomly initialized when training begins and can be gradually refined over time as additional training iterations are performed. At step 608, the system processes the prediction input according to the current weights / parameters of the model to generate an estimated ejection fraction characteristic. At step 610, the estimated ejection fraction characteristic is compared with the target ejection fraction characteristic to determine an output error. This output error is then backpropagated through the network using gradient descent to update the current weights / parameters of the neural network. The process then returns to step 604, another training data pair is selected, and operations 604-612 are repeated until a training end condition occurs.

[0049] The ejection rate prediction model can be trained on a large dataset, but the model itself can be made relatively small. After training, the model can be made relatively small and can predict ejection rate characteristics with fewer computational requirements than may have been required during training.

[0050] Process 600 generally belongs to a supervised learning process, but in some implementations, the ejection rate prediction model can also be trained using unsupervised learning techniques.

[0051] FIG. 7 is a flowchart of an exemplary process 700 for predicting the survival rate of a subject from a heart condition such as a low or very low ejection rate from ECG data for the subject. In some implementations, process 700 is performed by a computing system including a survival estimation model such as model 120 (FIG. 1) of ECG processing system 110. At step 702, the system obtains ECG data for a patient. The ECG data can include one or more channels, each channel representing an ECG signal for a respective lead of the ECG procedure, and the data from each channel can be time-aligned. The ECG data can span a single cardiac cycle, a portion of a cardiac cycle, or multiple cardiac cycles of the patient. At step 704, the system's prediction input generator generates one or more prediction inputs from the ECG data and feeds them into the survival estimation model. In some implementations, the prediction inputs represent a time series of values for the ECG waveform. In other implementations, the prediction inputs represent values of morphological features of the ECG waveform (706). At step 708, the prediction inputs are provided to the survival estimation model, and at step 710, the model processes the prediction inputs to determine an estimated survival rate of the patient based on the patient's ECG. The estimated survival rate can be, for example, a prediction for 1 month, 1 year, 2 years, 3 years, 4 years, 5 years, or 10 years. The system can then provide the estimated survival rate as an output for presentation to the patient, the patient's healthcare provider, or both (step 712).

[0052] FIG. 8 is a diagram of an exemplary neural network model 800 that estimates the ejection rate characteristics of a subject from ECG data. The model 800 processes an ECG prediction input that characterizes one or more channels of a patient's ECG. As can be seen in FIG. 8, the model 800 can include an input layer of perceptrons and / or sigmoid neurons, an output layer, and one or more hidden layers between the input layer and the output layer. For example, data from multiple channels can be processed and pooled until the data is provided to the fully connected layers of the network. The model 800 can output an estimate or classification of the ejection rate characteristics for a patient.

[0053] In some implementations, the neural network model 800 includes multiple convolutional layers for feature extraction. Each convolutional layer can include a fixed or variable length filter that focuses on the current lead of the ECG or multiple leads simultaneously (e.g., k×1 or k×m leads). Following the convolutional layers, the network can include one or more fully connected layers each having the same or different number of neurons. Zero, one, or more recurrent neural network layers can be added for temporal feature extraction before, after, or in parallel with the convolutional layers. Skip layers and neurons may or may not be included.

[0054] FIG. 9 shows the results 900 of a first neural network implementation of an ejection rate prediction model that classifies patients into groups according to whether the patient's ejection rate is predicted to be above or below 50 percent.

[0055] FIG. 10 shows the results 1000 of a second neural network implementation of an ejection rate prediction model that classifies patients into groups according to whether the patient's ejection rate is predicted to be above or below 35 percent.

[0056] FIG. 11 is a graph 1100 showing the correlation between the model output and the true ejection rate values in the study.

[0057] This document describes multiple examples of detecting the (low) ejection fraction of a patient's dysfunction using data from a standard 12-lead ECG. However, in other implementations, fewer leads, including single-lead ECGs, can be used to effectively detect examples of low ejection fraction, or individuals who are likely to develop a low ejection fraction within a certain future period (e.g., 5 years) can be identified. The ejection fraction prediction model can be trained and implemented based on data from single-lead, any multi-lead configuration, and any number of ECG leads including the standard 12-lead ECG. For example, it has been shown that a single-lead ECG based on lead 1 is particularly effective in providing reliable data for the ejection fraction prediction model to generate an ejection fraction prediction. Advantageously, ejection fraction predictions based on single-lead ECGs or other electrodes different from the conventional ones (e.g., fewer than the standard 12-lead ECG) can expand access to screening devices that implement the ejection fraction prediction models described herein. For example, a pair of electrodes can be integrated into or communicably coupled to a smartphone or other portable electronic device, enabling an individual to conveniently capture their own ECG by, for example, touching the electrodes with their fingertips and quickly obtain the prediction results. In other implementations, electrodes can also be provided within a wearable patch for continuous monitoring during a period.

[0058] In certain examples, yet another advantage that can be achieved by at least some of the implementations described herein is the ability to detect a low ejection fraction based on a short single time profile of ECG data (e.g., from a single lead or multiple leads) for a patient. For example, the system may only need to process a relatively short sample of the ECG data to generate a high-confidence prediction of the patient's ejection fraction state, such as whether the patient has a low EF. In some implementations, the ejection fraction prediction model is configured to predict whether a patient has a low EF (e.g., an EF of less than 35, 40, or 50 percent) based on an ECG sample of 60 seconds or less. In some implementations, the ejection fraction prediction model is configured to predict whether a patient has a low EF (e.g., an EF of less than 35, 40, or 50 percent) based on an ECG sample of 30 seconds or less. In some implementations, the ejection fraction prediction model is configured to predict whether a patient has a low EF (e.g., an EF of less than 35, 40, or 50 percent) based on an ECG sample of 20 seconds or less. In some implementations, the ejection fraction prediction model is configured to predict whether a patient has a low EF (e.g., an EF of less than 35, 40, or 50 percent) based on an ECG sample of 15 seconds or less. In some implementations, the ejection fraction prediction model is configured to predict whether a patient has a low EF (e.g., an EF of less than 35, 40, or 50 percent) based on an ECG sample of 10 seconds or less. In some implementations, the ejection fraction prediction model is configured to predict whether a patient has a low EF (e.g., an EF of less than 35, 40, or 50 percent) based on an ECG sample of 5 seconds or less. In some implementations, the ejection fraction prediction model is configured to predict whether a patient has a low EF (e.g., an EF of less than 35, 40, or 50 percent) based on an ECG sample of 2 seconds or less. Thus, the acquisition time for the ECG can be made relatively short, which is convenient for the patient.In addition, by using a machine learning model such as a convolutional neural network trained on a large dataset, high-speed processing can be enabled, and a computing system that executes this model can return ejection fraction prediction results based on an ECG input in a relatively short amount of time.

[0059] For certain implementations, models for estimating or predicting an individual's ejection fraction were described. However, these techniques can more generally be extended to models that facilitate the detection of various current or impending structural heart diseases, including ejection fraction abnormalities, left ventricular myocardial mass abnormalities (e.g., increased left ventricular myocardial mass, low left ventricular myocardial mass), valvular heart disease, ischemic heart disease, atrial abnormalities, the presence of a shunt, or patent foramen ovale (PFO), dilation of cardiac chambers (e.g., left atrium, right atrium, left ventricle, right ventricle), or combinations and / or other structural heart diseases thereof.

[0060] FIG. 12 is a block diagram of computing devices 1200, 1250 that can be used as a client or one or more servers to implement the systems and methods described herein. Computing device 1200 is intended to represent various forms of digital computers, such as laptops, desktops, workstations, personal digital assistants, servers, blade servers, mainframes, and other appropriate computers. Computing device 1250 is intended to represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, and other similar computing devices. Additionally, computing device 1200 or 1250 can include a universal serial bus (USB) flash drive. The USB flash drive can store an operating system and other applications. The USB flash drive can include input / output components, such as a wireless transmitter or a USB connector that can be inserted into a USB port of another computing device. The components, connections, and relationships shown here, as well as the functions, are meant to be illustrative only and are not meant to limit the implementations described and / or claimed herein.

[0061] The computing device 1200 includes a processor 1202, a memory 1204, a storage device 1206, a high-speed interface 1208 that connects to the memory 1204 and the high-speed expansion port 1210, and a low-speed interface 1212 that connects to the low-speed bus 1214 and the storage device 1206. The components 1202, 1204, 1206, 1208, 1210, and 1212 are interconnected using various buses respectively and can be mounted on a common motherboard or in other suitable ways. The processor 1202 can process instructions for execution within the computing device 1200, including instructions stored in the memory 1204 or on the storage device 1206 for displaying graphical information as a GUI on an external input / output device such as a display 1216 coupled to the high-speed interface 1208. In other implementations, multiple processors and / or multiple buses can be used as appropriate, along with multiple memories and multiple types of memories. Also, multiple computing devices 1200 can be connected, with each device providing some of the necessary operations (e.g., as a server bank, a group of blade servers, or a multiprocessor system).

[0062] The memory 1204 stores information within the computing device 1200. In one implementation, the memory 1204 is one or more volatile memory units. In another implementation, the memory 1204 is one or more non-volatile memory units. The memory 1204 can also be another form of computer-readable medium such as a magnetic or optical disk.

[0063] Memory device 1206 can provide a large-capacity storage device for computing device 1200. In one implementation, memory device 1206 can be a computer-readable medium such as a floppy (registered trademark) disk device, a hard disk device, an optical disk device or a tape device, a flash memory, or other similar solid-state memory device, or an array of devices including a storage area network or other devices within a configuration, or can include these. A computer program product can be tangibly implemented in an information carrier. The computer program product can also include instructions that, when executed, implement one or more of the methods as described above. The information carrier is a computer or machine-readable medium such as memory 1204, memory device 1206, or memory on processor 1202.

[0064] High-speed controller 1208 manages bandwidth-intensive operations for computing device 1200, and low-speed controller 1212 manages lower-bandwidth-intensive operations. Such a function assignment is merely illustrative. In one implementation, high-speed controller 1208 is coupled to memory 1204, display 1216 (e.g., via a graphics processor or accelerator), and high-speed expansion port 1210, and high-speed expansion port 1210 can accept various expansion cards (not shown). In this implementation, low-speed controller 1212 is coupled to memory device 1206 and low-speed expansion port 1214. The low-speed expansion port can include various communication ports (e.g., USB, Bluetooth (registered trademark), Ethernet (registered trademark), wireless Ethernet), and can be coupled to one or more input / output devices such as a keyboard, a pointing device, a scanner, or a networking device such as a switch or a router, for example via a network adapter.

[0065] As shown in the figure, computing device 1200 can be implemented in a plurality of different forms. For example, computing device 1200 can be implemented as a single standard server 1220, or can be implemented multiple times within a group of such servers. Computing device 1200 can also be implemented as part of a rack server system 1224. In addition, computing device 1200 can be implemented within a personal computer such as a laptop computer 1222. Alternatively, the components from computing device 1200 can be combined with other components within a mobile device (not shown) such as device 1250. Such devices can each include one or more of computing devices 1200, 1250, and the entire system can be composed of a plurality of computing devices 1200, 1250 that communicate with each other.

[0066] Among other components, computing device 1250 includes a processor 1252, a memory 1264, input / output devices such as a display 1254, a communication interface 1266, and a transceiver 1268. Device 1250 can also include a storage device such as a microdrive or other device to provide additional storage. Components 1250, 1252, 1264, 1254, 1266, and 1268 are interconnected using various buses, and some of these components can be mounted on a common motherboard or in other suitable ways.

[0067] Processor 1252 can execute instructions within computing device 1250, including instructions stored in memory 1264. The processor can be implemented as a chipset consisting of chips including a plurality of separate analog and digital processors. Additionally, the processor can be implemented using any of a plurality of architectures. For example, processor 1252 can be a CISC (Complex Instruction Set Computer) processor, a RISC (Reduced Instruction Set Computer) processor, or a MISC (Minimal Instruction Set Computer) processor. The processor can provide coordination of other components of device 1250, such as, for example, control of the user interface, applications executed by device 1250, and wireless communication by device 1250.

[0068] Processor 1252 can communicate with a user via control interface 1258 and display interface 1256 coupled to display 1254. Display 1254 can be, for example, a TFT (Thin Film Transistor Liquid Crystal Display) display or an OLED (Organic Light Emitting Diode) display, or other suitable display technology. Display interface 1256 can comprise suitable circuitry for driving display 1254 to present graphical and other information to the user. Control interface 1258 can receive commands from the user and convert them for transmission to processor 1252. Additionally, external interface 1262 can be provided to communicate with processor 1252 to enable proximity area communication between device 1250 and other devices. External interface 1262 can provide, for example, wired communication in some implementations, or wireless communication in other implementations, and can also use multiple interfaces.

[0069] Memory 1264 stores information within computing device 1250. Memory 1264 can be implemented as one or more of one or more computer-readable media, one or more volatile memory units, or one or more non-volatile memory units. Also, extended memory 1274 can be provided and connected to device 1250 via expansion interface 1272, and expansion interface 1272 can include, for example, a SIMM (Single In-line Memory Module) card interface. Such extended memory 1274 can provide additional storage space for device 1250, or can also store applications or other information for device 1250. Specifically, extended memory 1274 can include instructions for implementing or supplementing the processes described above, and can also include secure information. Thus, for example, extended memory 1274 can be provided as a security module for device 1250 and can be programmed with instructions that enable secure use of device 1250. In addition, a secure application can be provided along with additional information via the SIMM card, such as by placing identification information on the SIMM card so that it cannot be hacked.

[0070] The memory can include, for example, flash memory and / or NVRAM memory as discussed below. In one implementation, a computer program product is tangibly implemented in an information carrier. The computer program product includes instructions that, when executed, implement one or more of the methods as described above. The information carrier is a computer or machine-readable medium such as, for example, memory 1264, extended memory 1274, or memory on processor 1252 that can be received via transceiver 1268 or external interface 1262.

[0071] Device 1250 can communicate wirelessly via communication interface 1266, which can include digital signal processing circuitry if necessary. Communication interface 1266 can provide communication in various modes or protocols such as, among others, GSM (registered trademark) voice calls, SMS, EMS, or MMS messaging, CDMA, TDMA, PDC, WCDMA (registered trademark), CDMA2000, or GPRS. Such communication can be performed, for example, via radio transceiver 1268. In addition, short - range communication can be performed using Bluetooth (registered trademark), WiFi, or other such transceivers (not shown). In addition, a GPS (Global Positioning System) receiver module 1270 can provide additional navigation and location - related wireless data to device 1250, and this data can be used as appropriate by applications running on device 1250.

[0072] Device 1250 can also communicate audibly using voice codec 1260, which can receive vocalization information from a user and convert it into usable digital information. Voice codec 1260 can similarly generate audible sounds for the user, for example, via a speaker within the handset of device 1250. Such sounds can include sounds from voice calls, can include recorded sounds (such as voice messages, music files, etc.), and can also include sounds generated by applications operating on device 1250.

[0073] Computing device 1250 can be implemented in a plurality of different forms as shown in the figure. For example, computing device 1250 can be implemented as a mobile phone 1280. Computing device 1250 can also be implemented as part of a smartphone 1282, a personal digital assistant, or other similar mobile device.

[0074] The various implementations of the systems and techniques described in this specification can be realized in digital electronic circuits, integrated circuits, specially designed ASICs (application-specific integrated circuits), computer hardware, firmware, software, and / or combinations thereof. These various implementations can include implementations in one or more computer programs that are executable and / or interpretable on a programmable system, the programmable system including at least one programmable processor, which can be special-purpose or general-purpose, coupled to receive data and instructions from, and to transmit data and instructions to, a memory system, at least one input device, and at least one output device.

[0075] These computer programs (also known as programs, software, software applications, or code) include machine instructions for a programmable processor and can be implemented in high-level procedural and / or object-oriented programming languages, and / or in assembly / machine language. As used herein, the terms "machine-readable medium" and "computer-readable medium" refer to any computer program product, apparatus, and / or device (e.g., magnetic disks, optical disks, memory, programmable logic device (PLD)) used to provide machine instructions and / or data to a programmable processor, including a machine-readable medium that receives machine instructions as a machine-readable signal. The term "machine-readable signal" refers to any signal used to provide machine instructions and / or data to a programmable processor.

[0076] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user, and a keyboard and a pointing device (e.g., a mouse or trackball) by which the user can provide input to the computer. Other types of devices can be used as well to provide interaction with the user. For example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback), and the input from the user can be received in any form including acoustic, speech, or tactile input.

[0077] The systems and techniques described herein can be implemented within a computing system that includes back-end components (e.g., as a data server), or middleware components (e.g., an application server), or front-end components (e.g., a client computer having a graphical user interface or a web browser by which a user can interact with an implementation of the systems and techniques described herein), or any combination of such back-end, middleware, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include local area networks ("LANs"), wide area networks ("WANs"), peer-to-peer networks (having ad hoc or static members), grid computing infrastructures, and the Internet.

[0078] A computing system can include clients and servers. The clients and servers are generally remote from each other and typically interact via a communication network. The relationship between a client and a server results from computer programs being executed on respective computers and having a client-server relationship with each other.

[0079] [Exemplary Implementation] <Overview> This example belongs to a study based on artificial intelligence ("AI") developed by the techniques described herein to detect patients suffering from asymptomatic left ventricular dysfunction (ALVD) with high fidelity using a non-invasive 10-second digital ECG.

[0080] Background ALVD exists in 2 - 9% of the general population, is associated with a decline in quality of life and lifespan, and is generally treatable once detected. The area under the curve (AUC) for the BNP screening blood test is 0.79 - 0.89. In this study, a hypothesis was made that the use of artificial intelligence (AI) would make it possible to identify left ventricular systolic dysfunction with an electrocardiogram (ECG), which is a ubiquitous and inexpensive test.

[0081] Method This study involved training a convolutional neural network to identify patients suffering from ventricular dysfunction defined as an ejection fraction (EF) ≤ 35% using pairs of digital 12-lead ECGs and echocardiograms stored from 44,959 patients in the Mayo Clinic Data Vault. This network was then tested on 52,870 patients secured for external validation.

[0082] Results Of the 52,870 patients tested, 4,131 (7.8%) had an EF ≤ 35%. The AUC of the ROC was 0.93. The sensitivity, specificity, and accuracy were 86.3%, 85.7%, and 85.7%, respectively. Of the 1,335 patients with abnormal AI screening but normal EF (false positives), 147 (11%) had at least one abnormal EF in the future (5-year incidence 9.5%). The fact that the risk of developing a low LVEF in the future thus increases fourfold suggests that the network may be detecting early, asymptomatic, metabolic, or structural abnormalities that appear on the ECG.

[0083] Conclusion By applying artificial intelligence to the ubiquitously and typically low-cost test of the ECG, the ECG can serve as a powerful tool for screening left ventricular dysfunction and further identifying individuals at high risk of its future manifestation.

[0084] <Method> Data Source and Study Population After approval by the institutional review board, the study obtained data from the MAYO CLINIC Digital Data Vault. 163,892 adult patients (18 years or older) were identified by at least one digital standard 10-second 12-lead ECG obtained in the supine position from January 1994 to February 2017, and at least one transthoracic echocardiogram (TTE) obtained within 14 days of the index ECG (Figure 13). For patients with multiple ECG and TTE data sets meeting these criteria, the earliest pair was used for network creation, validation, or testing, and later TTE data was used for follow-up analysis. A pre-conceptual proof-of-assessment was performed, and an internal investigation was obtained using 2,200 ECG-TTE data pairs excluded from this analysis, leaving a cohort of 97,829 patients, and the first ECG-TEE pair data set of these patients was used for the primary analysis.

[0085] ECG was acquired at a sampling rate of 500 Hz using a General Electric - market ECG machine (Wisconsin market) and stored using the MUSE (Medical Use of Standard Electrocardiogram) data management system for later retrieval. Comprehensive two - dimensional or three - dimensional Doppler echocardiography was available for all patients. Quantitative data at acquisition were recorded within a custom database (Echocardiogram Image Management System, EIMS) developed by the MAYO CLINIC. Left ventricular ejection fraction (EF) was routinely measured or estimated using a standardized method, and in most reports, more than one value could be recorded. For the purposes of this study, the ejection fraction value used in the model was the first value available from the standard hierarchical sequence, and EF was determined using three - dimensional echocardiography (Yamani H, Cai Q, Ahmad M, Three - dimensional echocardiography in evaluation of left ventricular indices. Echocardiography 2012;29:66 - 75), a two - plane approach using the Simpson's method, two - dimensional methods (Quinones MA, Waggoner AD, Reduto LA et al., A new, simplified and accurate method for determining ejection fraction with two - dimensional echocardiography. Circulation 1981;64:744 - 53), M - mode, and in the absence of any of the above, the reported visually estimated EF. When the estimate was a range, the mid - point was used as a single EF value. Left ventricular EF was classified as low (≤35%), mildly reduced (35 - 49%), or normal (≥50%).

[0086] Primary and secondary outcomes The primary outcome was the ability of the AI network to identify patients with an LVEF of 35% or less using only the ECG signal. This value was selected because of its clear clinical and therapeutic importance (Russo AM, Stainback RF, Bailey SR, et al. ACCF / HRS / AHA / ASE / HFSA / SCAI / SCCT / SCMR 2013 appropriate use criteria for implantable cardioverter-defibrillators and cardiac resynchronization therapy: a report of the American College of Cardiology Foundation appropriate use criteria task force, Heart Rhythm Society, American Heart Association, American Society of Echocardiography, Heart Failure Society of America, Society for Cardiovascular Angiography and Interventions, Society of Cardiovascular Computed Tomography, and Society for Cardiovascular Magnetic Resonance. J Am Coll Cardiol 2013;61:1318-68). The secondary outcome was the ability of the AI network to identify individuals who had a normal EF at the time of screening but had an increased risk of subsequent low EF during follow-up.

[0087] Overview of AI model development This study involved the development of a Convolutional Neural Network (CNN) using the Keras framework along with the backend of TensorFlow (registered trademark) (Google, Mountain View, California) and Python (van Rossum G. Python tutorial, Technical Report CS-R9526. Amsterdam 1995 May). The CNN applied to images (or videos) operates in such a way that it can extract very minute patterns within the dataset using convolutions. Each 12-lead ECG was considered as an “image” of 12×5000 (i.e., 12 leads sampled at 500 Hz for a duration of 10 seconds) (van Rossum G. Python tutorial, Technical Report CS-R9526. Amsterdam 1995 May). The network was composed of N single-lead convolutional layers, followed in each convolutional layer by a non-linear “Relu” activation function, a batch normalization layer (Ioffe S, Szegedy C, Batch normalization: accelerating deep network training by reducing internal covariate shift International Conference on Machine Learning 2015), and a max-pooling layer (Nagi J, Ducatelle F, Di Caro G et al., Max-pooling convolutional neural networks for vision-based hand gesture recognition. 2011 IEEE International Conference 2011:342-7). The features extracted from each raw digital signal ECG lead were fused within another convolutional layer that had simultaneous access to all leads.Following the last convolutional layer, the data was fed into a fully-connected network with two hidden layers including a dropout layer to avoid overfitting and an output layer activated using the "Softmax" function (Cristianini N, Shawe-Taylor J, New York, NY: Cambridge University Press; 1999).

[0088] AI Model Training Due to the large size of the inventors' dataset, approximately 50% of the dataset was used for network training. This allowed the inventors to obtain a very large dataset for testing the network and better evaluating its robustness (Figure 13). After the initial split into development and test (holdout) datasets, the development dataset was further split into training data (80% of the development set) and internal validation data (20%).

[0089] For training, ECGs were fed into the network, and the Adam optimizer (Kingma DP, Ba J. Adam: A Method for Stochastic Optimization 2014) was used to update the network weights with binary cross-entropy as the loss function. After each epoch, the network was tested using the internal validation dataset, and training was stopped after optimization. Network hyperparameters were also adjusted during this process, and the network with the lowest loss value was selected. All ECGs were passed through a low-pass filter (100 hz) to remove high-frequency noise and quantization errors.

[0090] Primary Result - AI Expanded ECGs to Identify Low EF After selecting the optimal network using the validation data, the same validation set was used to create a Receiver Operating Characteristic (ROC) curve and the area under the curve (AUC) was measured as a primary assessment of network strength. Two thresholds for the probability of having a low LVEF were selected using the ROC of the validation dataset. The first threshold was selected by giving equal weights to sensitivity and specificity, and the second threshold was selected to give 90% sensitivity in the validation dataset. The CNN model was then used on the test data to test its ability to predict low LVEF. The two thresholds were used to calculate sensitivity, specificity, and accuracy in test data not used for model training or threshold selection. The influence of age and gender on the network prediction function was also evaluated by creating separate networks with those variables as inputs and training the network to determine whether age and gender could be determined from the ECG alone.

[0091] Secondary results - AI extended the ECG to predict future low EF At the initial stage of the course of any disease affecting EF, a hypothesis was put forward that the ECG signal would show a subtle abnormal pattern due to metabolic and structural disturbances that did not affect a sufficient amount of myocardium to cause a decrease in EF. Furthermore, a hypothesis was put forward that CNN would classify some of these cases, which initially appear as false positive tests but may become true positive tests over time, as abnormal. To test this hypothesis, a sub-study was designed to identify patients who met the following conditions: 1) the network predicted that the patient had a low EF, 2) an echocardiogram was performed on the patient within 14 days, demonstrating a normal EF (≧50%) indicating a false positive finding by the algorithm, and 3) at least one additional echocardiogram (not used in training or testing) was available for the patient at a later date. True negative cases (where both the algorithm and the clinical determination agreed that the patient did not have a low EF) were used to create a control group. Subjects were selected for the control group to have an LVEF≧50%. Kaplan-Meier analysis was used to show the incidence of low EF over time for true negatives vs. false positives. Next, Cox proportional hazards regression was used to estimate the hazard for low EF after adjustment for age and gender. In addition, a sensitivity analysis was performed by classifying the predicted probability of future low EF expression to determine if there was a monotonic pattern.

[0092] Statistical Considerations For the measurement of diagnostic performance (AUC ROC, sensitivity), the sample size is large enough such that normal confidence intervals (CIs) are expected to have a width of less than 0.5%. Therefore, these CIs are not reported with the estimates as the precision is high. Continuous data are presented as mean + / − SD. Two-sided p-values are presented for the Cox model. Survival analysis was performed using SAS version 9.4. CNN was trained using KERAS (version 2.0.3) and TENSORFLOW (version 1.0.1).

[0093] Research Support The research was conceived, funded, and executed entirely by the MAYO CLINIC. There was no industrial support of any kind.

[0094] <Results> Study Population A total of 625,326 patients who received an ECG-TTE pair were screened, and the study cohort selected for analysis was identified (Figure 13). The first ECG-TTE data pair from patients who had an ECG and echocardiogram performed at intervals within 2 weeks constituted an analysis dataset of 97,829 patients, including 35,970 in the training set, 8,989 in the validation set, and 52,870 in the holdout test set. No patients were in more than one group (Figure 13). The mean age of the overall patient population was 61.8 ± 16.5 years, and 7.8% of the population had an EF ≤ 35%. Table 1 shows patient characteristics for the training, validation, and test sets. In the test dataset, 4,131 patients (7.8%) had an EF of 35% or less, 6,740 patients (12.7%) had an LVEF higher than 35% and lower than 50%, and 41,999 patients (79.5%) had an LVEF of 50% or higher. More than 89% of the TTEs were performed within 1 day of the index ECG.

[0095] Primary Results In Figure 14, the AUC of the holdout test dataset was 0.93, identical to the AUC of the internal validation dataset (0.93). When selecting a threshold without prioritizing sensitivity, the overall accuracy was 85.7%, specificity was 85.7%, sensitivity was 86.3%, and negative predictive value was 98.7%. Using the threshold to give 90% sensitivity in the validation set and applying the algorithm to the test dataset, the sensitivity was 89.1%, specificity was 83%, overall accuracy was 83.5%, and negative predictive value was 98.9%. When patients with no known comorbidities (see Figure 16) were analyzed separately by the network, the AUC increased to 0.98, with sensitivity of 95.6%, specificity of 92.4%, negative predictive value of 99.8%, and accuracy of 92.5%.

[0096] Network performance was not improved by additional input of age and gender. To understand whether the network was "determining" age and gender based on ECG only, the network was retrained to predict age and gender from the ECG. Age was predicted to be 8.7 ± 6 years, and gender was accurately determined at 87% when the AUC was 0.94 from the ECG. This indicates that the network was able to determine age and gender with high confidence from the ECG alone, and since the network should have been able to explain these variables, it explains why its performance was not affected by age or gender.

[0097] Distribution of EF values by AI algorithm classification When selecting a threshold without preference for sensitivity (i.e., a threshold that gives equal sensitivity and specificity using validation data), 10,544 (19.9%) of the patients in the test set were identified by the network as having a low EF. Of these 10,544 patients, 33.8% had an EF of 35% or less, 29.5% had an EF of 36 - 50%, and 36.6% had a normal EF. Within the group identified by the network as normal, 1.3% had an EF of 35% or less, 8.6% had an LVEF of 36 - 50%, and the remainder (90.1%) had a normal LVEF (Figure 17).

[0098] Secondary outcome - AI extended the ECG to predict future low EF Among the patients identified by the network as having a normal EF and also having a confirmatory simultaneous normal EF (''true negative'') by echocardiogram, 11,515 patients underwent follow-up echocardiogram. Among these true negative patients, 302 developed a low EF during a median (IQR) follow-up of 3.8 (1.4 - 7.5) years (Figure 4: 10-year incidence of 4.4%). In contrast, 1,335 patients were shown by the network to have a low EF, but the simultaneous echocardiogram demonstrated a normal EF (''false positive''). Among these 1,335 patients with an initial ''false positive'' result, 147 (Figure 4: 10-year incidence of 20.8%) developed left ventricular dysfunction during a median follow-up of 3.4 (IQR 1.2 - 6.8) years. This represents a four-fold increased risk of future low EF when the AI algorithm defined the ECG as abnormal (age and gender adjusted, HR = 4.1 [3.3 - 5.0], p < 0.001), suggesting that the network identified ECG abnormalities before the appearance of overt ventricular dysfunction.

[0099] <Discussion> Left ventricular systolic dysfunction is associated with a decline in quality of life, increased morbidity, and increased mortality (McDonagh TA, McDonald K, Maisel AS, Screening for asymptomatic left ventricular dysfunction using B-type natriuretic Peptide. Congest Heart Fail 2008;14:5-8). Major cardiovascular professional organizations have endorsed treatment based on evidence that improves detected symptoms and survival (Al-Khatib SM, Stevenson WG, Ackerman MJ et al., 2017 AHA / ACC / HRS Guideline for Management of Patients With Ventricular Arrhythmias and the Prevention of Sudden Cardiac Death: Executive Summary: A Report of the American College of Cardiology / American Heart Association Task Force on Clinical Practice Guidelines and the Heart Rhythm Society. Circulation 2017, Yancy CW, Jessup M, Bozkurt B et al., 2013 ACCF / AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation / American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2013;62:e147-239).However, there is a lack of effective screening for ventricular dysfunction (McDonagh TA, McDonald K, Maisel AS, Screening for asymptomatic left ventricular dysfunction using B-type natriuretic Peptide. Congest Heart Fail 2008;14:5-8, Redfield MM, Rodeheffer RJ, Jacobsen SJ, Mahoney DW, Bailey KR, Burnett JC, Jr. Plasma brain natriuretic peptide to detect preclinical ventricular systolic or diastolic dysfunction: a community-based study. Circulation 2004;109:3176-81). In this study, it was found that by applying artificial intelligence using a convolutional neural network to the standard 12-lead ECG, a common clinical test that is inexpensive and widely available, left ventricular dysfunction can be detected with an AUC of 0.93. The performance of this test is comparable to that of other common screening tests such as prostate-specific antigen for prostate cancer (0.92), mammography for breast cancer (0.85), and cervical cytology for cervical cancer (0.70). In contrast to BNP, the accuracy was not affected by age or gender. In addition to effectively identifying individuals with ventricular systolic dysfunction, it is important that the network also predicted those who initially had normal LV function but later developed low EF. Patients with normal EF but abnormal network screening ("false positives") had a four-fold increased risk of developing ventricular dysfunction over the next five years (a 10% risk in 5 years). This suggests that the network detected early, asymptomatic, metabolic, or structural abnormalities that appear on the ECG.

[0100] Congestive heart failure afflicts more than 5 million people in the United States alone and consumes more than $30 billion in annual healthcare costs (Heidenreich PA, Albert NM, Allen LA, et al., Forecasting the impact of heart failure in the United States: a policy statement from the American Heart Association. Circ Heart Fail 2013;6:606-19, Mozaffarian D, Benjamin EJ, Go AS, et al., Heart disease and stroke statistics--2015 update: a report from the American Heart Association. Circulation 2015;131:e29-322). Early detection and prevention are of paramount importance in healthcare management. Asymptomatic left ventricular dysfunction affects more than 7 million Americans and even more people worldwide. It was found that 6% of the patients in the inventors' population had a low EF, which is consistent with previous studies (McDonagh TA, McDonald K, Maisel AS, Screening for asymptomatic left ventricular dysfunction using B-type natriuretic Peptide. Congest Heart Fail 2008;14:5-8). BNP and NT-BNP have been proposed for the detection of left ventricular systolic dysfunction.Bhalla and collaborators evaluated BNP to screen for systolic and diastolic dysfunction, with an AUC of 0.60 for BNP and 0.70 for NTproBNP, and the results were found to be improved to 0.70 and 0.73 with the addition of impedance cardiography (Bhalla V, Isakson S, Bhalla MA, et al., Diagnostic ability of B-type natriuretic peptide and impedance cardiography: testing to identify left ventricular dysfunction in hypertensive patients. Am J Hypertens 2005;18:73S-81S). In a Mayo Clinic study from Olmsted County evaluating individuals 45 years of age and older, the AUC was found to be higher in more severely affected individuals (0.82 - 0.92) compared to other systolic dysfunction (0.51 - 0.74). Furthermore, the optimal discrimination level for BNP varied by age and gender. In contrast, in this study, excellent AI network performance was found to be invariant to age and gender. This was verified by creating a network that could determine the age and gender of patients within the range of 8 ± 6.3 years with 87% accuracy based on ECG alone.Previous literature has described that the ECG changes with age and gender (Daly C, Clemens F, Lopez Sendon JL, et al., Gender differences in the management and clinical outcome of stable angina. Circulation 2006;113:490-8, Khane RS, Surdi AD, Bhatkar RS, Changes in ECG pattern with advancing age. J Basic Clin Physiol Pharmacol 2011;22:97-101, Kuo TB, Lin T, Yang CC, Li CL, Chen CF, Chou P, Effect of aging on gender differences in neural control of heart rate. Am J Physiol 1999;277:H2233-9, Stramba-Badiale M, Locati EH, Martinelli A, Courville J, Schwartz PJ, Gender and the relationship between ventricular repolarization and cardiac cycle length during 24-h Holter recordings. Eur Heart J 1997;18:1000-6). During the training process, the network integrated and explained these age and gender characteristics so as not to affect its discriminatory ability to identify ventricular dysfunction. This ability is considered to be inherent in neural network selection.

[0101] The specific ECG features used by the convolutional neural network without a teacher to classify whether an individual has low EF or not are not known due to the nature of the neural network. However, by training the network on a large cohort of approximately 45,000 ECG and EF data pairs, as demonstrated by an AUC of 0.93 when tested on a population of 52,870 individuals, the network was exposed to a sufficient number of electrocardiogram variants to reliably classify individuals with low ejection fraction. In contrast to the conventional application of neural networks in medicine to mimic human skills, such as the identification of mammographic lesions (Salazar-Licea LA, Pedraza-Ortega JC, Pastrana-Palma A, Aceves-Fernandez MA, Location of mammograms ROI's and reduction of false-positive. Comput Methods Programs Biomed 2017;143:97-111), this study extends the use of AI and expands the capabilities beyond human skills.

[0102] An important feature of the inventors' network is the use of 12-lead ECG, an inexpensive and standard ubiquitous test, as its input. In many parts of the United States (Gruca TS, Pyo TH, Nelson GC, Providing Cardiology Care in Rural Areas Through Visiting Consultant Clinics. J Am Heart Assoc 2016;5) and in developing countries, access to cardiac care and imaging is limited.The availability of portable, inexpensive tests for ventricular systolic dysfunction enables optimal use of limited imaging resources and allows individuals to benefit from proven therapies such as beta-adrenergic blockers, angiotensin receptor blockers, and, when available, implantable devices (defibrillators and cardiac resynchronization systems) (Yancy CW, Jessup M, Bozkurt B, et al., 2013 ACCF / AHA guideline for the management of heart failure: a report of the American College of Cardiology Foundation / American Heart Association Task Force on Practice Guidelines. J Am Coll Cardiol 2013;62:e147-239, Yancy CW, Jessup M, Bozkurt B, et al., 2017 ACC / AHA / HFSA Focused Update of the 2013 ACCF / AHA Guideline for the Management of Heart Failure: A Report of the American College of Cardiology / American Heart Association Task Force on Clinical Practice Guidelines and the Heart Failure Society of America. J Am Coll Cardiol 2017;70:776-803).With the emergence of smartphone-compatible electrodes (Yasin OZ, Attia Z, Dillon JJ, et al., Noninvasive blood potassium measurement using signal-processed, single-lead ecg acquired from a handheld smartphone. J Electrocardiol 2017;50:620-5), mobile applications can enable their use in resource-limited areas. Also, due to the software-based nature of the "sample" of the tests on the network conducted in this study, continuous feedback and improvement are possible, and system improvements are rapidly distributed.

[0103] In this study, based on well-established results and therapeutic implications of this value, an EF of 35% or less was selected as the detection threshold (Yancy CW, Jessup M, Bozkurt B, et al., 2017 ACC / AHA / HFSA Focused Update of the 2013 ACCF / AHA Guideline for the Management of Heart Failure: A Report of the American College of Cardiology / American Heart Association Task Force on Clinical Practice Guidelines and the Heart Failure Society of America. J Am Coll Cardiol 2017;70:776-803). However, the identification of an EF below 50% is also clinically significant. Therefore, a "false positive" value of 45% is actually clinically meaningful (Figure 15). In addition, the network can be easily configured to use a different detection threshold, for example, the identification of EF < 40% instead. The ECG-TTE pairs were not acquired simultaneously. However, this study included over 100,000 ECG-TTE pair datasets, and since over 89% of the TTEs were performed within 24 hours of the ECG, the potential for errors related to time delay is small. The convolutional neural network in this study was specifically trained to detect low EF, i.e., diastolic dysfunction, and does not detect heart failure. However, there are established evidence-based therapies that reduce morbidity and mortality for the treatment of low EF, which may be asymptomatic and thus its detection is a top priority.

[0104] Although multiple implementations have been described in detail above, other modifications are possible. Additionally, the logical flow shown in the figures does not require the specific order or sequence shown to achieve the desired result. Other steps can be provided, or steps can be omitted from the described flow, other components can be added to the described system, or removed from the described system. Accordingly, other implementations are also within the scope of the following claims.

Claims

1. receiving, by a system comprising one or more computers, electrocardiogram (ECG) data describing a subject's ECG over a period of time; processing the ECG data to generate predicted inputs; providing the prediction input to a survival prediction model generated by processing a plurality of training data to predict a survival prediction for the subject; outputting a prediction of survival estimate for said subject; Including, the survival estimate is the future survival rate of the subject; The estimated ejection fraction characteristic is mapped to the future survival rate for the subject. Computer-implemented method.

2. The computer-implemented method of claim 1 , wherein the survival estimation model is a regression model.

3. The computer-implemented method of claim 1 , wherein the survival prediction model is a machine learning model.

4. The computer-implemented method of claim 2 , wherein the survival prediction model is a neural network.

5. 5. The computer-implemented method of claim 4, wherein the neural network is a feedforward neural network, a convolutional neural network, or a recurrent neural network.

6. The computer-implemented method of claim 1 , wherein the prediction input is a value of one or more morphological features of the ECG.

7. 7. The computer-implemented method of claim 6, wherein the one or more morphological features include at least one of T wave amplitude, P wave amplitude, P wave area, T wave area, left T wave slope, right T wave slope, left P wave slope, right P wave slope, T wave duration, P wave duration, PR interval, QRS duration, QRS amplitude, QRS area, QRS energy, QRS peak-to-peak ratio, or QT segment length.

Images