Electrocardiogram-based deep learning for cardiac prediction

Abstract

Electrocardiogram-based deep learning for cardiac prediction is described. An electrocardiogram analysis module may include a data preprocessor configured to normalize an electrocardiogram to generate a standardized input for an electrocardiogram-based cardiac prediction. The electrocardiogram analysis module may further include a deep learning model including a neural network and at least one dense layer, the deep learning model trained to identify features associated with a cardiac condition that reflect underlying cardiac changes that result from or predispose development of the cardiac condition, and generate a cardiac prediction based on the identified features.

Description

Electrocardiogram-Based Deep Learning for Cardiac PredictionRELATED APPLICATION

[0001] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 716.516. filed November 5, 2024. entitled “Electrocardiogram-Based Deep Learning for Heart Failure Prediction,” the entire disclosure of which is hereby incorporated by reference herein in its entirety.STATEMENT REGARDING GOVERNMENT SUPPORT

[0002] This invention was made with government support under Grant Nos. HL134893, HL140224, HL153669, HL092577, HL105780, and HL169839 awarded by the National Institutes of Health. The government has certain rights in the invention.BACKGROUND

[0003] Heart failure is a complex cardiovascular condition characterized by the inability of the heart to pump blood effectively. Heart failure affects millions of people worldwide and is associated with significant morbidity, mortality, and healthcare costs. Left ventricular ejection fraction (LVEF). which represents the percentage of blood pumped out of the left ventricle with each heartbeat, is a measure of cardiac function that is closely associated with heart failure. Accordingly, LVEF may serve as an indicator of cardiovascular health. Early detection and risk stratification of heart failure and left ventricular function may be beneficial for implementing timely interventions and potentially improving patient outcomes.

[0004] Current methods for predicting heart failure risk and assessing left ventricular function ty pically involve a combination of clinical factors, laboratory tests, and imaging studies. These approaches often utilize multiple inputs, which may include demographics, comorbidities, biomarkers, and echocardiographic parameters. While these methods have shown utility in research settings, their implementation in routine clinical practice can present challenges due to the extensive data collection and potential variability in risk factor assessment.

[0005] Electrocardiography is a widely available, non-invasive, and relatively inexpensive diagnostic tool used in various healthcare settings. An electrocardiogram (ECG) records the electrical activity of the heart over time, typically using 12 leads placed on the body surface to capture electrical signals from different angles, providing a comprehensive view' of cardiacFIG. 1 Patents 1 Docket No. : BI-11127-PCTelectrical activity. Traditional ECG interpretation focuses on identifying specific abnormalities or patterns associated with cardiac conditions. However, the potential of the ECG to provide comprehensive cardiovascular assessment for future cardiovascular events, particularly heart failure, and current left ventricular function may not be fully realized.SUMMARY

[0006] Electrocardiogram-based deep learning for cardiac prediction is described. An electrocardiogram analysis module may include a data preprocessor configured to normalize an electrocardiogram to generate a standardized input for electrocardiogram-based cardiac prediction. The electrocardiogram analysis module may further include a deep learning model including a neural network and at least one dense layer, the deep learning model trained to identify, via the neural network, features associated with a cardiac condition that reflect underlying cardiac changes that result from or predispose development of the cardiac condition, and generate, via the at least one dense layer, a cardiac prediction based on the identified features.

[0007] This Summary introduces a selection of concepts in a simplified form that are further described below in the Detailed Description. As such, this Summary is not intended to identify essential features of the claimed subject matter, nor is it intended to be used as an aid in determining the scope of the claimed subject matter.BRIEF DESCRIPTION OF THE DRAWINGS

[0008] The detailed description is described with reference to the accompanying figures.

[0009] FIG. 1 is an illustration of an environment in an example implementation that is operable to employ electrocardiogram-based deep learning for cardiac prediction as described herein.

[0010] FIG. 2 depicts an example training process that may be used in generating the at least one deep learning model of the electrocardiogram (ECG) analysis module of FIG. 1.

[0011] FIG. 3 illustrates an example implementation of a convolutional block structure that may be used in the convolutional layers of the convolutional neural network (CNN) of FIG. 2.

[0012] FIG. 4 depicts an example implementation of using the at least one deep learning model to assess electrocardiogram data for heart failure prediction.

[0013] FIG. 5 depicts an example implementation of using the at least one deep learning model to assess electrocardiogram data for left ventricular ejection fraction prediction.FIG. 1 Patents 2 Docket No. : BI-11127-PCT

[0014] FIG. 6 depicts an example procedure for training and validating a machine learning model to output a heart failure risk prediction according to one or more implementations.

[0015] FIG. 7 depicts an example procedure for generating a heart failure risk prediction from electrocardiogram data using a machine learning model according to one or more implementations.

[0016] FIG. 8 illustrates an example system including various components of an example device that can be implemented as any type of computing device as described and / or utilized with reference to FIGS. 1-7 to implement the techniques described herein.

[0017] FIGS. 9A and 9B illustrate a summary of the training data used for developing the deep learning model for heart failure prediction.

[0018] FIG. 10 shows an example analysis of deep learning model performance on auxiliary tasks of age regression and sex classification.

[0019] FIG. 11 depicts a set of graphs comparing a performance of different heart failure prediction models across three datasets.

[0020] FIG. 12 depicts a set of graphs showing cumulative risk of heart failure over time for different test groups.

[0021] FIG. 13 depicts a set of graphs showing a cumulative risk of heart failure events over time for three different test groups.

[0022] FIG. 14 depicts a set of graphs comparing the distribution of predicted heart failure risk between tw o models, ECG2HF and PCP-HF.

[0023] FIG. 15 depicts a set of graphs show ing the calibration of heart failure risk predictions for three different test sets.

[0024] FIG. 16 depicts a set of graphs comparing the performance of electrocardiogram-based deep learning models for heart failure prediction.

[0025] FIG. 17 depicts a set of ECG graphs showing median w aveforms for multiple leads of electrocardiogram data.

[0026] FIG. 18 depicts a set of bar graphs comparing the area under the receiver operating characteristic curve (AUROC) for heart failure prediction across different comorbidity statuses and datasets.

[0027] FIG. 19 depicts a set of bar graphs comparing average precision for heart failure prediction across different comorbidity statuses and datasets.

[0028] FIG. 20 depicts an example of cumulative risk curves for heart failure events over time across three test datasets among individuals without common cardiac comorbidities.FIG. 1 Patents 3 Docket No. : BI-11127-PCT

[0029] FIG. 21 depicts a set of bar graphs comparing the area under the receiver operating characteristic curve (AUROC) for heart failure prediction across different age groups and datasets.

[0030] FIG. 22 depicts a set of bar graphs comparing average precision for heart failure prediction across different age groups and datasets.

[0031] FIG. 23 depicts a set of bar graphs comparing the area under the receiver operating characteristic curve (AUROC) for heart failure prediction across different ECG types and datasets.

[0032] FIG. 24 depicts a set of bar graphs comparing average precision for heart failure prediction across different ECG types and datasets.

[0033] FIG. 25 depicts a set of graphs comparing the area under the receiver operating characteristic curve (AUROC) of electrocardiogram-based deep learning for heart failure prediction across different test sets and QRS duration categories.

[0034] FIG. 26 depicts a set of graphs comparing the average precision of electrocardiogrambased deep learning for heart failure prediction across different test sets and QRS duration categories.

[0035] FIG. 27 depicts a set of bar graphs comparing the area under the receiver operating characteristic curve (AUROC) for heart failure prediction across different racial groups and datasets.

[0036] FIG. 28 depicts a set of bar graphs comparing average precision for heart failure prediction across different racial groups and datasets.

[0037] FIG. 29 depicts a set of bar graphs comparing model performance metrics for heart failure prediction across different pacemaker status categories.

[0038] FIG. 30 depicts a set of bar graphs comparing model performance metrics for heart failure prediction across different atrial fibrillation status categories.DETAILED DESCRIPTIONOverview

[0039] As mentioned above, heart failure (HF) is a complex cardiovascular condition characterized by the inability of the heart to pump blood effectively. As used herein, the term “heart failure” is meant to include any condition of the heart that negatively affects or has a reasonable potential to negatively affect the abi li ty of the heart to pump blood. By way of example, heart failure may be defined as a clinical syndrome resulting from structural or functional impairment of ventricular filling or ejection of blood. Left ventricular ejectionFIG. 1 Patents 4 Docket No. : BI-11127-PCTfraction (LVEF), which represents the percentage of blood pumped out of the left ventricle with each heartbeat, may indicate a type or severity of HF. A low LVEF (e.g., less than 50%), for instance, may indicate a higher risk of HF. Early detection and risk stratification of HF and / or an assessment of left ventricular function may be beneficial for implementing timely interventions and potentially improving patient outcomes. However, diagnosing HF can be challenging due to its complex nature and the non-specific symptoms it may present. HF can manifest through various symptoms such as shortness of breath, fatigue, and fluid retention, which can overlap with other conditions. Additionally, HF can develop gradually, making early detection difficult without advanced diagnostic tools. The heterogeneity of HF presentations and the reliance on a comprehensive assessment of cardiac function may further complicate the diagnostic process. LVEF is typically assessed using imaging modalities such as echocardiography, but this assessment can be challenging due to factors such as technical expertise requirements, equipment availability7, cost, and variability' in measurements.

[0040] Using ECGs, clinicians may diagnose and / or monitor a wide range of cardiovascular disease (CVD) conditions, including HF. ECG measures have been demonstrated to predict clinical outcomes in both healthy community-based samples and among patients with CVD. As one example, the left heart, comprising the left atrium and left ventricle, pumps oxygenated blood to the tissues of the body. Changes in the electrical activity' of the left heart may be assessed as a part of diagnosing various cardiovascular conditions, such as HF, and predicting patient outcomes. By way of example, this assessment may be performed by trained clinicians (e.g., cardiologists) manually interpreting ECG tracings. Standard ECG measures, including QRS duration, ST-segment changes, T-wave abnormalities, and left ventricular hypertrophy criteria, have all been shown to be associated with incident disease in healthy participants and clinical outcomes (including all-cause mortality, cardiovascular death, and risk of hospitalization) in disease states like HF. However, this process can be time-consuming and subject to interobserver variability. Moreover, subtle ECG changes that may indicate increased HF risk may not be readily apparent to human interpreters.

[0041] As another example, right ventricular abnormalities are known prognostic markers across a range of cardiovascular diseases, including atrial fibrillation (AF) and HF. While ECG can provide insights into right ventricular function, the interpretation of these findings often relies on significant expertise. As a result, detailed assessment of right ventricular function from ECG is time-consuming, relies on significant expertise, and may be limited by interobserver variability’. Given the limited availability of advanced cardiac imaging modalities, there is a need for a rapid and accurate method of assessing cardiac structure andFIG. 1 Patents 5 Docket No. : BI-11127-PCTfunction from ECG data in order to facilitate both ECG interpretation workflows and the use of ECG measures to stratify cardiovascular risk, particularly for HF and left ventricular function assessment.

[0042] To overcome these issues, electrocardiogram-based deep learning for cardiac prediction is disclosed herein. In accordance with the described techniques, one or more deep learning models are used to process ECG data and generate a cardiac prediction, such as a heart failure risk prediction and / or a LVEF prediction. The one or more deep learning models are trained on vast quantities of ECG data in order to identify subtle patterns in the ECG waveform that may indicate current heart failure, an increased risk of future heart failure, and / or left ventricular function. Moreover, the techniques described herein provide a model architecture that may be adapted to output one or more auxiliary predictions, non-limiting examples of which include an all-cause mortality prediction, an age prediction, a sex prediction, and a diagnostic code prediction.

[0043] By way of example, the techniques described herein enable generation of at least one machine learning model (e.g., at least one deep learning model) that is able to output an accurate cardiac prediction from a single ECG. By leveraging the full ECG waveform data, including information that may not be typically used or interpreted during manual clinical analysis, the at least one machine learning model “learns” how to interpret latent information from ECGs that may not be readily interpretable by human observers. By way of example, via the training process described herein, the at least one machine learning model may leam to identify complex patterns that go beyond what is measurable by a human observer. For instance, the at least one machine learning model may include a convolutional neural network (CNN) that, through training, leams to extract features of an input ECG that are relevant to a particular prediction task (or tasks) for which the at least one deep learning model is being trained (e.g., heart failure risk prediction or LVEF prediction). The at least one machine learning model thus leams ECG features associated with developing heart failure and / or LVEF. These features, which may be captured in ECG feature maps output by the CNN, represent a high-dimensional abstraction of the ECG data. By leveraging this information, the at least one machine learning model may leam to make accurate predictions about cardiac function, structure, and / or the risks of cardiac conditions such as HF that may not be evident from visual inspection or manual measurement performed by a human.

[0044] As used herein, “features associated with a cardiac prediction” may refer to patterns, characteristics, and / or signals within ECG data that correlate with or indicate the cardiac prediction (e.g., a risk of having and / or developing heart failure, and / or a LVEF prediction),FIG. 1 Patents 6 Docket No. : BI-11127-PCTincluding patterns, characteristics, and / or signals within the ECG data that are not interpretable by a human and / or using manual analysis workflows. The features associated with the cardiac prediction may include morphological changes in ECG waveforms, temporal patterns in cardiac electrical activity, voltage amplitude variations, interval measurements, and complex multi-dimensional patterns, just to name a few examples. The features associated with the cardiac prediction may encompass both explicit measurements (e.g.. QRS voltages, PR intervals, QT intervals, P-wave durations, and T-wave morphologies) as well as latent or implicit patterns that are learned and identified by the at least one machine learning model through the analysis of large training datasets. In some implementations, these features may reflect underlying cardiac structural and functional changes that result from heart failure and / or predispose an individual to developing heart failure.

[0045] The techniques described herein represent an advance in computer engineering and provide a substantial advancement over existing practices. The data acquired to prepare the machine learning models are technical data relating to ECG recordings. The methods and systems described herein are more consistent, accurate, and efficient than manual / human analysis, which is prone to bias and does not scale to the amount of ECG data that is generated today.

[0046] Moreover, the techniques described herein also enable non-expert providers to obtain clinically relevant information from an ECG. For instance, detailed ECG interpretation often relies on expert analysis, but such experts may be unavailable at the point of care, particularly in resource-limited or time-sensitive settings. As such, the machine learning models described herein may be used to identify individuals at high risk of heart failure as soon as the ECG is completed, enabling enhanced clinical decision support. This may also reduce specialist dependency, enabling specialists to devote more time to higher-level patient care decisions and less time on routine ECG interpretation. By providing a rapid and accurate method of obtaining a cardiac prediction regarding heart failure risk and / or left ventricular function from ECG data, these techniques may facilitate both ECG interpretation workflows and the use of ECG data to stratify cardiovascular risk. This approach represents an advancement in the technical fields of computer engineering and medical diagnostics, which may improve patient care and outcomes in the management of heart failure and related cardiovascular diseases.

[0047] In some aspects, the techniques described herein relate to a system for electrocardiogram-based cardiac prediction, including: an electrocardiogram analy sis module implemented in a transitory or non-transitory computer-readable storage medium, the electrocardiogram analysis module including: a data preprocessor configured to normalize anFIG. 1 Patents 7 Docket No. : BI-11127-PCTelectrocardiogram to generate a standardized input for the electrocardiogram-based cardiac prediction; and a deep learning model including a neural network and at least one dense layer, the deep learning model trained to: identify, via the neural network, features associated with a cardiac condition that reflect underlying cardiac changes that result from or predispose development of the cardiac condition; and generate, via the at least one dense layer, a cardiac prediction based on the identified features.

[0048] In some aspects, the techniques described herein relate to a system, wherein the neural network includes a convolutional neural network.

[0049] In some aspects, the techniques described herein relate to a system, wherein the convolutional neural network includes multiple convolutional blocks, each convolutional block including a plurality of convolutional layers, and wherein outputs of at least two convolutional layers of the plurality of convolutional layers are combined via a concatenation.

[0050] In some aspects, the techniques described herein relate to a system, wherein the cardiac condition is heart failure, and the cardiac prediction includes a probability score indicating a likelihood of developing heart failure within a period of time.

[0051] In some aspects, the techniques described herein relate to a system, wherein the cardiac prediction further includes a plurality of probability scores indicating respective likelihoods of developing heart failure for different time bins within the period of time.

[0052] In some aspects, the techniques described herein relate to a system, wherein the at least one dense layer is further trained to generate auxiliary predictions based on the identified features, the auxiliary predictions including at least one of an all-cause mortality prediction, an age prediction, a sex prediction, or a diagnostic code prediction.

[0053] In some aspects, the techniques described herein relate to a system, wherein the cardiac condition is a low left ventricular ejection fraction, and the cardiac prediction includes a left ventricular ejection fraction estimate.

[0054] In some aspects, the techniques described herein relate to a system, wherein the data preprocessor is further configured to upsample and zero-pad the electrocardiogram to generate the standardized input for the electrocardiogram-based cardiac prediction.

[0055] In some aspects, the techniques described herein relate to a system, wherein the standardized input includes full electrocardiogram waveform data as a time-series of voltage measurements for each lead of the electrocardiogram, wherein the time-series of voltage measurements are sampled at a standardized frequency over a predetermined time period.

[0056] In some aspects, the techniques described herein relate to a system, further including a training module configured to: train the deep learning model using a training sample includingFIG. 1 Patents 8 Docket No. : BI-11127-PCTa first portion of a first subset of electrocardiogram training data, the electrocardiogram training data including electrocardiograms obtained from healthy patients and from patients diagnosed with heart failure; and refine the trained deep learning model using a validation sample including a second portion of the first subset of the electrocardiogram training data.

[0057] In some aspects, the techniques described herein relate to a system, wherein at least a portion of ground truth labels for the training sample are generated by a transformer-based natural language processing (NPL) model trained to detect heart failure hospitalization using free text discharge summaries in medical records associated with the electrocardiogram training data.

[0058] In some aspects, the techniques described herein relate to a system, wherein the training module is further configured to internally validate the trained and refined deep learning model using an internal test sample including a third portion of the first subset of the electrocardiogram training data.

[0059] In some aspects, the techniques described herein relate to a system, wherein the training module is further configured to externally validate the deep learning model using an external test sample including a second subset of the electrocardiogram training data, wherein the first subset of the electrocardiogram training data and the second subset of the electrocardiogram training data include electrocardiogram recordings from different data sources.

[0060] In some aspects, the techniques described herein relate to a computer-implemented method for generating a cardiac prediction, said method including: generating, by a data preprocessor, a standardized input for an electrocardiogram that is to be processed by a deep learning model trained to output the cardiac prediction; extracting, by a neural network of the deep learning model, features of the standardized input; outputting, by the neural netw ork, an electrocardiogram feature map representing a high-dimensional abstraction of the features of the standardized input; and generating, by an output layer of the deep learning model, the cardiac prediction based at least in part on the electrocardiogram feature map, wherein the cardiac prediction includes a prediction associated with developing heart failure.

[0061] In some aspects, the techniques described herein relate to a computer-implemented method, wherein generating the standardized input includes at least one of normalizing, upsampling, or zero-padding the electrocardiogram, and wherein the standardized input includes a time-series of voltage measurements for each lead of the electrocardiogram that are sampled at a predetermined frequency over a predetermined time period.

[0062] In some aspects, the techniques described herein relate to a computer-implemented method, wherein the neural network includes a convolutional neural network having a pluralityFIG. 1 Patents 9 Docket No. : BI-11127-PCTof convolutional blocks, each convolutional block of the plurality of convolutional blocks including at least one convolutional layer and at least one concatenation, and wherein the output layer includes one or more dense layers.

[0063] In some aspects, the techniques described herein relate to a computer-implemented method, wherein the prediction associated with developing heart failure is at least one probability score indicating a likelihood of developing heart failure within a specified time frame.

[0064] In some aspects, the techniques described herein relate to a computer-implemented method for a cardiac prediction associated with heart failure, including: training a deep learning model to output the cardiac prediction, the training including: initially training the deep learning model using a training sample subset of a first portion of electrocardiogram training data by adjusting weights and biases of the deep learning model based on a difference between an output of the deep learning model for the cardiac prediction and a ground truth label; and refining the initially trained deep learning model using a validation sample subset of the first portion of the electrocardiogram training data, the refining including adjusting hyperparameters of the deep learning model; and generating the cardiac prediction for an individual using the trained deep learning model, the generating including: generating, by a data preprocessor operatively connected to the trained deep learning model, a standardized input of an electrocardiogram obtained from the individual by preprocessing the electrocardiogram; extracting, by the trained deep learning model, features of the standardized input; outputting, by at least a portion of the deep learning model, an electrocardiogram feature map summarizing the features of the standardized input; and generating the cardiac prediction based on the electrocardiogram feature map.

[0065] In some aspects, the techniques described herein relate to a computer-implemented method, wherein the training further includes: internally validating the refined deep learning model using an internal test sample subset of the first portion of the electrocardiogram training data; and externally validating the internally validated deep learning model using an external test sample including a second portion of the electrocardiogram training data.

[0066] In some aspects, the techniques described herein relate to a computer-implemented method, wherein the cardiac prediction includes at least one probability score indicating a likelihood of the individual having or developing heart failure within a specified time period.

[0067] In some aspects, the techniques described herein relate to a computer-implemented method, wherein the cardiac prediction includes an estimate of left ventricular ejection fraction.FIG. 1 Patents 10 Docket No. : BI-11127-PCT

[0068] In the following discussion, an example environment is first described that may employ the techniques described herein. Example implementation details and procedures are then described that may be performed in the example environment as well as other environments. Consequently, performance of the example procedures is not limited to the example environment, and the example environment is not limited to performance of the example procedures.

[0069] As used herein, the singular forms "a." ''an,” and "the” include both singular and plural referents unless the context clearly dictates otherwise.

[0070] The term “optional” or “optionally” means that the subsequent described event, circumstance or substituent may or may not occur, and that the description includes instances where the event or circumstance occurs and instances where it does not.

[0071] The recitation of numerical ranges by endpoints includes all numbers and fractions subsumed within the respective ranges, as well as the recited endpoints.

[0072] The terms “about” or “approximately” as used herein when referring to a measurable value such as a parameter, an amount, a temporal duration, and the like, are meant to encompass variations of and from the specified value, such as variations of + / - 10% or less, + / -5% or less, + / -!% or less, and + / -0.1% or less from the specified value, insofar as such variations are appropriate to perform in the disclosed techniques. It is to be understood that the value to which the modifier “about” or “approximately” refers is itself also specifically disclosed.

[0073] The terms “subject,” “individual,” and “patient” are used interchangeably herein to refer to a vertebrate, preferably a mammal, more preferably a human. Mammals include, but are not limited to, murines, simians, humans, farm animals, sport animals, and pets.

[0074] In the description of the figures, like numerals represent like (but not necessarily identical) elements throughout the figures.Example Environment

[0075] FIG. 1 is an illustration of an environment 100 in an example implementation that is operable to employ electrocardiogram-based deep learning for cardiac prediction as described herein. The illustrated environment 100 includes a service provider system 102. a client device 104, an electrocardiogram system 106, and a computing device 108 that are communicatively coupled, one to another, via a network 110. The network 110 may enable wired and / or wireless electronic communication, for example. Although the computing device 108 is illustrated as separate from the service provider system 102 and the client device 104, this functionality may be incorporated as part of the service provider system 102 and / or the client device 104, furtherFIG. 1 Patents 11 Docket No. : BI-11127-PCTdivided among other entities, and so forth. By way of example, an entirety of or portions of the functionality of the computing device 108 may be incorporated as part of the service provider system 102 and / or the client device 104. Additionally, or alternatively, an entirety of or portions of the client device 104 may be incorporated as part of the service provider system 102 and / or the computing device 108.

[0076] Computing devices that are usable to implement the service provider system 102, the client device 104, and the computing device 108 may be configured in a variety of ways. A computing device, for instance, may be configured as a desktop computer, a laptop computer, a mobile device (e.g., assuming a handheld configuration such as a tablet or mobile phone), and so forth. Thus, the computing device may range from full resource devices with substantial memory and processor resources (e.g.. personal computers) to a low-resource device with limited memory and / or processing resources (e.g., mobile devices). Additionally, a computing device may be representative of a plurality of different devices, such as multiple servers utilized to perform operations “over the cloud,” as further described in relation to FIG. 8.

[0077] The service provider system 102 is illustrated as including an application manager module 112 that is representative of functionality to provide access to the computing device 108 to a user of the client device 104 via the network 110. The application manager module 112, for instance, may expose content or functionality of the computing device 108 that is accessible via the network 110 by an application 114 of the client device 104. The application 114 may be configured as a network-enabled application, a browser, a native application, and so on, that exchanges data with the service provider system 102 via the network 110. The data can be employed by the application 114 to enable the user of the client device 104 to communicate with the service provider system 102, such as to receive application updates and features when the service provider system 102 provides functionality to manage the application 114.

[0078] In the context of the described techniques, the application 114 includes functionality to train and / or use at least one machine learning model to analyze ECG data and output a cardiac prediction 116, as will be elaborated herein. By way of example, the cardiac prediction 116 may be related to heart failure. In one or more implementations, the cardiac prediction 116 includes a probability score indicating the likelihood of heart failure or risk of developing heart failure within a specified time frame. Additionally, the cardiac prediction 116 may provide an estimated time frame for potential heart failure onset, which could range from months to years. In some cases, the cardiac prediction 116 may also include an indication of heart failure severity, categorizing the risk as low, moderate, or high. The cardiac prediction 116 may furtherFIG. 1 Patents 12 Docket No. : BI-11127-PCTindicate specific ECG features or patterns that contributed to the risk assessment, providing clinicians with actionable insights. In some implementations, the cardiac prediction 116 may also include recommendations for follow-up tests or interventions based on the predicted risk level. In at least one variation, the cardiac prediction 116 includes a measurement of left ventricular ejection fraction (LVEF), which may correspond to how much blood the left ventricle pumps out during each contraction (e.g., heartbeat). In general, a lower ejection fraction corresponds to weaker pumping actions of the heart. As such, the LVEF may provide one indication of the function or dysfunction of the left side of the heart. A low ejection fraction (e.g., less than 50%), for instance, may indicate a higher risk of heart failure. Accordingly, the LVEF is related to heart failure.

[0079] In the illustrated example, the application 114 includes an interface 118 that is implemented at least partially in hardware of the client device 104 for facilitating communication between the client device 104 and the computing device 108. By way of example, the interface 118 includes functionality to receive inputs to the computing device 108 from the client device 104 (e.g., from a user of the client device 104) and output information, data, and so forth from the computing device 108 to the client device 104, including the cardiac prediction 116.

[0080] The computing device 108 illustrated in FIG. 1 is further configured to receive an ECG signal 120 from the electrocardiogram system 106. The electrocardiogram system 106 includes ECG sensors 122 configured to detect electrical activity of the heart of a subject (e.g., a patient) during an ECG recording. By way of example, the ECG sensors 122 may include one or more electrodes that detect voltage differences on the skin surface resulting from the heart’ s electrical activity. ECG signals are ty pically in the range of millivolts; the size of each electrical wave is termed the amplitude, and the number of cardiac cycles per minute is the heart rate. For medical applications, the frequency content of ECG signals typically lies within the range of 0.05-150 Hz. This frequency content range characterizes the physiological components of the ECG signal and is distinct from the sampling frequency of the electrocardiogram system 106, which is generally higher, as further mentioned below. After the ECG sensors 122 detect the electrical signals from the body, these signals are amplified and filtered to produce the ECG signal 120. The ECG signal 120 may be in the form of a time-varying voltage signal, for instance.

[0081] The terms “record” or “recording” may be used herein to refer to acquiring data through the process of detecting and processing electrical signals from the heart. The term “data” may be used herein to refer to one or more datasets acquired with an electrocardiogram system, such as the electrocardiogram system 106. In at least one implementation, data acquired via theFIG. 1 Patents 13 Docket No. : BI-11127-PCTelectrocardiogram system 106 is processed via a data processor 124 of the computing device 108 to generate ECG data 126, which may be stored in a data storage device 128. The ECG data 126 may comprise individual heartbeat waveforms as well as longer recordings, e.g., multi-lead ECG strips. The data storage device 128 may represent one or more databases and other types of storage capable of storing the ECG data 126. The data storage device 128 may also store a variety of other data, such as patient demographic information, electronic health record information, and so forth.

[0082] By way of example, the data processor 124 may process the ECG signal 120 in realtime during a recording session (e.g., a period of time where a healthcare provider acquires the ECG signal 120 via the electrocardiogram system 106), as the electrical signals are received and transmitted to the computing device 108. The term '’real-time” is defined to include a procedure that is performed without intentional delay (e.g., substantially at the time of occurrence). In the context of electrocardiography, for instance, real-time denotes generating the ECG data 126 substantially as the ECG signal 120 is acquired. As a non-limiting example, the electrocardiogram system 106 may acquire data at a real-time sampling rate ranging between 250 and 1000 Hz. However, it should be understood that the real-time sampling rate may be dependent on the specific application and the amount of detail required. Accordingly, when acquiring a relatively large amount of data, the real-time processing may be adjusted. Thus, some implementations may have real-time sampling rates that are considerably faster than 1000 Hz, while other embodiments may have real-time sampling rates slower than 250 Hz. In at least one variation, the data may be stored temporarily in a buffer (not shown) during a recording session and processed in less than real-time by the data processor 124 in an off-line operation.

[0083] The ECG data 126 generated by the computing device 108 from the ECG signal 120 may be updated at a same or similar rate at which the ECG signal 120 is acquired. The data storage device 128 may store the processed ECG data 126. In at least one implementation, the ECG data 126 are stored in a manner to facilitate retrieval thereof according to its order or time of acquisition. The data storage device 128 may comprise any known data storage medium. It is to be appreciated that while the data processor 124 and the data storage device 128 are illustrated as part of the computing device 108, in at least one variation, the data processor 124 and / or the data storage device 128 are part of the electrocardiogram system 106 and / or another computing device.

[0084] In one or more implementations, the data processor 124 may process the ECG signal 120 in different analysis modules (e.g., QRS detection, rhythm analysis, ST segment analysis,FIG. 1 Patents 14 Docket No. : BI-11127-PCTQT interval measurement, and the like) to extract various features and measurements. When multiple ECG leads are obtained, the data processor 124 may also be configured to analyze the relationships between different leads. For example, one or more modules may perform signal filtering, baseline wander removal, QRS complex detection, heart rate calculation, arrhythmia detection, ST segment analysis, T wave altemans analysis, and the like, and combinations thereof. The modules may include, for example, a feature extraction module to identify points in the ECG waveform such as P waves, QRS complexes, and T waves. In ECG analysis, for instance, normal sinus rhythm may show a characteristic pattern of P waves, QRS complexes, and T waves, whereas various abnormalities may result in changes to this pattern. A display module may be provided that reads the ECG data 126 from the data storage device 128 and displays the ECG waveform or a derived measurement in real-time while a procedure (e.g., an ECG recording procedure) is being performed on the patient and / or after completion of the procedure.

[0085] Further, the components of the electrocardiogram system 106 and / or the computing device 108 may be coupled to one another to form a single structure, may be separate but located within a common room, or may be remotely located with respect to one another. For example, one or more of the modules described herein may operate in a data server that has a distinct and remote location with respect to other components of the electrocardiogram system 106 and / or the computing device 108, such as the ECG sensors 122. Optionally, the electrocardiogram system 106 may be a unitary system that is capable of being moved (e.g., portably) from room to room. For example, the electrocardiogram system 106 may include wheels, may be transported (e.g., on a cart), or may comprise a handheld device.

[0086] In at least one implementation, the ECG data 126, or a portion thereof, may be processed by an ECG analysis module 130. By way of example, the ECG analysis module 130 is representative of the functionality implemented at least partially in hardware of the computing device 108 to analyze the ECG data 126, such as one or more ECG recordings, and output the cardiac prediction 116. In the example shown in FIG. 1, the ECG analy sis module 130 includes a data preprocessor 132 and at least one deep learning model 134 for analyzing the ECG data 126 to generate the cardiac prediction 116. The at least one deep learning model 134 is a trained machine learning model. By way of example, the ECG analysis module 130 may include multiple different deep learning models that correspond to different types of machine learning models, where the underlying models learn using different approaches (e.g., supervised learning, unsupervised learning, and / or reinforcement learning), and / or multiple different deep learning models having a same model architecture but that are trained usingFIG. 1 Patents 15 Docket No. : BI-11127-PCTdifferent input data and / or to output a different type of cardiac prediction 116. By way of example, these models may include regression models (e.g.. linear, polynomial, and / or logistic regression models), classifiers, neural networks, and reinforcement learning based models, to name just a few.

[0087] The at least one deep learning model 134 may be configured as (or include) other types of models without departing from the spirit or scope of the described techniques. These different machine learning models may be built or trained (or the model otherwise learned), respectively, using different inputs and / or different algorithms due, at least in part, to different architectures and / or learning paradigms. Accordingly, it is to be appreciated that the following discussion of the functionality of the ECG analysis module 130 is applicable to a variety of machine learning models. For explanatory purposes, however, the functionality of the at least one deep learning model 134 will be described generally with respect to a convolutional neural network (CNN). The CNN, for instance, may include one temporal dimension. By way of example, the at least one deep learning model 134 may be based on a ID CNN architecture to process temporal information in ECG signals. Additional details of the CNN will be described herein, e.g., with respect to FIG. 2. In one or more implementations, the CNN is combined with additional architectures and / or model portions to produce the cardiac prediction 116. Moreover, the cardiac prediction 116 may be different based on the particular deep learning model 134 used, as elaborated herein.

[0088] The computing device 108 further includes a training module 136 that is implemented at least partially in hardware of the computing device, at least in part, to deploy deep learning to generate the at least one deep learning model 134. By way of example, the training module 136 includes a model training manager 138 that is configured to manage the at least one deep learning model 134. This model management may include, for example, building the at least one deep learning model 134, training the at least one deep learning model 134, updating the model(s), and so forth. For instance, the model training manager 138 may be configured to carry out this model management using, at least in part, training data 140 maintained in a training data storage device 142. As illustrated in the environment 100 of FIG.1, the training data 140 may include a model derivation subset 144 and an external test subset 146. For example, the model training manager 138 may use at least a portion of the model derivation subset 144 of the training data 140 as input for training the at least one deep learning model 134 and may use at least a portion of the external test subset 146 for evaluating performance of the at least one deep learning model 134 after the at least one deep learning model 134 is at least initially trained. The model derivation subset 144 and the external test subset 146 mayFIG. 1 Patents 16 Docket No. : BI-11127-PCTinclude ECG recordings from different data sources, for example. As such, the external test subset 146 may be used to verify that the at least one deep learning model 134 achieves performance goals on data from diverse sources. Moreover, although the training data 140 may include multiple ECG recordings from a single patient, each separate recording may be treated as a separate training sample. Ellipses denote that more than one training data set may be stored in the training data storage device 142.

[0089] It is to be appreciated that although the model derivation subset 144 and the external test subset 146 are shown stored in the same training data storage device 142, in at least one variation, the model derivation subset 144 and the external test subset 146 are distributed among multiple storage locations. Alternatively, or in addition, the training data storage device 142 may be stored in a location that is external to the computing device 108 and accessed by the computing device 108 (e.g., over the network 110). As such, it is to be appreciated that the relative arrangement of the various modules and data storage devices in FIG. 1 is non-limiting, and variations are possible.

[0090] In one or more implementations, the model derivation subset 144 is further subdivided into a training sample 148, a validation sample 150, and an internal test sample 152. By way of example, the training sample 148 may comprise a largest portion of the model derivation subset 144, while the validation sample 150 and / or the internal test sample 152 may comprise a smallest portion of the model derivation subset 144. As a non-limiting example, the training sample 148 comprises 70% of the model derivation subset 144, the validation sample 150 comprises 15% of the model derivation subset 144, and the internal test sample 152 comprises 15% of the model derivation subset 144, although other divisions are possible. The training sample 148, for instance, may comprise between 50% and 80% of the model derivation subset 144, the validation sample 150 may comprise between 10% and 40% of the model derivation subset 144, and the internal test sample 152 may comprise betw een 5% and 30% of the model derivation subset 144.

[0091] Broadly speaking, the training sample 148 may be input to the at least one deep learning model 134 during a training process, where the at least one deep learning model 134 leams patterns and relationships in the data. During the training process, weights and parameters of the at least one deep learning model 1 4 may be adjusted to reduce (e g., minimize) errors betw een an output of the model and a ground truth label associated w ith a corresponding ECG recording (e.g., the heart failure risk prediction, as determined by a clinician). Following completion of the training process, the at least one deep learning model 134 is able to accurately predict the cardiac prediction 116 of the training sample 148.FIG. 1 Patents 17 Docket No. : BI-11127-PCT

[0092] The validation sample 150 may be input to the at least one deep learning model 134 during a model refinement (e.g., fine-tuning) process, where the at least one deep learning model 134 is adjusted to prevent or reduce overfitting / underfitting of the model to the training sample 148. By way of example, the model refinement process may be performed following each round (or epoch) of training to evaluate how well the at least one deep learning model 134 performs on data that is different from the training sample 148. During the model refinement process, for instance, a complexity, learning rate, and / or regularization of the at least one deep learning model 134 may be adjusted (e.g., by the model training manager 138, automatically and / or based on user input) based on the performance of the at least one deep learning model 134 with the validation sample 150. As an illustrative example, if the at least one deep learning model 134 accurately predicts the cardiac prediction 116 of the training sample 148 but not the validation sample 150, overfitting of the at least one deep learning model 134 to the training sample 148 is indicated. As such, the model refinement process enables settings of the at least one deep learning model 134 and / or its training to be fine-tuned so that the at least one deep learning model 134 can be generalized to unseen data (e.g., data that the at least one deep learning model 134 has not been trained on).

[0093] The internal test sample 152 may be input to the at least one deep learning model 134 during an internal validation process that is performed after the at least one deep learning model 134 is trained and fine-tuned. The internal test sample 152 comprises data that was unseen by the at least one deep learning model 134 during the training and model refinement processes described above but that is derived from the same dataset (e.g., the model derivation subset 144). The internal validation process evaluates the performance of the at least one deep learning model 134 on similar data to that used during the training and model refinement processes. If the at least one deep learning model 134 does not meet acceptable or desired performance criteria (e.g., as defined by model developers) during the internal validation process, the at least one deep learning model 134 may be returned to the training and / or model refinement processes so that changes can be made. For example, changes may be made to feature selection, the model architecture, regularization techniques, hyperparameter tuning, and the like.

[0094] The model training manager 138 may leverage the functionality of the data preprocessor 132 to process the training data 140 during the training, refinement, and validation processes described above. The data preprocessor 132, for instance, may remove patient identifying information and standardize ECG data input into the at least one deep learning model 134 such as by normalizing, upsampling, and / or zero-padding the input electrocardiogram.FIG. 1 Patents 18 Docket No. : BI-11127-PCT

[0095] As used herein, “normalizing” the ECG data may refer to adjusting ECG voltage measurements to a standardized scale and / or range, which may include scaling voltages (e.g., to have a mean of zero and a standard deviation of one), adjusting amplitudes to a predetermined range, and / or other normalization techniques to ensure consistent formatting of the ECG data prior to input into the at least one deep learning model 134. The standardization performed by the data preprocessor 132 may ensure that the at least one deep learning model 134 leams clinically relevant features associated with developing heart failure and / or the LVEF rather than variations in data acquisition parameters, recording equipment characteristics, and / or formatting differences between different electrocardiogram systems 106.

[0096] Once the at least one deep learning model 134 is at least initially trained and internally validated, the model training manager 138 may use the external test subset 146 of the training data 140 to evaluate generalizability of the at least one deep learning model 134. As mentioned above, the external test subset 146 comprises ECG data from a different data source, such as a different healthcare facility7and / or patient population. By way of example, the external test subset 146 may be input into the at least one deep learning model 134. and the at least one deep learning model 134 may output the cardiac prediction 116 for respective ECG recordings. The cardiac prediction 116 may be compared to the ground truth labels to evaluate an accuracy of the at least one deep learning model 134 on this novel dataset. The external test subset 146, for instance, may be used to verify that the performance of at least one deep learning model 134 is not specific to the data source of the model derivation subset 144.

[0097] In response to the at least one deep learning model 134 meeting desired or acceptable performance metrics, the at least one deep learning model 134 may be deployed for determining the cardiac prediction 116 of newly obtained ECG data, including ECG data for which there is no ground truth label. By way of example, the ECG data 126 may correspond to ECG recordings that have not been evaluated by a clinician or technician with respect to the cardiac prediction 116. The ECG data 126 may be input into the (trained and validated) at least one deep learning model 134 at or around the time of acquisition, and the at least one deep learning model 134 may output the cardiac prediction 116 accordingly, thus enabling a streamlined ECG analysis workflow.

[0098] In at least one implementation, the cardiac prediction 116 includes a heart failure risk prediction that is represented as a probability7score indicating a likelihood of heart failure or risk of developing heart failure. The cardiac prediction 116 may also include an estimated time frame for potential heart failure onset or an indication of heart failure severity. Additionally, orFIG. 1 Patents 19 Docket No. : BI-11127-PCTalternatively, the cardiac prediction 116 may include a LVEF prediction, which may provide an estimated LVEF value or classification (e.g., low versus preserved).

[0099] Although the above discussion is focused on ECG inputs to the at least one deep learning model 134 and receiving the cardiac prediction 116 as the output, it is to be appreciated that additional inputs and / or outputs are possible. By way of example, the at least one deep learning model 134 may be trained on auxiliary (e.g., secondary’) tasks that may help the at least one deep learning model 134 learn shared representations of the input data, which increase the performance of the at least one deep learning model 134 for the cardiac prediction 116. Examples of auxiliary’ tasks include predicting age, sex, and / or other demographic features. Additional or alternative auxiliary tasks may be discussed herein below with respect to example model applications.

[0100] The client device 104 is shown displaying, via a display device 154, cardiac prediction 116. Alternatively, or in addition, the client device 104 may display, via the display device 154, the ECG data 126. It is to be appreciated that the cardiac prediction 116 may be also stored in a memory of the computing device 108 and / or the client device 104 for subsequent access.

[0101] In this way, the ECG analysis module 130 enables automated ECG analysis for identifying and / or monitoring heart failure risk, which may be used in patient stratification, risk assessment, and / or treatment monitoring.Electrocardiogram-Based Deep Learning for Cardiac Prediction

[0102] FIG. 2 depicts an example training process 200 that may be used in generating the at least one deep learning model 134 of the ECG analysis module 130 of FIG. 1. It is to be appreciated that the example training process 200 denotes one implementation of a training process that may be used in generating at least a portion of the at least one deep learning model 134.

[0103] In the example training process 200, a training instance 202 includes an ECG input 204 and a ground truth label 206 associated with the ECG input 204. The ECG input 204, for instance, includes a sequence of voltage measurements obtained during a single acquisition process. In at least one implementation, the ECG input 204 comprises a 12-lead ECG. In at least one variation, the ECG input 204 may comprise a single-lead ECG, a 3-lead ECG, a 5- lead ECG, a 6-lead ECG, or another lead configuration. The specific lead configuration used may be selected based on factors such as the desired analysis, available equipment, or patient characteristics. During the training, the ECG input 204 is part of a corresponding portion of the training data 140 (e.g., the training sample 148).FIG. 1 Patents 20 Docket No. : BI-11127-PCT

[0104] Each ECG input 204 may be separately evaluated by the at least one deep learning model 134 to generate a model output 208 (e.g., an output produced by the at least one deep learning model 134), which corresponds to the single ECG input 204. As such, the training instance 202 includes an input portion (e.g., the ECG input 204) and an associated expected output portion (e.g., the ground truth label 206), and a great many training instances 202 may be used during the training process. The model output 208 corresponds to a prediction for a particular classification and / or measurement task(s) the at least one deep learning model 134 is being trained for. By way of example, the at least one deep learning model 134 may be trained to output (e.g., as the model output 208) the cardiac prediction 116. In one or more implementations, a first model of the at least one deep learning model 134 may be trained to output (e.g., as the model output 208) a heart failure risk prediction, while a second model of the at least one deep learning model 134 may be trained to output a LVEF prediction. Additionally, the at least one deep learning model 134 may be trained on auxiliary tasks such as predicting age, sex, and mortality risk. The ground truth label 206 defines a true or expected output of the at least one deep learning model 134 for the ECG input 204. By way of example, the ground truth label 206 may be or may be derived from clinical labels and / or adjudicated by an expert for comparison to the model output 208 during training.

[0105] In at least one implementation where cardiac prediction 116 is a heart failure risk prediction, the ground truth label 206 includes a heart failure classification. The heart failure classification may be a binary input, where the presence of the heart failure event is denoted by a one, and the absence of the heart failure event is denoted by a zero. The presence of heart failure may be defined based on an International Classification of Diseases, 9th or 10th revision (ICD 9 / 10) code for heart failure being indicated in corresponding medical records of the ECG input 204 and / or classified using a transformer-based natural language processing (NLP) model trained to detect heart failure hospitalization using free text discharge summaries in the corresponding medical records. Additional details regarding the use of the NLP model in the adjudication of heart failure are described in Cunningham et al., JACC Heart Fail. 2023;! l(7):952-954, and Cunningham et al., JAMA Cardiol. 2023 (doi: 10. lOOl / jamacardio.2023.4859), the entire contents of which are incorporated herein by reference. By using the transformer-based NLP model, the ground truth label 206 includes a more accurate heart failure classification than by using the ICD 9 / 10 codes alone.

[0106] Additionally, or alternatively, rather than a single binary classification, the heart failure classification may be divided into a plurality of time bins, where a "‘one” denotes the presence of a heart failure diagnosis within the corresponding time bin and "zero" denotes the absenceFIG. 1 Patents 21 Docket No. : BI-11127-PCTof the heart failure diagnosis within the corresponding time bin. Together, the time bins may span months to years of follow-up. As an illustrative example, the time bins may span a period that is between five and ten years, with each time bin spanning days to months (e.g., 72 days). The heart failure classification, for instance, may be input as an event vector of a length corresponding to the number of time bins. In at least one implementation, the heart failure classification may be further associated with a censoring vector that masks time bins in which the individual from which the training instance 202 is derived was censored (e.g., there is not corresponding follow-up information for the corresponding individual). Censoring occurs when the study ends before heart failure occurs in the individual or when the individual is lost to follow-up before experiencing the event. By way of example, censored data may include patients who did not experience heart failure during the observation period but might have if the study had continued for a longer time. As such, the censoring vector may be equal to zero for time bins in which the individual was censored and one for time bins for which they remained in the study and / or examined via follow-up. As such, the missingness of data may not influence model training.

[0107] In one or more implementations where the cardiac prediction 116 is the LVEF prediction, the ground truth label 206 may include a LVEF measurement. The LVEF measurement may be a continuous value representing the percentage of blood pumped out of the left ventricle with each heartbeat, for example, typically ranging from approximately 20% to 80%. As a non-limiting example, the LVEF values may be obtained from echocardiographic measurements performed within a specified time window of the ECG recording (e g., within 30 days, 90 days, or 1 year). In at least one implementation, the LVEF ground truth may be categorized into qualitative classifications such as reduced LVEF (e.g., <40%), mildly reduced LVEF (e.g., 40-49%), or preserved LVEF (e.g., >50%). The LVEF ground truth labels may be derived from structured echocardiography reports or extracted from clinical databases where quantitative LVEF measurements are available.

[0108] It is to be appreciated that the ground truth label 206 may further include additional labels, such as those corresponding to auxiliary training tasks (e.g., sex, age, time to death, and the presence of an ICD 9 / 10 code for heart failure in the primary billing code position).

[0109] In the example shown in FIG. 2, the ECG input 204 is provided to the data preprocessor 132, which generates a standardized input 210. As mentioned above with respect to FIG. 1, the data preprocessor 132 may remove patient identifying information and standardize ECG data input into the at least one deep learning model 134. For example, the data preprocessor 132 may apply filtering techniques to remove noise and baseline wander from the ECG signal. ItFIG. 1 Patents 22 Docket No. : BI-11127-PCTmay also normalize the amplitude of the ECG waveforms to a standard range. Additionally, the data preprocessor 132 may truncate or extend the ECG data to a predetermined time duration (e.g., 10 seconds). By way of example, ECG data with shorter durations than the predetermined time duration may be zero-padded to contain a standardized number of measurements. The data preprocessor 132 may upsample or downsample data to a predetermined frequency (e.g., 500 Hz). By way of example, data recorded at 250 Hz may be upsampled to 500 Hz. The data preprocessor 132 may also handle missing or corrupted data points through interpolation or other imputation methods, such as zero-padding. The preprocessing performed by the data preprocessor 132 may thus provide consistent, high- quality inputs to the at least one deep learning model 134, which may improve the performance and generalizability of the at least one deep learning model 134 across different ECG recording conditions and equipment.

[0110] The data preprocessor 132 may be operatively connected to the at least one deep learning model 134 to provide the standardized input 210 to the at least one deep learning model 134. As a non-limiting example, the standardized input 210 comprises a time-series of 5000 voltage measurements for each of the 12 leads, sampled at 500 Hz over 10 seconds. In the example training process 200 shown in FIG. 2, the at least one deep learning model 134 comprises a CNN 212 having convolutional layers 214. The convolutional layers 214 are shown as including a first convolutional block 216 (e.g., “convolutional block 1”), a second convolutional block 218 (e.g.. “convolutional block 2”), and an Nthconvolutional block 220 (e.g., “convolutional block N”), where N is an integer representing the total number of convolutional blocks. Ellipses denote that additional convolutional blocks may be present between the second convolutional block 218 and the Nthconvolutional block 220. As a nonlimiting example, there may be four convolutional blocks. In general, the convolutional layers 214 may be configured to extract features of the standardized input 210. An example architecture of the convolutional blocks is described below with respect to FIG. 3.

[0111] As used herein in the context of machine learning, the term “features” may refer to individual measurable properties, characteristics, and / or patterns of the input data that are used by the at least one deep learning model 134 to make predictions or decisions. Features may be numerical values extracted or computed from raw data, such as an electrocardiogram signal, that represent relevant aspects of the data for the task at hand. Initial features, for instance, may be values from the standardized input 210 itself, often represented as voltage readings over time. Each time step in the standardized input 210 is a potential feature that reflects the heart's electrical activity at that instant. Extracted features (e.g., extracted by the CNN 212) mayFIG. 1 Patents 23 Docket No. : BI-11127-PCTinclude, for example, basic shapes or waveforms in the ECG, like peaks or troughs; temporal patterns in the ECG that may reflect heartbeat rhythms or irregularities; and / or combinations of patterns that might correspond to higher-level physiological or health indicators. In the at least one deep learning model 134, features serve as the basis for learning data patterns and relationships, allowing the at least one deep learning model 134 to generalize from training data to make predictions on new, unseen data.

[0112] In the present example shown in FIG. 2, the convolutional layers 214 are grouped together in blocks. By way of example, a block may comprise multiple convolutional layers that are stacked together with operations such as pooling, batch normalization, and activation functions to form a unit that is able to extract more complex or abstract features than a single convolutional layer alone. In at least one variation, one or more of the convolutional layers 214 are arranged as single convolutional layers. For instance, the first convolutional block 216, the second convolutional block 218, and / or the Nthconvolutional block 220 instead may be a single convolutional layer. In at least one implementation, the CNN 212 is a dense convolutional network, where each block connects to all subsequent blocks, allowing features from earlier blocks to be directly accessible to later layers.

[0113] The first convolutional block 216, for example, may apply a kernel (e.g., a filter) to the standardized input 210, which scans across the dimension to extract low-level features like peaks, valleys, and edges across the ECG signal. The features detected by the first convolutional block 216 may be fed to the subsequent convolutional layers 214 (e.g., the blocks) in sequence, allowing the CNN 212 to detect increasingly complex patterns in the standardized input 210. Accordingly, each block may progressively learn more complex features, helping the model understand the ECG data at multiple scales. By way of example, the second convolutional block 218 may learn higher-level representations, like specific morphologies of waveforms, variations in wave amplitude, or frequency patterns, and the N* convolutional block 220 may leam global patterns that span across multiple heartbeats. Alternatively, or in addition, the CNN 212 may perform progressive downsampling, where the number of input measurements decreases through striding or pooling operations. This may include reducing the temporal dimension to enable the at least one deep learning model 134 to focus on capturing larger patterns and relationships in the data over longer time spans, while reducing the computational load.

[0114] The first convolutional block 216, as well as others of the convolutional layers 214, may comprise weights 222, a bias 224, an activation function 226. and hyperparameters 228. By way of example, the weights 222 and the bias 224 may be randomly initialized and thenFIG. 1 Patents 24 Docket No. : BI-11127-PCT“learned"’ during the training process, as elaborated below. The CNN 212, for instance, performs a series of convolutions. A convolution is a mathematical operation where the kernel slides over an input ECG signal and performs element-wise multiplication with the values of the signal at each temporal position. The results are summed up to produce a single output value for that temporal position, and this process is repeated across the ECG signal (e.g., the standardized input 210) to produce a feature map. The kernel comprises a matrix of numbers, which are the weights 222 of the kernel, that is applied to the input. The bias 224 is a single number added to the result of the convolution. After each convolution, the feature map may be passed through the activation function 226, which may introduce non-linearity. In at least one implementation, the activation function 226 is a swish activation function. After several convolution operations, a pooling layer may be used to reduce the size of the feature map.

[0115] The hyperparameters 228 are not learned during the training process but can be adjusted to increase performance. The hyperparameters 228 may comprise depth, stride, and zeropadding. Depth controls the number of neurons within a given convolutional layer of the convolutional layers 214. Reducing the depth may increase the speed of the CNN 212 but may also reduce the accuracy of the CNN 212. Stride determines how much the kernel moves or “slides” across the input. A stride of two means that the kernel moves two positions along the ECG signal at each step, effectively skipping one sample each time. A higher stride value would thus enable the CNN 212 to process data more quickly and reduces the spatial dimensions of the feature map. Zero-padding adds zeros around the edges of the input signal, allowing the kernel to be applied to edge samples where it would otherwise not fit, and ensuring that the output of the convolution remains the same size as the input w hen desired.

[0116] In at least one implementation, the CNN 212 outputs an ECG feature map 230, which may be a multi-dimensional representation of the features extracted from the standardized input 210. The ECG feature map 230 may capture various characteristics of the ECG signal at different levels of abstraction, from low -level features like signal peaks and valleys to higher- level patterns such as w aveform morphologies and rhythm irregularities. The ECG feature map 230 may be processed through a flatten layer 232. which may transform the ECG feature map 230 into an ECG vector 234. The ECG vector 234 may be a one-dimensional vector that summarizes the features or characteristics of the standardized input 210 that have been learned by the CNN 212. In some implementations, the dimensionality of the ECG vector 234 may be reduced compared to the ECG feature map 230, which may help in managing computational complexity while retaining information for cardiac prediction.FIG. 1 Patents 25 Docket No. : BI-11127-PCT

[0117] The ECG vector 234 is input into one or more dense layers 236, which are fully connected layers where every neuron (node) is fully connected to every neuron in the previous and next layer. The one or more dense layers 236 may function to aggregate information learned by previous layers and make final predictions for the model output 208. Accordingly, the one or more dense layers 236 may comprise an output layer (or final layer) of the at least one deep learning model 134. The one or more dense layers 236 may leam complex relationships between the features represented in the ECG vector 234, which may enable the model to capture subtle patterns that may be indicative of heart failure risk, for instance, and / or the LVEF. Similar to the convolutional layers 214, the one or more dense layers 236 may comprise the weights 222, the bias 224, the activation function 226, and the hyperparameters 228. It is to be appreciated that values of at least a portion of these parameters and / or type of activation function used are specific to a given convolutional layer 214 or dense layer 236. The one or more dense layers 236, for instance, may take the ECG vector 234 as an input vector, multiply it by a matrix of the weights 222, add the bias 224, and then apply the activation function 226 to produce the model output 208.

[0118] The model output 208 is received by the model training manager 138, which may perform a loss calculation 238. The loss calculation 238 may use a loss function to compute the difference between the model output 208 and the ground truth label 206, e.g., a loss 240. The loss 240. for instance, is a measure of the error of the at least one deep learning model 134 in determining the model output 208. Various types of loss functions may be used depending on the specific task and model architecture. For classification tasks, cross-entropy loss may be employed. For regression tasks, mean squared error or mean absolute error may be used.

[0119] In one or more implementations, a custom loss function may be used to address specific aspects of the heart failure prediction task. By way of example, when the model output 208 includes predicting a time until a first heart failure event (e.g., VPREDICT, a vector of length n representing a heart failure survival probability at each time bin, where n is the number of time bins) and the ground truth label 206 includes the heart failure classification as an event vector (e.g., VHF) and associated censoring vector (e.g., VCENS0R), the loss function may be an encoding and loss function that minimizes a negative log likelihood of the model output 208. As a non-limiting example, the loss function may be:where ^survival 1 (VHF * ^PREDICT) and Tevent—(VcENSOR * ^PREDICT + (1 ? ENSOR) ■FIG. 1 Patents 26 Docket No. : BI-11127-PCT

[0120] When training the at least one deep learning model 134 for LVEF prediction, the loss calculation 238 may use mean squared error or mean absolute error loss functions for regression tasks, or cross-entropy loss for classification tasks when LVEF is categorized into discrete ranges.

[0121] The loss calculation 238 may use the selected loss function to compute the difference between the model output 208 and the ground truth label 206, resulting in the loss 240. The loss 240 may be a measure of the error of the at least one deep learning model 134 in determining the model output 208.

[0122] A goal of the training is to minimize the loss 240 by adjusting the weights 222 and biases 224 of the given deep learning model of the at least one deep learning model 134. In order to do so, the model training manager 138 may employ backpropagation 242 to compute how the parameters are to be updated based on a gradient of the loss 240 with respect to each parameter. By way of example, the backpropagation 242 may utilize an optimization algorithm such as gradient descent (e.g., Adam). The backpropagation 242 results in adjustments 244, which are used to update the weights 222 and biases 224 of the at least one deep learning model 134.

[0123] As such, following many rounds of training with a large number of training instances 202, the model output 208 becomes consistent with the ground truth label 206 due to the at least one deep learning model 134 “learning” to minimize the loss 240 between the model output 208 and the ground truth label 206.

[0124] In this way, the training process 200 provides a comprehensive framework for developing and refining the at least one deep learning model 134 for the cardiac prediction 116. This approach may enable efficient learning from large datasets of ECG recordings, which may improve the accuracy and generalizability of the cardiac prediction 116. The trained at least one deep learning model 134 may be deployed in various clinical settings to analyze newly acquired ECG data from patients. For example, in a hospital or primary care clinic, the model could be integrated into existing ECG analysis workflows. When a patient undergoes an ECG recording, the data may be automatically processed by the ECG analysis module 130, providing an immediate cardiac assessment, such as for heart failure risk. This real-time analysis may assist healthcare providers in several ways. As one example, patients at high risk of heart failure, who may benefit from additional diagnostic tests or preventive interventions, may be identified. For patients with known heart conditions, the cardiac prediction 116 may aid in monitoring disease progression and assessing the effectiveness of current treatments. As another example, the ability of the at least one deep learning model 134 to perform auxiliaryFIG. 1 Patents 27 Docket No. : BI-11127-PCTtasks, such as predicting age and sex, may offer additional clinical value. These predictions could serve as a quality control measure, potentially flagging discrepancies in patient information or ECG lead placement errors. Such checks may enhance the overall reliability of the ECG analysis process. Moreover, in emergency settings, the rapid risk assessment provided by the cardiac prediction 116 could support triage decisions, helping to prioritize patients who may require urgent cardiac care. This could be particularly valuable in resource-constrained environments or during high-volume periods.

[0125] FIG. 3 illustrates an example implementation 300 of a convolutional block structure 302 that may be used in at least a portion of the convolutional layers 214 of the CNN 212 of FIG. 2. The convolutional block structure 302 may include multiple convolutional layers and operations configured to extract features from the input ECG data.

[0126] In the implementation 300, the convolutional block structure 302 includes a first convolutional layer 304 (e.g., “convolutional layer 1”), a second convolutional layer 306 (e.g., “convolutional layer 2”), and a third convolutional layer 308 (e.g., “convolutional layer 3’"). The first convolutional layer 304 may include a ID convolution 310, which applies a onedimensional convolution operation to the input data. The output of the 1 D convolution 310 may be passed through an activation function 226, which allows the CNN 212 to model complex relationships in the data that a purely linear transformation would miss. Following the activation function 226, a spatial dropout operation 312 may be applied to reduce overfilling by randomly setting a fraction of input units to zero during training. The spatial dropout operation 312 may improve generalization of the CNN 212, for example. The first convolutional layer 304 may conclude with a max pool operation 314, which reduces the spatial dimensions of the feature maps and helps to achieve translation invariance. By way of example, the max pool operation 314 may divide the input into pooling regions and output the maximum value for each such region. This process may retain the most prominent features, thereby making the output of the first convolutional layer 304 less sensitive to small translations or shifts in the input ECG signal. This property' may enable the CNN 212 to maintain consistent performance even when ECG features, such as QRS complexes or T waves, appear at different time points across various ECG samples.

[0127] The output from the first convolutional layer 304 may be input into the second convolutional layer 306. The second convolutional layer 306 may also include the ID convolution 310 followed by the activation function 226 and the spatial dropout operation 312. The second convolutional layer 306 may not include the max pool operation 314, at least in one implementation. This may allow the second convolutional layer 306 to maintain the spatialFIG. 1 Patents 28 Docket No. : BI-11127-PCTdimensions of its input, potentially preserving more fine-grained features of the ECG signal. In some examples, omitting the max pool operation 314 in the second convolutional layer 306 may enable the CNN 212 to capture more complex, higher-level features. Additionally, or alternatively, this approach may help in retaining the full resolution of the features extracted by the first convolutional layer 304, which may be beneficial for detecting subtle ECG abnormalities, such as those associated with heart failure risk.

[0128] The outputs from the first convolutional layer 304 and the second convolutional layer 306 may be combined in a first concatenation 316. The first concatenation 316 may allow the network to preserve and utilize features from both the first convolutional layer 304 and the second convolutional layer 306, which may capture both low-level and higher-level features of the ECG signal.

[0129] The concatenated output may be processed by the third convolutional layer 308, which follows a similar structure to the second convolutional layer 306, with the ID convolution 310, the activation function 226, and the spatial dropout operation 312. Similar to the second convolutional layer 306, the third convolutional layer 308 may not include the max pool operation 314.

[0130] The output from the third convolutional layer 308 may be combined with the output from the first convolutional layer 304 in a second concatenation 318. The second concatenation 318 may allow the convolutional block structure 302 to maintain a direct connection to the earliest features extracted from the input, which may help in preserving low-level information throughout the convolutional block structure 302.

[0131] The convolutional block structure 302 incorporates skip connections in the first concatenation 316 and the second concatenation 318. These skip connections may enable information from earlier levels of the convolutional block structure 302 to be preserved in the final output. This may provide a multi-scale representation of the input data and may help gradients flow more effectively during the backpropagation 242. Moreover, concatenating features from different layers in the convolutional block structure 302 may enhance feature richness.

[0132] The specific configuration of the convolutional block structure 302 may be adjusted based on the characteristics of the ECG data and the cardiac prediction task. For example, the number of filters in each convolutional layer, the size of the convolution kernels, the dropout rate, and the pooling size may be tuned to optimize performance. Additionally, multiple instances of this convolutional block structure 302 may be stacked to form deeper networks, which may allow for the extraction of more complex and abstract features from the ECG data.FIG. 1 Patents 29 Docket No. : BI-11127-PCT

[0133] FIG. 4 depicts an example implementation 400 of using the at least one deep learning model 134 to assess electrocardiogram data for heart failure prediction. Components previously introduced in FIGS. 1-3 are numbered the same and function as previously described.

[0134] The implementation 400 includes processing newly acquired ECG data from an individual for which there is no ground truth label 206 for heart failure prediction via the ECG analysis module 130 to generate the cardiac prediction 116, which in this example, includes a heart failure risk prediction. As such, the ECG data 126, or a portion thereof, comprise the ECG input 204 in the implementation 400. The ECG data 126 are fed into the ECG analysis module 130, where the data preprocessor 132 generates the standardized input 210. This standardized input 210 is processed by the at least one deep learning model 134, which includes the CNN 212 with the convolutional layers 214. As described with respect to FIGS. 2 and 3, for instance, these convolutional layers may include multiple blocks that progressively extract features from the standardized input 210, such as waveform morphologies, rhythm patterns, and temporal relationships.

[0135] The CNN 212 outputs the ECG feature map 230, representing a high-dimensional abstraction of the input ECG data 126. The ECG feature map 230 is transformed by the flatten layer 232 into the one-dimensional ECG vector 234. The ECG vector 234 is processed through the one or more dense layers 236, which may leam complex non-linear relationships between the extracted features to produce the model output 208.

[0136] In the implementation 400, the model output 208 includes several predictions derived from the learned ECG features. The primary output is the cardiac prediction 1 16, which in this example includes a time to heart failure prediction 402. The time to heart failure prediction 402 may include a probability or risk score for the individual developing heart failure at different time bins. The time to heart failure prediction 402, for instance, may be an event vector including a plurality of probability or risk scores, such as one probability or risk score for each time bin. A survival curve, for instance, may be constructed from the time to heart failure prediction 402. In at least one implementation, the time to heart failure prediction 402 may be further summarized as an estimated timeframe for when the individual may develop heart failure and / or an overall probability or risk score for developing heart failure (e.g., a cumulative probability or risk score). The individual may be stratified (e.g., into high risk versus low risk, tertiles of risk, quintiles of risk, etc.) based on the cumulative probability' or risk score compared to one or more thresholds.

[0137] In at least one implementation, the model output 208 includes additional predictions. An all-cause mortality prediction 404 may estimate the risk of death from any cause within aFIG. 1 Patents 30 Docket No. : BI-11127-PCTspecified timeframe based on the ECG characteristics of the ECG data 126. The all-cause mortality prediction 404 may help clinicians assess overall patient health and prioritize interventions, for instance. An age prediction 406 may estimate the biological age of the individual based on the ECG data 126, which may potentially serve as a measure of cardiovascular age or overall health status. A sex prediction 408 may estimate the biological sex of the individual based on the ECG data 126. which may be used as a quality control measure or to identify sex-specific cardiovascular risks. An ICD code prediction 410 may estimate the likelihood of specific ICD 9 / 10 codes being included in the individual’s medical health record based on the ECG data 126. These additional predictions (e.g., the all-cause mortality prediction 404, the age prediction 406, the sex prediction 408, and the ICD code prediction 410) are examples of auxiliary predictions 412. The auxiliary predictions 412. for instance, may be produced by additional output nodes in the one or more dense layers 236 or by separate branches of the CNN 212 that share early layers with the cardiac prediction 116 task. In at least one implementation, the model is trained to simultaneously produce these multiple outputs, which may enable the at least one deep learning model 134 to leverage shared representations learned from the ECG data (e.g., the training data 140). This multi-tasking learning approach may improve the generalization and performance on the cardiac prediction 116 task by leveraging correlations between related cardiovascular outcomes and patient characteristics, for instance.

[0138] Additional example details of the usage of the model output 208 are described herein with respect to the Example Application.

[0139] FIG. 5 depicts an example implementation 500 of using the at least one deep learning model 134 to assess electrocardiogram data for left ventricular ejection fraction prediction. Components previously introduced in FIGS. 1-4 are numbered the same and function as previously described.

[0140] The implementation 500 includes processing newly acquired ECG data from an individual for which there is no ground truth label 206 for LVEF prediction via the ECG analysis module 130 to generate, as the cardiac prediction 116, a LVEF prediction 502. As such, the ECG data 126, or a portion thereof, comprise the ECG input 204 in the implementation 500. The ECG data 126 are fed into the ECG analysis module 130, where the data preprocessor 132 generates the standardized input 210. This standardized input 210 is processed by the at least one deep learning model 134, which includes the CNN 212 with the convolutional layers 214. As described with respect to FIGS. 2 and 3, for instance, these convolutional layers may include multiple blocks that progressively extract features from theFIG. 1 Patents 31 Docket No. : BI-11127-PCTstandardized input 210, such as waveform morphologies, rhythm patterns, and temporal relationships.

[0141] The CNN 212 outputs the ECG feature map 230, representing a high-dimensional abstraction of the input ECG data 126. The ECG feature map 230 is transformed by the flatten layer 232 into the one-dimensional ECG vector 234. The ECG vector 234 is processed through the one or more dense layers 236, which may leam complex non-linear relationships between the extracted features to produce the model output 208.

[0142] In the implementation 500, the model output 208 includes the LVEF prediction 502 derived from the learned ECG features. The LVEF prediction 502 may include an estimated LVEF value representing the percentage of blood pumped out of the left ventricle with each heartbeat. The LVEF prediction 502, for instance, may be output as a continuous value (e.g.. a percentage value) and / or as a categorical classification according to one or more thresholds. As a non-limiting example, the LVEF prediction 502 may categorize left ventricular function as preserved LVEF (e.g., >50%), mildly reduced LVEF (e g., 40-49%), or reduced LVEF (e.g., <40%). In some implementations, the LVEF prediction 502 may further identify severely reduced LVEF (e.g., <30%). Accordingly, the individual may be stratified based on the LVEF prediction 502 compared to established clinical thresholds for cardiac function assessment, which may help guide clinical decision-making, treatment selection, and cardiac risk assessment. The LVEF prediction 502 may assist clinicians in identifying patients with reduced cardiac function who may benefit from heart failure therapies, device interventions, or closer monitoring.

[0143] Having discussed example details of the techniques for electrocardiogram-based deep learning for cardiac prediction, consider now example procedures to illustrate additional aspects of the techniques.Example Procedures

[0144] This section describes example procedures for electrocardiogram-based deep learning for cardiac prediction in one or more implementations. Aspects of the procedures may be implemented in hardware, firmware, or software, or a combination thereof. The procedures are shown as a set of blocks that specify operations performed by one or more devices and are not necessarily limited to the orders shown for performing the operations by the respective blocks. In at least some implementations, at least a portion of the procedure is performed by a suitably configured device, such as the computing device 108 of FIG. 1, by executing instructions stored in a non-transitory computer-readable storage medium.FIG. 1 Patents 32 Docket No. : BI-11127-PCT

[0145] FIG. 6 depicts an example procedure 600 for training and validating a machine learning model to output a heart failure risk prediction according to one or more implementations. Where appropriate, reference will be made to components previously introduced in FIGS. 1-5.

[0146] At least one machine learning model is trained to output a cardiac prediction using a training sample comprising a first portion of a first subset of electrocardiogram training data (block 602). By way of example, the at least one machine learning model may be included in the at least one deep learning model 134 introduced with respect to FIG. 1. In one or more implementations, the at least one machine learning model may include the CNN 212 with multiple convolutional layers 214 for extracting features from standardized ECG inputs. The training module 136 may train the at least one machine learning model using the training sample 148 of the model derivation subset 144 of the training data 140.

[0147] In at least one implementation, the training process includes exposing the at least one machine learning model to a diverse set of ECG data from the model derivation subset 144, allowing the at least one machine learning model to “learn” patterns (e.g., features) in the ECG data that are associated with a specific training task. For instance, the at least one machine learning model “learns” via the adjustment of the weights 222 and biases 224, which may be adjusted through backpropagation 242 based on the calculated loss 240 between the model output 208, e.g., corresponding to the cardiac prediction of the at least one machine learning model for an ECG input (e.g., the ECG input 204 of the training instance 202), and the ground truth labels 206. This iterative process may continue until the at least one machine learning model achieves satisfactory performance on the model derivation subset 144 of the training data 140.

[0148] In at least one implementation, such as depicted in FIG. 4, the at least one machine learning model may be trained to output the cardiac prediction 116, which may include the time to heart failure prediction 402. The at least one machine learning model may be trained to output the time to heart failure prediction 402 as a probability or risk score for the individual developing heart failure at different time bins and / or as a cumulative score for an entire time period examined (e.g., 10 years). As such, the ground truth labels 206 may include known heart failure outcomes for the ECG data in the training data 140. Given this, the model training manager 138 may form a training instance that includes an input portion corresponding to a given ECG and an associated output portion with a heart failure outcome.

[0149] In at least one implementation, the at least one model may be trained to produce the auxiliary predictions 412 in addition to the cardiac prediction 116. The auxiliary predictions 412 may include the all-cause mortality prediction 404, which estimates the risk of death fromFIG. 1 Patents 33 Docket No. : BI-11127-PCTany cause within a specified timeframe; the age prediction 406, which may estimate the biological age of the individual; the sex prediction 408, which may estimate the biological sex of the individual; and the ICD code prediction 410, which may estimate the likelihood of specific ICD codes being included in the individual’s medical health record.

[0150] In at least one variation, such as depicted in FIG. 5, the at least one machine learning model may be trained to output the LVEF prediction 502, which may provide an estimated left ventricular ejection fraction value or classification. The at least one machine learning model may be trained to output the LVEF prediction 502 as a continuous percentage value (e.g., 45%) representing the percentage of blood pumped out of the left ventricle with each heartbeat, and / or as a categorical classification corresponding to established clinical thresholds such as preserved LVEF (>50%), mildly reduced LVEF (40-49%), or reduced LVEF (<40%). As such, the ground truth labels 206 may include known LVEF measurements for the ECG data in the training data 140, which may be obtained from echocardiographic or other clinical assessments performed within a specified time window of the ECG recording. Given this, the model training manager 138 may form a training instance that includes an input portion corresponding to a given ECG and an associated output portion with an LVEF measurement or classification.

[0151] The trained at least one machine learning model is refined using a validation sample comprising a second portion of the first subset of the electrocardiogram training data (block 604). By way of example, the model training manager 138 may use the validation sample 150 to fine-tune the model hyperparameters 228 and adjust the model weights 222 and biases 224. During the refinement process, the at least one machine learning model is exposed to new ECG data from the model derivation subset 144, which is not seen by the at least one machine learning model during the initial training process (e.g., as performed at block 602). This process helps identify and correct any overfitting that may have occurred during the training process. The hyperparameters 228, such as the learning rate, batch size, and / or regularization strength, may be adjusted to achieve a desirable performance on the validation sample 150 while maintaining good generalization to unseen data, for instance.

[0152] The trained and refined at least one machine learning model is internally validated using an internal test sample comprising a third portion of the first subset of the electrocardiogram training data (block 606). By way of example, the model performance may be evaluated on the internal test sample 152 to assess generalization. The internal validation process may provide an initial measure of an ability of the at least one machine learning model to make accurate cardiac predictions on data from a same data source as the training sample 148 and theFIG. 1 Patents 34 Docket No. : BI-11127-PCTvalidation sample 150 (e.g., the model derivation subsetl44) but that was not used during the training (block 602) or refinement (block 604) processes described above.

[0153] During the internal validation process, for instance, outputs of the at least one machine learning model may be compared against the ground truth label 206 for the internal test sample 152. Various performance metrics, such as accuracy, precision, recall, and Fl score for classification tasks, or mean absolute error and root mean squared error for regression tasks, may be calculated. The internal validation process may help identify any potential issues with the performance of the at least one machine learning model and guide further refinement if the performance is not adequate.

[0154] The internally validated at least one machine learning model is externally validated using an external test sample comprising a second subset of the electrocardiogram training data (block 608). By way of example, in response to the at least one machine learning model meeting an acceptable or desired performance criteria, an external validation process may be performed using the external test subset 146. During the external validation process, the external test subset 146 may be input to the internally validated at least one machine learning model. The external test subset 146 may be used to verify that the performance of at least one deep learning model 134 is not specific to the data source of the model derivation subset 144. The external test subset 146, for instance, may comprise ECG data from one or more separate cohorts, often collected from a different institution and / or different patient population than the model derivation subset 144. The external validation process may provide an evaluation of the at least one machine learning model with respect to a real-world scenario and may help identify biases or limitations that were not apparent in the internal validation. The external validation may provide confidence in the ability of the at least one machine learning model to make accurate heart failure risk and / or LVEF predictions across diverse patient populations and clinical settings.

[0155] In this way, the procedure 600 enables generation of at least one machine learning model that is able to output an accurate cardiac prediction 116 from an input ECG. By leveraging the full ECG waveform data, the at least one machine learning model “learns” how to interpret latent information from ECGs that may not be readily interpretable by human observers. The at least one machine learning model may leam to identify complex patterns that go beyond what is physically / manually measurable. These features, captured in the ECG feature map 230, represent a high-dimensional abstraction of the ECG data. By leveraging this information, the at least one machine learning model may leam to make accurate predictionsFIG. 1 Patents 35 Docket No. : BI-11127-PCTabout heart failure risk and / or LVEF that may not be evident from visual inspection or manual measurement performed by a human.

[0156] FIG. 7 depicts an example procedure 700 for generating a cardiac prediction from electrocardiogram data using a machine learning model according to one or more implementations. Where appropriate, reference will be made to components previously introduced in FIGS. 1-6.

[0157] A standardized input is generated for an electrocardiogram (ECG) to be processed by a machine learning model trained to output a cardiac prediction (block 702). By way of example, the data preprocessor 132 may generate the standardized input 210 from the ECG input 204, which corresponds to the ECG data 126 recorded from an individual. The ECG input 204, for instance, may be a single 12-lead ECG. The standardized input 210 may be processed to have consistent dimensions, sampling rate, and other characteristics with respect to other instances of the ECG input 204 to ensure uniform input to the machine learning model. Moreover, data overlays and non-ECG data (e.g., patient information, text) may be removed to generate the standardized input 210.

[0158] Features of the standardized input are extracted via a convolutional neural network of the machine learning model (block 704). By way of example, the machine learning model (e.g., the at least one deep learning model 134) may utilize the CNN 212 to extract features from the standardized input 210. These features may represent various aspects of cardiac electrical activity depicted in the ECG. The particular features extracted depend on the specific task(s) for which the machine learning model is trained. By way of example, during training, if a particular feature helps to minimize the loss 240, the training may reinforce this feature by adjusting the corresponding weights 222 and bias 224. In contrast, a feature that does not contribute to reducing the loss 240 may not be emphasized.

[0159] As such, the CNN 212 may be trained to extract task-relevant features such that the extracted features may vary from model to model of the at least one deep learning model 134. For instance, when the machine learning model is trained to output the heart failure risk prediction (e.g., the time to heart failure prediction 402) as the cardiac prediction 116, the CNN 212 may extract features relevant to predicting heart failure risk. Similarly, when the machine learning model is trained to output the LVEF prediction 502 as the cardiac prediction 116, the CNN 212 may extract features relevant to predicting the LVEF.

[0160] The convolutional neural network outputs an ECG feature map summarizing the features of the standardized input (block 706). By way of example, the CNN 212 may output the ECG feature map 230, which encapsulates the extracted features in a compactFIG. 1 Patents 36 Docket No. : BI-11127-PCTrepresentation. The ECG feature map 230, for instance, may capture both spatial and temporal information to distill the information of the standardized input 210 into a lower-dimensional, meaningful representation that can be used for downstream tasks like classification or regression.

[0161] The machine learning model generates the heart failure risk prediction based on the ECG feature map (block 708). By way of example, the ECG feature map 230 may be processed through the flatten layer 232 to produce the ECG vector 234. The ECG vector 234 may then be fed to one or more dense layers 236, which output the cardiac prediction 116. In at least one implementation, such as depicted in FIG. 4, the one or more dense layers 236 may generate, as the cardiac prediction 116, the time to heart failure prediction 402 based on the ECG vector 234. This prediction may include an estimated timeframe for when the individual may develop heart failure and / or an overall probability or risk score for developing heart failure. In at least one example, the time to heart failure prediction 402 may include the probability of the individual developing heart failure within a ten-year period. The ten-year period may be subdivided into a plurality of time windows, such as a number of years (non-limiting examples of which include five years, two years, and one year), a number of months (non-limiting examples of which include eighteen months, twelve months, ten months, nine months, eight months, seven months, six months, five months, four months, three months, two months, and one month), or a number of days (non-limiting examples of which include 90 days, 72 days, 30 days, 14 days, and 7 days).

[0162] Optionally, the machine learning model generates auxiliary predictions based on the ECG feature map (block 710). By way of example, the auxiliary predictions 412 may include additional outputs derived from the ECG data, such as an all-cause mortality prediction 404, an age prediction 406, a sex prediction 408, and / or an ICD code prediction 410. These auxiliary predictions 412 may provide supplementary information that may be used for clinical decisionmaking and / or quality' control purposes. For example, the age prediction 406 and the sex prediction 408 could serve as a cross-check against patient demographic information, while the ICD code prediction 410 may suggest potential diagnoses for further investigation. By generating these auxiliary predictions 412 alongside the cardiac prediction 116, the machine learning model may offer a more comprehensive analysis of a patient’s cardiovascular health based on the ECG data.

[0163] In this way, the procedure 700 enables use of a machine learning model that is trained to output an accurate cardiac prediction 116 based on ECG data. The electrocardiogram-based deep learning approach for cardiac prediction described herein may be advantageous due to theFIG. 1 Patents 37 Docket No. : BI-11127-PCTcomplex nature of cardiac assessment, such as that used for heart failure diagnosis. Heart failure can manifest through various non-specific symptoms that overlap with other conditions, making early diagnosis challenging. LVEF assessment typically relies on imaging modalities such as echocardiography, but this assessment can be challenging due to factors such as variations in technical expertise, equipment availability, cost, and in the measurements themselves. The ability of the machine learning model to analyze ECG data and provide a cardiac prediction may assist clinicians in identifying at-risk patients before overt clinical heart failure symptoms appear, potentially enabling timely interventions. Moreover, the ability of the machine learning model to generate auxiliary predictions such as all-cause mortality risk, biological age estimation, and ICD code predictions may provide a more comprehensive view of cardiovascular health. This holistic assessment may aid clinicians in patient stratification and tailoring management strategies. As yet another example, the ability of the machine learning model to process and analyze ECG data for the cardiac prediction may provide a standardized and objective assessment tool to complement clinical judgment. This may be useful in cases with atypical presentation or conflicting clinical indicators. The cardiac prediction 116, for instance, may serve as an additional data point for clinicians to consider in their diagnostic process.

[0164] The procedure 700 may also enable clinicians to efficiently assess heart failure risk, which may increase the number of patients who can be screened in a given timeframe. This approach may also enable non-expert providers to obtain clinically relevant information from an ECG. For instance, detailed ECG interpretation often relies on expert analysis, but such experts may be unavailable at the point of care, particularly in resource-limited or timesensitive settings. As such, the procedure 700 may be used to identify' individuals at high risk of heart failure and / or having impaired LVEF as soon as the ECG is completed, enabling enhanced decision support. This may also reduce specialist dependency, enabling specialists to devote more time to higher-level patient care decisions and less time on routine ECG interpretation.

[0165] Having described example procedures in accordance with one or more implementations, consider now an example system and device that can be utilized to implement the various techniques described herein.Example System and Device

[0166] FIG. 8 illustrates an example system generally at 800 that includes an example computing device 802 that is representative of one or more computing systems and / or devicesFIG. 1 Patents 38 Docket No. : BI-11127-PCTthat may implement the various techniques described herein. This is illustrated through inclusion of the computing device 108. The computing device 802 may be, for example, a server of a service provider, a device associated with a client (e.g., a client device), an on-chip system, and / or any other suitable computing device or computing system.

[0167] The example computing device 802 as illustrated includes a processing system 804, one or more computer-readable media 806, and one or more I / O interfaces 808 that are communicatively coupled, one to another. Although not shown, the computing device 802 may further include a system bus or other data and command transfer system that couples the various components, one to another. A system bus can include any one or combination of different bus structures, such as a memory bus or memory controller, a peripheral bus, a universal serial bus, and / or a processor or local bus that utilizes any of a variety of bus architectures. A variety of other examples are also contemplated, such as control and data lines.

[0168] The processing system 804 is representative of functionality to perform one or more operations using hardw are. Accordingly, the processing system 804 is illustrated as including hardware elements 810 that may be configured as processors, functional blocks, and so forth. This may include implementation in hardware as an application specific integrated circuit or other logic device formed using one or more semiconductors. The hardw are elements 810 are not limited by the materials from which they are formed or the processing mechanisms employed therein. For example, processors may be comprised of semiconductor(s) and / or transistors (e.g., electronic integrated circuits (ICs)). In such a context, processor-executable instructions may be electronically executable instructions.

[0169] The computer-readable media 806 is illustrated as including memory / storage 812. The memory / storage 812 represents memory / storage capacity associated with one or more computer-readable media. The memory / storage 812 may include volatile media (such as random-access memory (RAM)) and / or nonvolatile media (such as read only memory (ROM), Flash memory, optical disks, magnetic disks, and so forth). The memory / storage 812 may include fixed media (e.g., RAM, ROM, a fixed hard drive, and so on) as well as removable media (e.g., flash memory, a removable hard drive, an optical disc, and so forth). The computer- readable media 806 may be configured in a variety of other ways as further described below.

[0170] Input / output interface(s) 808 are representative of functionality to allow" a user to enter commands and information to computing device 802, and also allow" information to be presented to the user and / or other components or devices using various input / output devices. Examples of input devices include a keyboard, a cursor control device (e.g., a mouse), a microphone, a scanner, touch functionality (e.g., capacitive or other sensors that are configuredFIG. 1 Patents 39 Docket No. : BI-11127-PCTto detect physical touch), a camera (e.g., which may employ visible or non- visible wavelengths such as infrared frequencies to recognize movement as gestures that do not involve touch), and so forth. Examples of output devices include a display device (e.g., a monitor or projector), speakers, a printer, a network card, tactile-response device, and so forth. Thus, the computing device 802 may be configured in a variety of ways as further described below to support user interaction.

[0171] Various techniques may be described herein in the general context of software, hardware elements, or program modules. Generally, such modules include routines, programs, objects, elements, components, data structures, and so forth that perform particular tasks or implement particular abstract data types. The terms ‘‘module,’' “functionality,’' and “component” as used herein generally represent software, firmware, hardware, or a combination thereof. The features of the techniques described herein are platform-independent, meaning that the techniques may be implemented on a variety of commercial computing platforms having a variety of processors.

[0172] For instance, the terms “module,” “functionality,” and “component” may include a hardware and / or software system that operates to perform one or more functions. For example, a module, functionality, or component may include a computer processor, a controller, or another logic-based device that performs operations based on instructions stored on a tangible and non-transitory computer-readable storage medium, such as a computer memory. Alternatively, a module, functionality, or component may include a hard-wired device that performs operations based on hard-wired logic of the device. Various modules, systems, and components shown in the attached figures may represent the hardware that operates based on software or hardwired instructions, the software that directs hardware to perform the operations, or a combination thereof.

[0173] An implementation of the described modules and techniques may be stored on or transmitted across some form of computer-readable media. The computer-readable media may include a variety of media that may be accessed by the computing device 802. By way of example, and not limitation, computer-readable media may include “computer-readable storage media” and “computer-readable signal media.”

[0174] “Computer-readable storage media” may refer to media and / or devices that enable persistent and / or non-transitory storage of information in contrast to mere signal transmission, carrier waves, or signals per se. Thus, computer-readable storage media refers to non-signal bearing media. The computer-readable storage media includes hardware such as volatile and non-volatile, removable and non-removable media, and / or storage devices implemented in aFIG. 1 Patents 40 Docket No. : BI-11127-PCTmethod or technology suitable for storage of information such as computer readable instructions, data structures, program modules, logic elements / circuits, or other data. Examples of computer-readable storage media may include, but are not limited to, RAM, ROM, EEPROM, flash memory or other memory technology, CD-ROM, digital versatile disks (DVD) or other optical storage, hard disks, magnetic cassettes, magnetic tape, magnetic disk storage or other magnetic storage devices, or other storage device, tangible media, or article of manufacture suitable to store the desired information and which may be accessed by a computer.

[0175] “Computer-readable signal media” may refer to a signal-bearing medium that is configured to transmit instructions to the hardware of the computing device 802, such as via a network. Signal media typically may embody computer readable instructions, data structures, program modules, or other data in a modulated data signal, such as carrier waves, data signals, or other transport mechanism. Signal media also include any information delivery media. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal. By way of example, and not limitation, communication media include wired media such as a wired network or direct-wired connection, and wireless media such as acoustic, RF, infrared, and other wireless media.

[0176] As previously described, hardware elements 810 and computer-readable media 806 are representative of modules, programmable device logic and / or fixed device logic implemented in a hardware form that may be employed in some examples to implement at least some aspects of the techniques described herein, such as to perform one or more instructions. Hardware may include components of an integrated circuit or on-chip system, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), a complex programmable logic device (CPLD), and other implementations in silicon or other hardware. In this context, hardware may operate as a processing device that performs program tasks defined by instructions and / or logic embodied by the hardware as well as a hardware utilized to store instructions for execution, e.g., the computer-readable storage media described previously.

[0177] Combinations of the foregoing may also be employed to implement various techniques described herein. Accordingly, software, hardware, or executable modules may be implemented as one or more instructions and / or logic embodied on some form of computer- readable storage media and / or by one or more hardware elements 810. The computing device 802 may be configured to implement particular instructions and / or functions corresponding to the software and / or hardware modules. Accordingly, implementation of a module that is executable by the computing device 802 as software may be achieved at least partially inFIG. 1 Patents 41 Docket No. : BI-11127-PCThardware, e.g., through use of computer-readable storage media and / or hardware elements 810 of the processing system 804. The instructions and / or functions may be executable / operable by one or more articles of manufacture (for example, one or more computing devices 802 and / or processing systems 804) to implement techniques, modules, and examples described herein.

[0178] The techniques described herein may be supported by various configurations of the computing device 802 and are not limited to the specific examples of the techniques described herein. This functionality may also be implemented all or in part through use of a distributed system, such as over a '‘cloud” 814 via a platform 816 as described below.

[0179] The cloud 814 includes and / or is representative of a platform 816 for resources 818, which are depicted including the computing device 108. The platform 816 abstracts underlying functionality of hardware (e.g., servers) and software resources of the cloud 814. The resources 818 may include applications and / or data that can be utilized while computer processing is executed on servers that are remote from the computing device 802. The resources 818 can also include ser ices provided over the Internet and / or through a subscriber network, such as a cellular or Wi-Fi network.

[0180] The platform 816 may abstract resources and functions to connect the computing device 802 with other computing devices. The platform 816 may also serve to abstract scaling of resources to provide a corresponding level of scale to encountered demand for the resources 818 that are implemented via the platform 816. Accordingly, in an interconnected device example, implementation of functionality described herein may be distributed throughout the system 800. For example, the functionality may be implemented in part on the computing device 802 as well as via the platform 816 that abstracts the functionality of the cloud 814.

[0181] Having discussed example details of the techniques for electrocardiography-based deep learning for cardiac prediction, consider now the following examples to illustrate usage of the techniques.Example ApplicationArtificial Intelligence-Enabled Electrocardiogram Analysis to Predict Incident Heart Failure

[0182] Heart failure (HF) is a significant public health burden affecting over 64 million individuals worldwide. The incidence and prevalence of HF are increasing due to population aging and improved survival rates from related cardiovascular diseases. Despite advances in management, HF remains a leading cause of hospitalization and mortality, underscoring a critical unmet need for early risk stratification to enable preventive measures.FIG. 1 Patents 42 Docket No. : BI-11127-PCT

[0183] Multiple clinical models have been developed to predict incident HF. However, most rely on a number of inputs, including demographics, comorbidities, laboratory results, imaging data, and electrocardiogram values. Even when these data are available, concerns regarding misclassification of clinical variables and the need for cumbersome calculation have limited the uptake of HF risk estimators in real-world clinical care. Furthermore, most models may be derived in highly selected research cohorts, which may limit generalizability. The need for better methods to estimate incident HF risk is highlighted by recent guidelines, which emphasize the effectiveness of emerging therapies to prevent HF events, but do not endorse any specific risk model for identifying at-risk individuals in the community.

[0184] Deep learning approaches have demonstrated the ability to discriminate incident HF risk using the 12-lead electrocardiogram (ECG), a non-invasive, inexpensive, and widely available diagnostic test. However, existing approaches have been limited by modest accuracy (area under the receiving operating characteristic curv e [AUROC] 0.76), inconsistent performance (AUROC 0.72-0.81), and lack of public availability, limiting access and potential clinical implementation. Furthermore, prior models have been developed using diagnostic codes to ascertain HF events, which are prone to misclassification. Contemporary natural language processing (NLP) methods capable of providing clinical endpoint committee-level accuracy for defining HF episodes using free-text data may provide a unique opportunity to assess whether training on a large number of high-fidelity HF events may result in models capable of discriminating future HF risk with favorable and consistent performance across multiple real-world healthcare populations.

[0185] According to the techniques described herein, a deep learning model was developed to predict incident HF using only the 12-lead ECG input, with HF episodes adjudicated using a previously validated natural language processing (NLP) algorithm. Such a model may facilitate accurate and efficient HF risk stratification across multiple large ambulatory care samples.METHODSStudy Samples

[0186] The deep learning-based HF prediction model (Electrocardiogram to Heart Failure, ECG2HF) was trained and validated using samples including patients receiving longitudinal primary and ambulatory' cardiology care within a multi-institutional healthcare network. Patients were aged 18-90 years with two or more office visits spaced between 1-3 years apart in a qualifying clinic between 2000-2019, and all patients were linked to a common electronic health record (EHR). For longitudinal analyses, start of follow-up was defined as the secondFIG. 1 Patents 43 Docket No. : BI-11127-PCToffice visit of the earliest qualifying pair, to allow an ascertainment period for baseline clinical characteristics. This cohort selection process and longitudinal analysis structure may improve data density, minimize missingness, and reduce selection bias compared to other methods of EHR ascertainment.

[0187] ECG2HF was trained on individuals who had one or more ECG performed within three years before the start of follow-up within a first hospital (e.g., Massachusetts General Hospital, MGH). ECG2HF was then evaluated among individuals without prevalent HF in an internal test set not included in model training (MGH Test), and an external test set comprising two ambulatory' care samples from patients receiving care from separate hospitals: 1) Brigham and Women’s Hospital (BWH), and 2) Beth Israel Deaconess Medical Center (BIDMC). There was no overlap of individuals or tracings between the hospitals.

[0188] FIGS. 9 A and 9B illustrate a summary 900 of the training data 140 used for developing the deep learning model for heart failure prediction. By way of example, as show n in FIG. 9A, the training sample 148 included 77,779 individuals, the validation sample 150 included 16,857 individuals, and the internal test sample 152 included 13,954 individuals. As shown in FIG. 9B, the external test subset 146 included 54,457 individuals in the BWH cohort and 25,457 individuals in the BIDMC cohort.Heart Failure Definition

[0189] The outcome was incident HF, defined using the combination of one or more International Classification of Diseases, 9th or 10th revision (ICD 9 / 10) codes for HF, and classification of HF from a transformer-based NLP model trained to detect HF hospitalization using free text discharge summaries. This approach was previously validated with an area under the receiving operating characteristic curve (AUROC) of 0.91 for HF hospitalization compared against expert clinician review utilizing standardized HF trial criteria, providing far more accurate classification of HF versus ICD 9 / 10 code alone. The NLP-based approach provided substantially more accurate classification compared to ICD 9 / 10 codes alone in multiple clinical trials, demonstrating very good agreement (83-87%) and favorable diagnostic test characteristics (e.g., sensitivity’ 94%, specificity 84%) compared with gold-standard clinical endpoint committee adjudications.ECG Acquisition and ECG2HF Architecture

[0190] ECG2HF is a ID convolutional neural netw ork designed to predict 10-year incident HF using only a single 12-lead ECG. For training, ECG2HF was exposed to all 12-lead ECGs performed within the three years prior to the start of follow-up in the MGH training set, and weights were updated based on performance in the validation subset. Since individuals withFIG. 1 Patents 44 Docket No. : BI-11127-PCTprevalent HF may provide particularly rich training signal (e.g., they provide clear examples of HF -related changes on the ECG), individuals with prevalent HF were included in ECG2HF training. Consequently, for training and validation, time to HF was defined from the date of ECG, where prevalent HF was encoded as a time-to-event of zero. ECGs were input as timeseries of 5000 voltage measurements for each of 12 leads, sampled at 500 Hz over 10 seconds (tracings sampled at 250 Hz were linearly upsampled to 500 Hz). To maximize model applicability, no specific filtering or other pre-processing was applied to the waveform prior to analysis.

[0191] ECG2HF utilizes an encoding and loss function explicitly accounting for survival time and censoring. To ensure model evaluation mirrored the intended clinical use case, only a single ECG was used (the tracing most closely preceding the start of follow-up), patients with preexisting HF were excluded, and only individuals aged 30-79 years (e.g., the published age range of prior HF clinical scores) were evaluated.

[0192] ECG2HF was trained to predict a time to a first HF event. Given prior evidence supporting improved performance of deep learning models with inclusion of related tasks as secondary prediction targets, ECG2HF was trained to simultaneously perform prediction of age, sex, time to death (to explicitly model the competing risk of death), and the presence of an incident HF ICD code in the primary position. This design resulted in improved accuracy for HF compared to other task regimes (Table 1 and FIG. 10).FIG. 1 Patents 45 Docket No. : BI-11127-PCTTable 1. Assessment of alternative model inputs and objectives

[0193] FIG. 10 shows an example analysis 1000 of deep learning model performance on auxiliary tasks of age regression and sex classification. The analysis 1000 includes a first graph 1002 showing precision-recall curves comparing model performance for male and female subjects and a second graph 1004 showing an age scatter plot comparing predicted ages to actual ages.

[0194] In the first graph 1002, a first plot 1006 represents female subjects, and a second plot 1008 represents male subjects. The curve for female subjects (the first plot 1006) demonstrates higher precision across most recall values compared to the curve for male subjects (the second plot 1008). Mean precision values for each group are indicated on the graph.

[0195] The second graph 1004 shows a positive correlation between predicted and actual ages, with most points clustered around a diagonal line, which represents perfect prediction. These results suggest that the deep learning model may effectively perform auxiliary tasks such as sex classification and age estimation from ECG data, which may provide additional context forFIG. 1 Patents 46 Docket No. : BI-11127-PCTinterpreting the heart failure risk prediction. The second graph 1004 includes a Pearson correlation with actual age of 0.742.ECG2HF Training

[0196] The input to ECG2HF was a 12-lead ECG in XML format, in which each lead is represented as a vector of relative voltage amplitude over 10 seconds sampled at either 250 or 500 Hz. ECGs sampled at 250 Hz were upsampled to 500 Hz, and ECGs with shorter durations were zero-padded to contain 5000 measurements. This allowed for a uniform input shape. For individuals with multiple ECGs within the baseline window, the ECG used for training at each epoch was randomly selected among all ECGs for that individual.

[0197] Although incident HF can be modeled as a binary classification task, such an approach does not account for censoring and may lead to poor sample-level calibration. To account for this, an encoding and loss function was developed that is aware of both the time until the event (i.e., HF) and missingness introduced by right censoring. The encoding quantizes the total follow-up time into n bins, in this case n = 25, so each time bin spans ~72 days. Each individual is represented by two binary vectors of length n, one representing a binary mask for dates that are censored and the other representing AF event status via one-hot encoding. The censoring vector, VCENSOR, is 0 f°revery time bin in which the individual was censored and 1 for time bins during which they remained in the risk set. The event vector, VHF, is 1 if a HF diagnosis falls within that time bin, otherwise it is 0. ECG2HF emits a vector. VPREDICT, of length n representing HF survival probability at each time bin. The loss function minimizes the negative log likelihood of ECG2HF’s predictions. The likelihood is factored into contributions from time bins survived and time bins with events. The minimized loss function was £ =

[0198] In this way, censored individuals do not contribute to the loss at time bins after they were lost to follow up. The first time bin is reserved to encode events prior to the start of follow up. The model predicts an independent survival probability’ (a sigmoid activation) for each time bin, therefore making no assumptions of proportional hazards and facilitating modeling of discontinuous survival curves. Since probability of mortality' was included as a training task, the competing risk of death is modeled analogously to heart failure.

[0199] ECG2HF maps between two distinct time scales, the 10 second ECG and the 10-year predicted survival curve. This mapping is learned by a 1 -dimensional convolutional neural network (CNN) with ECG input of shape (5000, 12) and output vector of 25 survivalFIG. 1 Patents 47 Docket No. : BI-11127-PCTprobabilities, one for each time bin. Architecture hyperparameters including width, depth, activation, normalization, and regularization were chosen via Bayesian hyperparameter optimization. The best performing architecture contains over 18 million neurons, uses rectified linear unit (ReLU) activations, 16 layers of convolutions, each with a 21 timestep convolutional kernel, dense residual connections, Poincare normalization on convolutional lay ers, max rather than average pooling, and 208 neurons in the fully connected layer. ECG2HF was optimized using rectified Adam stochastic gradient descent, with a learning rate of 2e-4, batch size of 48 and dropout rates of 0.2 for convolutional layers and 0.5 for fully connected layers.Net Reclassification Analyses

[0200] To estimate the effect of using ECG2HF versus PCP-HF for the identification of individuals at particularly high risk for HF, net reclassification improvement (NRI) metrics were calculated. The categorical NRI, case reclassification (NRI+), and non-case reclassification (NRI-) were calculated at the higher ends of the risk distribution for each test set (i.e., <10%, 10-20%, and >20%). Additionally, the continuous (threshold-free) NRI was calculated.Sensitivity Analyses

[0201] Several sensitivity analyses were performed. First, HF discrimination using ECG2HF versus age and sex was analyzed within subgroups of age (thresholds chosen to approximate tertiles of the age distribution), race (White versus Non- White to facilitate sufficient statistical power for each group), QRS duration (greater than or equal to 120 milliseconds versus less than 120 milliseconds), the presence of common cardiac comorbidities (e g., prevalent atrial fibrillation, myocardial infarction, hyperlipidemia, diabetes, or use of anti-hypertensive medications), presence of a pacemaker (available in MGH / BWH only), and whether the ECG had a clinical diagnostic interpretation of "‘normal” by a board-certified cardiologist (available only in MGH / BWH). Second, to assess the degree to which metrics may be affected by HF definition, HF defined as an ICD code in the primary position (rather than NLP) was assessed in MGH Test and BWH Test. Third, the performance of ECG2HF in the MGH / BWH test sets was assessed while restricting ECG acquisition and / or ascertainment of clinical variables to a 1-year window preceding start of cohort follow-up. Fourth, HF discrimination was assessed using the number of ECGs performed during the ECG ascertainment period (e g., within 3 years prior to start of follow-up).Pooled Cohorts Equations to Prevent Heart Failure (PCP-HF)

[0202] ECG2HF was then compared to the validated PCP-HF clinical scores for 10-year risk for incident HF. PCP-HF was selected as a comparator given its analogous design as a HF riskFIG. 1 Patents 48 Docket No. : BI-11127-PCTstratification tool for ambulatory patients without need for cardiac imaging, similar requirement for 12-lead ECG, and previously favorable performance compared to other prediction scores. For this analysis, a complete case approach was utilized. Baseline age, sex, body mass index, diabetes, anti-glycemic medication use, smoking status, systolic blood pressure, and blood pressure medication use were obtained from the EHR. Total cholesterol and HDL-C were ascertained from laboratory results within 3 years prior to the start of followup (and up to 1 year following in BIDMC). QRS duration was taken from the ECG closest to the start of follow-up. Since fasting glucose was not available in either cohort, published equations from the American Diabetes Association were used to convert glycated hemoglobin levels to average blood glucose levels, which were then calibrated to a fasting level. To ensure adequate sample size, the MGH Test and BWH Test samples were combined for the purposes of comparing ECG2HF to PCP-HF.Statistical Analysis

[0203] In each test set, person-time began at start of follow-up and ended at the earliest of an event, death, last encounter, or age 90. Death was ascertained using EHR-based definitions and the Social Security Death Index. In each test set. a 30-day blanking period after the start of follow-up was instituted to reduce misclassification of prevalent cases as potential incident HF events (e.g., events within 30 days of start of follow-up were considered prevalent and therefore excluded from incident analyses).

[0204] The primary metric for discrimination was the time-dependent AUROC at 10 years. Since HF events are relatively uncommon, the time-dependent average precision (AP) was additionally calculated, which provides a complementary' assessment emphasizing the ability to detect high-risk individuals. Analogous to AUROC, AP provides a composite measure of test precision and recall across a range of predictor thresholds. AP values are dependent on the sample-specific event rate, which also serves as the benchmark performance of a completely uninformative model (e.g., a completely uninformative model classifying all individuals as ‘positive' will achieve an average precision equal to the event rate). Therefore, the ratio of a model's AP to the event rate provides a metric of relative enrichment for event risk. Using both metrics, ECG2HF was compared to a Cox proportional hazards model including HF as the outcome and age and sex as covariates was used as a reference comparator. Both AUROC and AP were calculated using inverse probability of censoring weights.

[0205] Calibration of absolute risk estimates w as assessed by plotting absolute versus predicted risks, fitting corresponding smoothed curves using adaptive hazard regression, and calculating the integrated calibration index (ICI), a measure of the average prediction error weighted byFIG. 1 Patents 49 Docket No. : BI-11127-PCTthe empirical risk distribution. The ICI can be interpreted similarly to alternative error metrics such as the time-dependent Brier score (e.g., an error of zero represents perfect calibration), but the ICI more accurately reflects overall model calibration by explicitly incorporating the observed risk distribution. For AP, AUROC, and ICI estimates, 95% confidence intervals were calculated using bootstrapping (500 iterations for AP and AUROC, and 200 iterations for ICI). Models were compared (e g., ECG2HF versus PCP-HF) by quantifying differences in model metrics, with associated 95% confidence intervals for those differences calculated using bootstrapping. Here, a 95% confidence interval of the difference exclusive of zero signifies a difference in model performance significant at a two-sided alpha level of 0.05.

[0206] To estimate the effect of ECG2HF implementation, net reclassification indices (both continuous and categorical at <10%. 10-20%, and >20% 10-year HF risk) were calculated. To assess the robustness of the findings, several sensitivity analyses were also performed, including the assessment of a model combining ECG2HF and PCP-HF probabilities and enforcement of a 1-year window between ECG, clinical factors, and start of follow-up. Analyses were performed using Python v3. 10 and R v4.3.RESULTSBaseline Characteristics

[0207] The individuals from MGH were randomly split into training (70%), validation (15%), and testing (15%) subsets. The training and validation sets together comprise the MGH development set. which includes a total of 94,636 individuals (n = 4.949 heart failure events) with 364,304 ECGs (median ECGs per individual: 2 [quartile-1 : 1 , quartile-3: 4]) Median time between ECG and start of follow-up was 492 days [246, 707], ECG2HF was then evaluated in MGH Test (13,954 individuals, age 57±13 years, 48% women), BWH Test (54,457 individuals, age 57±13 years, 55% women), and BIDMC (25,457 individuals, age 57±13, 53% women) (FIGS. 9A and 9B). Baseline characteristics were generally similar across the test sets, although BIDMC had a relatively higher proportion of Black patients.Discrimination of Incident Heart Failure

[0208] Over 10 years, there were 441 HF events in MGH Test (10-year cumulative risk 4.6% [95% CI 4. 1-5.0]), 1,809 HF events in the BWH Test, and (5.0% [4.8-5.2]), and 901 events in BIDMC (4.4% [4. 1-4.7]). The cumulative risk of death prior to HF event was 7.9% [7.5-8.4] in MGH, 8.0% [7.7-8.2] in BWH, and 7.3% [7.0-7.7] in BIDMC.

[0209] ECG2HF consistently discriminated 10-year HF in each test set according to AUROC (MGH 0.86 [0.84-0.87]; BWH 0.85 [0.84-0.86]; BIDMC 0.84 [0.83-0.86]. p<0.01 versus age and sex for all). AP values demonstrated a consistent ~4-fold average enrichment for HF riskFIG. 1 Patents 50 Docket No. : BI-11127-PCTover the event rate (MGH 0.19 [CI 0.17-0.23] with event rate 0.046; BWH: 0.21 [0.20-0.23] with event rate 0.050, BIDMC 0.20 [0.18-0.22] with event rate 0.044, p<0.01 versus age and sex for all, FIG. 11). The cumulative incidence of HF was higher across quintiles of ECG2HF risk in each test set, particularly for the highest quintile (e.g., HF risk in quintile 5: 17.9% [95% CI 16.9-18.9] vs. quintile 1 : 0.27% [0.14-0.39] in BWH Test, FIG. 12). When compared to the lowest quintile, hazard ratios for HF were substantially greater in the highest quintile (MGH: 57.58 [25.64-129.26], BWH: 76.54 [48.65-120.4], BIDMC 60.55 [34.19-107.2]). Results were similar when ECG2HF risk was divided into tertiles (FIG. 13). Distributions of ECG2HF predicted risk stratified by incident HF status are shown in FIG. 14.

[0210] FIG. 11 depicts a set of graphs 1100 comparing a performance of different heart failure prediction models across three datasets. The set of graphs 1100 includes six individual graphs arranged in a grid. The top row shows time-dependent average precision graphs for MGH Test (a first graph 1102), BWH Test (a second graph 1104), and BIDMC (a third graph 1106) datasets. The bottom row displays time-dependent AUROC (Area Under the Receiver Operating Characteristic) graphs for the same datasets, including a fourth graph 1108 (the MGH Test dataset), a fifth graph 1110 (the BWH Test dataset), and a sixth graph 11 12 (the BIDMC dataset).

[0211] The set of graphs 1100 compares the performance of ECG2HF (open or white-filled circle), age and sex (dark-filled circles) models over different prediction windows. The first graph 1102, the second graph 1104. and the third graph 1106 further include a no skill model (black triangles). The x-axis for all graphs represents the HF prediction window in years, while the y-axis shows the time-dependent average precision for first graph 1102, the second graph 1104, and the third graph 1106 or the time-dependent AUROC for the fourth graph 1108, the fifth graph 1110, and the sixth graph 1112. Error bars are included for each data point to indicate the range of uncertainty.

[0212] The first graph 1102, second graph 1104, and third graph 1106 in the top row demonstrate the time-dependent average precision for the MGH Test, BWH Test, and BIDMC datasets, respectively. These graphs allow for a direct comparison of the models’ precision across different prediction windows and datasets.

[0213] The fourth graph 1108, the fifth graph 1110, and the sixth graph 1112 in the bottom row illustrate the time-dependent AUROC for the same datasets. The AUROC graphs provide insight into the models’ ability to distinguish between individuals who will and will not develop heart failure over different time horizons.FIG. 1 Patents 51 Docket No. : BI-11127-PCT

[0214] Across all graphs in the set of graphs 1100, the ECG2HF model consistently outperforms the age and sex model and the no skill baseline. This superior performance is observed across all datasets and prediction windows, highlighting the robustness and generalizability of the ECG2HF model for heart failure risk prediction.

[0215] FIG. 12 depicts a set of graphs 1200 showing cumulative risk of heart failure over time for different test groups. The set of graphs 1200 includes a first graph 1202, a second graph 1204, and a third graph 1206. The x-axis (e.g., horizontal axis) in each graph represents time in years, while the y-axis (e.g., vertical axis) represents the cumulative risk percentage. The first graph 1202 displays the cumulative risk percentage over 10 years for five quintiles of patients in the MGH Test dataset, with the MGH Test legend indicating 441 heart failure events. The second graph 1204 shows data for the BWH Test dataset, with the BWH Test legend indicating 1,809 heart failure events over a 10-year period. The third graph 1206 presents data for the BIDMC dataset, with the BIDMC legend indicating 901 heart failure events over a 10-year period. Each graph in the set of graphs 1200 shows the cumulative risk increasing over time, with the highest quintile consistently demonstrating the greatest risk across all three test groups.

[0216] FIG. 13 depicts a set of graphs 1300 showing a cumulative risk of heart failure events over time for three different test groups. The set of graphs 1300 includes a first graph 1302, a second graph 1304, and a third graph 1306. The x-axis (e g., horizontal axis) in each graph represents time in years, while the y-axis (e.g., vertical axis) represents the cumulative risk percentage. The first graph 1302, the second graph 1304, and the third graph 1306 show data over a 10-year period. All three graphs in the set of graphs 1300 present the data divided into three tertiles, with tertile 3 consistently showing the highest cumulative risk across all test cohorts. The number of heart failure events for each cohort is indicated on the respective graphs, with 441 events for the MGH Test dataset in the first graph 1302, 1,809 events for the BWH Test dataset in the second graph 1304, and 901 events for the BIDMC dataset in the third graph 1306.

[0217] FIG. 14 depicts a set of graphs 1400 comparing the distribution of predicted heart failure risk between two models, ECG2HF and PCP-HF. The set of graphs 1400 includes an MGH test graph 1402, a BWH test graph 1404, and a BIDMC graph 1406 for the ECG2HF model; and an MGH / BWH graph 1408 and a BIDMC PCP-HF graph 1410 for the PCP-HF model. The MGH / BWH cohort refers to a combined dataset of patients from Massachusetts General Hospital (MGH) and Brigham and Women's Hospital (BWH). Each graph shows two density curves: a first density curve 1412 representing individuals with heart failure events,FIG. 1 Patents 52 Docket No. : BI-11127-PCTand a second density curve 1414 representing individuals without heart failure events. The x- axis in each graph represents the predicted heart failure risk percentage, while the y-axis shows the density of individuals.

[0218] The ECG2HF graphs (the MGH test graph 1402, the BWH test graph 1404, and the BIDMC graph 1406) demonstrate a clearer separation between the first density7curve 1412 and the second density curve 1414 compared to the PCP-HF graphs (the MGH / BWH graph 1408 and the BIDMC PCP-HF graph 1410), which may suggest better discrimination of heart failure risk by the ECG2HF model.Calibration of Heart Failure Risk Estimates

[0219] Median ECG2HF predicted HF risk was similar across MGH Test (10-year estimated HF risk 1.5% [quartile-1: 0.5, quartile-3: 5.2]). BWH Test (1.6% [0.6, 5.5]), and BIDMC (1.6% [0.6, 5.0])). ECG2HF was generally well-calibrated across the majority of the predicted risk distribution, with ICI values reflecting low estimation error. Calibration of the age and sex model was largely similar, although it consistently provided a narrower range of risk estimates around its mean, reflecting more conservative risk estimation (FIG. 15).

[0220] FIG. 15 depicts a set of graphs 1500 showing the calibration of heart failure risk predictions for three different test sets. The set of graphs 1500 includes an MGH test graph 1502, a BWH test graph 1504, and a BIDMC graph 1506. Each graph compares the performance of the ECG2HF model against an age and sex model. In the MGH test graph 1502, a first ECG2HF plot 1508 shows an integrated calibration index of 0.0098. while a first age and sex plot 151 has a value of 0.013. The BWH test graph 1504 shows a second ECG2HF plot 1512 with an integrated calibration index of 0.011 and a second age and sex plot 1514 with a value of 0.0096. The BIDMC graph 1506 presents a third ECG2HF plot 1516 with an integrated calibration index of 0.0088 and a third age and sex plot 1518 with a value of 0. 12. The x-axis of each graph represents the predicted risk of heart failure at ten years (as a percentage), while the y-axis shows the incidence of heart failure at ten years (as a percentage). The integrated calibration index is an estimate of model error, where lower values depict better calibration.Comparison to the PCP-HF Scores

[0221] Comparisons between ECG2HF and the PCP-HF scores were performed in 13,642 individuals of the MGH / BWH sample and 3,044 participants of the BIDMC sample with complete data for PCP-HF calculation. According to AUROC, discrimination for 10-year incident HF was higher using ECG2HF versus PCP-HF in MGH / BWH (0.80 [0.77-0.83] vs. 0.74 [0.70-0.77], difference 0.061 [0.025-0.097]) and comparable in BIDMC (0.80 [0.76-0.84]FIG. 1 Patents 53 Docket No. : BI-11127-PCTvs. 0.76 [0.73-0.80], difference 0.038 [-0.0096-0.086]). Similarly, AP was greater in MGH / BWH (0.26 [0.21-0.31] vs. 0.18 [0.15-0.22] with event rate 0.063, difference 0.079 [0.029-0.14]) and comparable in BIDMC (0.21 [0.16-0.29] vs. 0.17 [0.13-0.22] with event rate 0.064, difference 0.047 [-0.016-0. 12]) (FIG. 16). The incremental value of ECG2HF over PCP- HF was even higher at shorter prediction intervals, with substantially greater discrimination for 2 -year incident HF according to AUROC in both MGH / BWH (0.85 [0.82-0.89] vs. 0.76 [0.72- 0.80], difference 0.091 [0.042-0.14]) and BIDMC (0.88 [0.83-0.92] vs. 0.73 [0.66-0.80]. difference 0.15 [0.081-0.21]). Two-year AP was also substantially higher in MGH / BWH (0.11 [0.076-0.18] vs. 0.036 [0.026-0.055] with event rate 0.010, difference 0.075 [0.042-0.13]), but comparable in BIDMC (0.10 [0.051-0.19] vs. 0.055 [0.027-0.13] with event rate 0.014, difference 0.047 [-0.035-0.13]) (FIG. 16). Predicted risks from ECG2HF and PCP-HF were modestly correlated (MGH / BWH r = 0.37 [95%CI 0.35-0.38], BIDMC r = 0.36 [0.33-0.39]). A model combining ECG2HF and PCP-HF probabilities achieved moderately higher discrimination than either model alone (MGH / BWH AUROC 0.82 [0.79-0.84], AP 0.27 [0.23- 0.32] with event rate 0.063; BIDMC AUROC 0.83 [0.79-0.87], AP 0.24 [0. 19-0.32] with event rate 0.064). Distributions of PCP-HF predicted risk stratified by incident HF status are shown in FIG. 14, as described above.

[0222] Both ECG2HF and PCP-HF were generally well-calibrated, with similarly low estimation error in both MGH / BWH and BIDMC (Table 2). At the high end of the predicted risk distributions for both ECG2HF and PCP-HF, there was some tendency to overestimate risk (FIG. 16).

[0223] ECG2HF provided favorable reclassification over PCP-HF in MGH / BWH (net reclassification improvement [NRI] 0.16 [95%CI 0.077-0.24]), which was driven by both increased sensitivity (case reclassification [NRI+] 13.7% [5.4-20.7]) and increased specificity (non-case reclassification [NRI-] 2.3% [1.3-3.3]). Continuous NRI was also favorable (0.27 [0.16-0.39]). Similar reclassification improvement was observed in BIDMC (NRI 0.23 [0.10- 0.35], NRI+ 21.0% [9.1-33.2], NRI- -1.7% [-0.40 to 3.7], continuous NRI 0.23 [0.061-0.39]). (Table 2).FIG. 1 Patents 54 Docket No. : BI-11127-PCTTable 2. Net reclassification of incident heart failure using ECG2HF versus PCP-HF

[0224] FIG. 16 depicts a set of graphs 1600 comparing the performance of electrocardiogrambased deep learning models for heart failure prediction. The set of graphs 1600 includes six individual graphs arranged in three rows. A first graph 1602 and a second graph 1604 in the top row show time-dependent AUROC for MGH / BWH Test and BIDMC datasets, respectively, where the ECG2HF model corresponds to open or white-filled circles, and the PCP-HF model corresponds to dark-filled circles. The x-axis represents the prediction window in years, while the y-axis shows the AUROC values. A third graph 1606 and a fourth graph 1 08 in the middle row display time-dependent average precision for the same datasets and further include a no skill model (black triangles). The x-axis again represents the prediction window in years, while the y-axis shows the average precision values. The bottom row contains a fifth graph 1610 and a sixth graph 1612, which compare the calibration of ECG2HF and PCP- HF models. For the fifth graph 1610 and the sixth graph 1612, the x-axis represents predicted risk, while the y-axis shows observed risk. In the fifth graph 1610, a first ECG2HF plot 1614 shows an integrated calibration index of 0.015, while a first PCP-HF plot 1616 has an integrated calibration index of 0.016. The sixth graph 1612 displays a second ECG2HF plot 1618 having an integrated calibration index of 0.018, and a second PCP-HF plot 1620 having an integratedFIG. 1 Patents 55 Docket No. : BI-11127-PCTcalibration index of 0.012. The graphs demonstrate the comparative performance of the ECG2HF and PCP-HF models across different metrics and datasets.ECG2HF Saliency and Behavior

[0225] Median waveform analysis demonstrated that individuals at high predicted risk of HF tended to have wider QRS complexes with lower amplitude P wave and longer PR interval (FIG. 17). Saliency maps demonstrated that the ECG P wave, QRS complex, and T wave appeared to have the greatest influence on ECG2HF risk predictions. The mean (standard deviation) QRS duration was consistently longer among individuals with incident HF versus without incident HF (MGH: 100.2 milliseconds (ms) [27.4] vs. 91.5 ms [16.1]; BWH: 107.2 ms [30.4] vs. 91.5 ms [17.4]; BIDMC: 101.0 ms [22.8] vs. 91.4 ms [14.4], p < 0.01 for all). When analyzed across strata of ECG2HF predicted risk, saliency appeared somewhat more distributed across the waveform for individuals at high predicted risk versus low predicted risk.

[0226] FIG. 17 depicts a set of ECG graphs 1700 showing median waveforms for multiple leads of electrocardiogram data. The ECG graphs 1700 are arranged in a 3^4 grid, representing the standard 12-lead ECG configuration (I, IE III. aVR. aVL. aVF, VI. V2, V3, V4, V5. V6). Each graph in the set displays two waveforms: a first waveform 1702 corresponding to low ECG2HF predicted risk, and a second waveform 1704 corresponding to high ECG2HF predicted risk. The first waveform 1702 and the second waveform 1704 are overlaid in each graph, allowing for direct comparison of their shapes and amplitudes across different leads. The x-axis of each graph represents time in milliseconds, while the y-axis represents voltage in millivolts. The first waveform 1702 and the second waveform 1704 show distinct differences in morphology.Sensitivity Analyses

[0227] ECG2HF discrimination was similar when HF was defined using an ICD code in the primary position (Table 3). Individuals within higher strata of ECG2HF risk were generally older and had higher rates of HF-related comorbidity. At the same time, ECG2HF retained substantial discrimination among individuals without common cardiac comorbidities (FIGS. 18 and 19), where an ECG2HF probability in the top quintile remained associated with elevated risk of incident HF (FIG. 20). The performance of both ECG2HF and PCP-HF was largely similar when the baseline ascertainment window was restricted to one year preceding start of follow-up in MGH / BWH. Consistent discrimination of incident HF using ECG2HF was observed across subgroups of age. with even greater relative value over age and sex amongst the youngest subgroups (e.g., AUROC among age <45 in MGH: 0.83 [95%CI 0.75-0.92] vs. 0.54 [0.41-0.66], difference 0.30 [0.16-0.43]; BWH: 0.90 [0.88-0.93] vs. 0.55 [0.50-0.61],FIG. 1 Patents 56 Docket No. : BI-11127-PCTdifference 0.35 [0.29-0.40]; BIDMC 0.83 [0.76-0.89] vs. 0.55 [0.46-0.64], difference 0.28 [0.18-0.38]) (FIGS. 21 and 22). ECG2HF discrimination was also generally consistent across ECGs read as normal versus abnormal (FIGS. 23 and 24) and having a narrow versus wide QRS duration (FIGS. 25 and 26). Discrimination persisted across strata of race (FIGS. 27 and 28), the presence or absence of a pacemaker (FIG. 29), and the presence or absence of atrial fibrillation in the clinical diagnostic read of the ECG used for HF risk inference (FIG. 30), although AUROC was moderately lower among individuals with a pacemaker and among tracings showing atrial fibrillation. The number of ECGs performed during the ECG ascertainment period (e.g., within 3 years prior to start of follow-up) was generally a weak discriminator of 10-year incident HF (e.g., AUROC (MGH: 0.68 [0.65-0.71], BWH: 0.73[0.72-0.74], BIDMC 0.69 [0.67-0.71]).Table 3. Performance of ECG2HF for incident HF defined using natural language processing versus ICD code in primary position

[0228] FIG. 18 depicts a set of bar graphs 1800 comparing the area under the receiver operating characteristic curve (AUROC) for heart failure prediction across different comorbidity statuses and datasets. The set of bar graphs 1800 is divided into three sections representing data from the MGH Test dataset, the BWH Test dataset, and the BIDMC dataset. A first bar graph 1802, a second bar graph 1804, and a third bar graph 1806 show results for the MGH Test dataset. A fourth bar graph 1808, a fifth bar graph 1810, and a sixth bar graph 1812 show results for the BWH Test dataset. A seventh bar graph 1814, an eighth bar graph 1816, and a ninth bar graph 1818 show results for the BIDMC dataset. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different categories including overall performance, performance in patients with no comorbidities, and performance in patients with comorbidities. The comorbidities includeFIG. 1 Patents 57 Docket No. : BI-11127-PCTprevalent atrial fibrillation, myocardial infarction, hyperlipidemia, diabetes, or use of antihypertensive medications. Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to a 95% confidence interval. The set of bar graphs 1800 demonstrates variations in AUROC across different comorbidity statuses and datasets, with the ECG2HF model generally showing higher AUROC values compared to the age and sex model across all categories and test sets. Values represent 10-year incident heart failure discrimination among individuals with common cardiac comorbidities versus without common cardiac comorbidities.

[0229] FIG. 19 depicts a set of bar graphs 1900 comparing average precision for heart failure prediction across different comorbidity statuses and datasets. The set of bar graphs 1900 is divided into three sections representing data from the MGH Test dataset, the BWH Test dataset, and the BIDMC dataset. A first bar graph 1902, a second bar graph 1904, and a third bar graph 1906 show results for the MGH Test dataset. A fourth bar graph 1908, a fifth bar graph 1910, and a sixth bar graph 1912 show results for the BWH Test dataset. A seventh bar graph 1914, an eighth bar graph 1916, and a ninth bar graph 1918 show results for the BIDMC dataset. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different categories including overall performance, performance in patients with no comorbidities, and performance in patients with comorbidities. The comorbidities include prevalent atrial fibrillation, myocardial infarction, hyperlipidemia, diabetes, or use of anti-hypertensive medications. Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to a 95% confidence interval, where arrows reflect extension beyond the plot area. The set of bar graphs 1900 demonstrates variations in average precision across different comorbidity statuses and datasets, with the ECG2HF model generally showing higher average precision values compared to the age and sex model across all categories and test sets. Values represent the time-dependent average precision for 10-y ear incident heart failure discrimination among individuals with common cardiac comorbidities versus without common cardiac comorbidities.

[0230] FIG. 20 depicts an example 2000 of cumulative risk curves for heart failure events over time across three test datasets among individuals without common cardiac comorbidities. The example 2000 includes an MGH test graph 2002, a BWH test graph 2004, and a BIDMC graph 2006, each displaying cumulative risk percentage on the y-axis and time in years on the x-axis. The common cardiac comorbidities include prevalent atrial fibrillation, myocardial infarction, hyperlipidemia, diabetes, or use of anti-hypertensive medications. Each graph in the exampleFIG. 1 Patents 58 Docket No. : BI-11127-PCT2000 shows two curves: a high-risk plot 2008 representing individuals in the highest quintile of ECG2HF predicted risk, and a non-high risk plot 2010 representing individuals in the lower quintiles of ECG2HF predicted risk. The MGH test graph 2002 shows cumulative risk data over a 10-year period. The BWH test graph 2004 displays cumulative risk data over a 10-year period, with the high-risk plot 2008 showing substantially higher cumulative risk compared to the non-high risk plot 2010. The BIDMC graph 2006 presents cumulative risk data over a 10- year period, similarly demonstrating separation between the high-risk plot 2008 and the non- high risk plot 2010. The high-risk plot 2008 in each graph shows a steeper increase in cumulative risk over time compared to the non-high risk plot 2010, which remains relatively flat near zero percent cumulative risk across all three test datasets. The example 2000 demonstrates the cumulative risk of incident heart failure across strata of ECG2HF predicted heart failure risk among individuals without common cardiac comorbidities, where high risk is defined as an ECG2HF probability in the sample-specific top quintile.

[0231] FIG. 21 depicts a set ofbar graphs 2100 comparing the area under the receiver operating characteristic curve (AUROC) for heart failure prediction across different age groups and datasets. The set of bar graphs 2100 is divided into three sections representing data from the MGH Test dataset, the BWH Test dataset, and the BIDMC dataset. A first bar graph 2102, a second bar graph 2104, a third bar graph 2106, and a fourth bar graph 2108 show results for the MGH Test dataset. A fifth bar graph 2110, a sixth bar graph 2112, a seventh bar graph 2114, and an eighth bar graph 2116 show results for the BWH Test dataset. A ninth bar graph 21 18, a tenth bar graph 2120, an eleventh bar graph 2122, and a twelfth bar graph 2124 show results for the BIDMC dataset. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different age ranges. Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to a 95% confidence interval. The graph demonstrates variations in AUROC across different age groups and datasets, with generally higher AUROC values for the ECG2HF model compared to the age and sex model, particularly in younger age groups. Values represent 10-year incident heart failure.

[0232] FIG. 22 depicts a set of bar graphs 2200 comparing average precision for heart failure prediction across different age groups and datasets. The set of bar graphs 2200 is divided into three sections representing data from the MGH Test dataset, the BWH Test dataset, and the BIDMC dataset. A first bar graph 2202, a second bar graph 2204, a third bar graph 2206, and a fourth bar graph 2208 show results for the MGH Test dataset. A fifth bar graph 2210. a sixth bar graph 2212, a seventh bar graph 2214, and an eighth bar graph 2216 show results for theFIG. 1 Patents 59 Docket No. : BI-11127-PCTBWH Test dataset. A ninth bar graph 2218, a tenth bar graph 2220, an eleventh bar graph 2222, and a twelfth bar graph 2224 show results for the BIDMC dataset. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different age ranges, while the y-axis represents average precision. Error bars are included on top of each bar to indicate the range of uncertainty in the measurements. Bars indicate 95% confidence intervals, where arrows reflect extension beyond the plot area. The set of bar graphs 2200 demonstrates variations in average precision across different age groups and datasets, with generally higher precision values for older age groups in all three datasets. Values represent 10-year incident heart failure.

[0233] FIG. 23 depicts a set of bar graphs 2300 comparing the area under the receiver operating characteristic curve (AUROC) for heart failure prediction across different ECG types and datasets. The set of bar graphs 2300 is divided into two sections representing data from the MGH Test dataset and the BWH Test dataset. A first bar graph 2302, a second bar graph 2304, and a third bar graph 2306 show results for the MGH Test dataset. A fourth bar graph 2308, a fifth bar graph 2310, and a sixth bar graph 2312 show results for the BWH Test dataset. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different ECG types (overall, normal ECG, and abnormal ECG), while the y-axis represents AUROC values. Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to the 95% confidence interval. The set of bar graphs 2300 demonstrates variations in AUROC across different ECG types and datasets, with different patterns of performance between the ECG2HF model and the age and sex model for normal and abnormal ECGs. Values represent 10-year incident heart failure in both MGH Test and BWH Test datasets. The graph illustrates the comparative performance of the ECG2HF model and the age and sex model in predicting heart failure risk across various ECG classifications and test sets.

[0234] FIG. 24 depicts a set of bar graphs 2400 comparing average precision for heart failure prediction across different ECG types and datasets. The set of bar graphs 2400 is divided into two sections representing data from the MGH Test dataset and the BWH Test dataset. A first bar graph 2402, a second bar graph 2404, and a third bar graph 2406 show results for the MGH Test dataset. A fourth bar graph 2408, a fifth bar graph 2410, and a sixth bar graph 2412 show results for the BWH Test dataset. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different ECG types (overall, normal ECG. and abnormal ECG), while the y-axis represents average precision. Error bars are included on top of each bar to indicate the range of uncertaintyFIG. 1 Patents 60 Docket No. : BI-11127-PCTin the measurements, corresponding to a 95% confidence interval. The set of bar graphs 2400 demonstrate variations in average precision across different ECG types and datasets, with different patterns of performance between the ECG2HF model and the age and sex model for normal and abnormal ECGs. Values represent 10-year incident heart failure in both MGH Test and BWH Test datasets.

[0235] FIG. 25 depicts a set of graphs 2500 comparing the area under the receiver operating characteristic curve (AUROC) of electrocardiogram-based deep learning for heart failure prediction across different test sets and QRS duration categories. The set of graphs 2500 is divided into three sections representing data from the MGH Test dataset, the BWH Test dataset, and the BIDMC dataset. A first bar graph 2502, a second bar graph 2504, and a third bar graph 2506 show results for the MGH Test dataset. A fourth bar graph 2508, a fifth bar graph 2510, and a sixth bar graph 2512 show results for the BWH Test dataset. A seventh bar graph 2514, an eighth bar graph 2516, and a ninth bar graph 2518 show results for the BIDMC dataset. In each pair of bars, dark outlined bars represent the ECG2HF model, and lighter gray, nonoutlined bars represent the age and sex model. The x-axis for each bar graph shows different QRS duration categories (overall, narrow QRS, and wide QRS), while the y-axis represents AUROC values. Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to a 95% confidence interval. The set of graphs 2500 demonstrates variations in the AUROC across different QRS duration categories and datasets, with different patterns of performance between the ECG2HF model and the age and sex model for narrow and wide QRS durations. Values represent 10-year incident heart failure.

[0236] FIG. 26 depicts a set of graphs 2600 comparing the average precision of electrocardiogram-based deep learning for heart failure prediction across different test sets and QRS duration categories. The set of graphs 2600 is divided into three sections representing data from the MGH Test dataset, the BWH Test dataset, and the BIDMC dataset. A first bar graph 2602, a second bar graph 2604, and a third bar graph 2606 show results for the MGH Test dataset. A fourth bar graph 2608, a fifth bar graph 2610, and a sixth bar graph 2612 show results for the BWH Test dataset. A seventh bar graph 2614, an eighth bar graph 2616, and a ninth bar graph 2618 show results for the BIDMC dataset. In each pair of bars, dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph show's different QRS duration categories (overall, narrow QRS, and wide QRS), while the y-axis represents average precision. Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to a 95% confidence interval. The set of graphs 2600 demonstrates variations inFIG. 1 Patents 61 Docket No. : BI-11127-PCTperformance across different QRS duration categories and datasets, with different patterns of performance between the ECG2HF model and the age and sex model for narrow and wide QRS durations. Values represent 10-year incident heart failure.

[0237] FIG. 27 depicts a set of bar graphs 2700 comparing the area under the receiver operating characteristic curve (AUROC) for heart failure prediction across different racial groups and datasets. The set of bar graphs 2700 is divided into two sections representing data from the MGH / BWH Test dataset and the BIDMC dataset. A first bar graph 2702, a second bar graph 2704, and a third bar graph 2706 show results for the MGH / BWH Test dataset. A fourth bar graph 2708, a fifth bar graph 2710, and a sixth bar graph 2712 show results for the BIDMC dataset. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different racial categories including overall performance (the first bar graph 2702 and the fourth bar graph 2708), white race (the second bar graph 2704 and the fifth bar graph 2710), and non-white race (the third bar graph 2706 and the sixth bar graph 2712). Race was categorized as white or non-white, given insufficient sample size to examine more granular categories. The MGH Test and BWH Test sets were combined to maximize precision of estimates in the Non-White race category. Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to a 95% confidence interval. The set of bar graphs 2700 demonstrates variations in AUROC across different racial groups and datasets, with the ECG2HF model generally showing higher AUROC values compared to the age and sex model across white race and non-white race categories. Values represent the time-dependent AUROC for 10-year incident heart failure discrimination across subgroups of race.

[0238] FIG. 28 depicts a set of bar graphs 2800 comparing average precision for heart failure prediction across different racial groups and datasets. The set of bar graphs 2800 is divided into two sections representing data from the MGH / BWH Test dataset and the BIDMC dataset. A first bar graph 2802, a second bar graph 2804, and a third bar graph 2806 show results for the MGH / BWH Test dataset. A fourth bar graph 2808, a fifth bar graph 2810, and a sixth bar graph 2812 show results for the BIDMC dataset. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different racial categories including overall performance (the first bar graph 2802 and the fourth bar graph 2808), white race (the second bar graph 2804 and the fifth bar graph 2810), and non-white race (the third bar graph 2806 and the sixth bar graph 2812). Race was categorized as white or non-white, given insufficient sample size to examine more granular categories. The MGH Test and BWH Test sets were combined to maximize precision ofFIG. 1 Patents 62 Docket No. : BI-11127-PCTestimates in the Non- White race category. Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to a 95% confidence interval. The set of bar graphs 2800 demonstrates variations in average precision across different racial groups and datasets, with the ECG2HF model generally showing higher average precision values compared to the age and sex model across white race and non-white race categories. Values represent the time-dependent average precision for 10-year incident heart failure discrimination across subgroups of race.

[0239] FIG. 29 depicts a set of bar graphs 2900 comparing model performance metrics for heart failure prediction across different pacemaker status categories. The set of bar graphs 2900 is divided into two rows, with a first bar graph 2902, a second bar graph 2904, and a third bar graph 2906 showing area under the receiver operating characteristic curve results. A fourth bar graph 2908, a fifth bar graph 2910, and a sixth bar graph 2912 show the average precision results. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different pacemaker status categories including overall performance (the first bar graph 2902 and the fourth bar graph 2908), individuals without a prevalent pacemaker (the second bar graph 2904 and the fifth bar graph 2910), and individuals with a prevalent pacemaker (the third bar graph 2906 and the sixth bar graph 2912). The MGH Test and BWH Test sets were combined to maximize precision of estimates in the pacemaker category. The analysis includes individuals with a prevalent pacemaker (n = 1.965) and without a prevalent pacemaker (n = 66,354). Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to a 95% confidence interval. The set of bar graphs 2900 demonstrates variations in performance between individuals with and without pacemakers, with the ECG2HF model generally showing higher performance values compared to the age and sex model across the pacemaker status categories.

[0240] FIG. 30 depicts a set of bar graphs 3000 comparing model performance metrics for heart failure prediction across different atrial fibrillation status categories. The set of bar graphs 3000 is divided into two rows, with a first bar graph 3002, a second bar graph 3004. and a third bar graph 3006 showing area under the receiver operating characteristic curve results. A fourth bar graph 3008, a fifth bar graph 3010, and a sixth bar graph 3012 show the average precision results. Dark outlined bars represent the ECG2HF model, and lighter gray, non-outlined bars represent the age and sex model. The x-axis for each bar graph shows different atrial fibrillation status categories including overall performance (the first bar graph 3002 and the fourth bar graph 3008), individuals without atrial fibrillation in the ECG diagnostic read (the second barFIG. 1 Patents 63 Docket No. : BI-11127-PCTgraph 3004 and the fifth bar graph 3010), and individuals with atrial fibrillation in the ECG diagnostic read (the third bar graph 3006 and the sixth bar graph 3012). The MGH Test and BWH Test sets were combined to maximize precision of estimates in the atrial fibrillation category. The analysis includes individuals with atrial fibrillation in the ECG diagnostic read (n = 1,659) and without atrial fibrillation in the ECG diagnostic read (n = 66,660). Error bars are included on top of each bar to indicate the range of uncertainty in the measurements, corresponding to a 95% confidence interval. The set of bar graphs 3000 demonstrates variations in performance between individuals with and without atrial fibrillation in the ECG diagnostic read used for ECG2HF inference, with the ECG2HF model generally showing higher performance values compared to the age and sex model across the atrial fibrillation status categories.DISCUSSION

[0241] The Example demonstrates that estimation of 10-year incident HF using a single 12- lead ECG is feasible using ECG2HF, a convolutional neural network. ECG2HF had consistent predictive utility across three test sets comprising nearly 100,000 ambulatory patients without heart failure. Among individuals with available data, ECG2HF offered favorable performance compared to PCP-HF, a validated 15-component clinical risk tool. These findings suggest that ECG2HF may offer an opportunity to perform accurate and efficient HF risk stratification in the context of ambulatory care, which is particularly relevant given the growing recognition of the potential of contemporary HF therapies available to prevent HF hospitalizations among those at risk, including sodium-glucose transport protein 2 (SGLT2) inhibitors, glucagon-like peptide-1 (GLP-1) receptor agonists, and non-steroidal mineralocorticoid receptor antagonists.

[0242] The Example supports and extends prior observations demonstrating the utility of deep learning models for detection and prediction of HF. Several ECG-based Al models have demonstrated the ability to classify the presence of left ventricular dysfunction. More recent efforts focused specifically on the prediction of future heart failure events have demonstrated moderate discrimination or inconsistent performance across test samples, while lack of available model code or weights has limited accessibility and potential clinical application. Furthermore, prior models have been developed using diagnostic codes to ascertain HF events, which are prone to misclassification and may limit model performance. ECG2HF extends prior work by leveraging model training in over 90,000 individuals to predict incident HF defined using a validated NLP model capable of adjudicating HF with clinical endpoint committeelevel accuracy, resulting in a freely available ECG-based model delivering robust andFIG. 1 Patents 64 Docket No. : BI-11127-PCTconsistent 10-year HF prediction across nearly 100,000 individuals spanning three distinct ambulatory healthcare samples.

[0243] The Example suggests that the 12-lead ECG contains sufficient information to stratify risk of future HF. Using only a single 12-lead ECG, ECG2HF estimated 10-year HF risk with substantially greater discrimination than age and sex, and with consistent performance across multiple test sets and absolute risk estimates that were generally well-calibrated. Direct comparisons to prior models are not possible as they are not publicly available, and numeric contrasts to published metrics may be misleading due to differences in study design. Nevertheless, the results extend previous findings by demonstrating highly consistent discrimination of 10-year incident HF using ECG2HF in multiple ambulatory healthcare populations (AUROC 0.84-0.86, AP 0.19-0.21 representing ~4-fold increase over event rate). Good performance of ECG2HF likely stems from two design factors: a) close alignment between model training and the downstream use case (e.g., development of a model specifically to predict future HF using an architecture that accounts for survival time and censoring), and b) application of a multiply validated NLP model demonstrating consistently more accurate ascertainment of HF events compared to ICD codes. ECG2HF risk estimates appear to depend on clinically plausible aspects of the ECG waveform, such as the terminal portion of the P wave and the early portion of the QRS, suggesting roles of left atrial depolarization and early ventricular activation on incident HF risk and providing important evidence of clinical plausibility. Future work is warranted to better understand the mechanisms by which ECG2HF can predict incident HF events, and the extent to which such mechanisms may overlap with models focusing solely on left ventricular systolic dysfunction.

[0244] The Example suggests that deep learning can identify individuals at high risk for HF with favorable performance compared to traditional risk factor scores. ECG2HF provided generally favorable discrimination performance compared to PCP-HF, a validated 15- component clinical risk score. Compared to PCP-HF, a validated 15-component clinical risk score, ECG2HF offered greater discrimination of HF risk according to multiple metrics, including average precision, which connotes a particular ability to identify the highest risk patients. Reclassification analyses also supported the ability to detect high HF risk with comparable or greater specificity. Most of the added value of ECG2HF over PCP-HF was observed in the prediction of shorter-term HF events (e.g., 2 years), which may be more clinically actionable. Greater discrimination of shorter-term events has been shown previously using ECG-based Al models and may highlight a specific role for screening focused on shortterm events, and the potential value of updating HF risk estimates overtime using serial ECGs.FIG. 1 Patents 65 Docket No. : BI-11127-PCTA greater incremental benefit of ECG analysis over clinical variables for shorter time windows may reflect a cumulative effect of clinical factors over time (e.g., age. diabetes) or suggest that ECG data become relatively less informative beyond 2 years. A model combining ECG2HF and PCP-HF probabilities achieved moderately greater discrimination over either model alone. Future work is warranted to assess whether combination models capable of providing HF risk estimates based on varying data types (including clinical variables when available) may improve risk estimation further by leveraging complementary information. The correlation between ECG2HF and PCP-HF was modest, suggesting that ECG2HF is not merely inferring the presence of traditional risk factors. Indeed, ECG2HF continued to discriminate HF risk among individuals with known cardiovascular risk factors as well as ECGs read as non-normal. ECG2HF may encode subtle ECG patterns indicative of structural or functional parameters such as cardiac filling pressures or degrees of electrical dyssynchrony. The abi 1 i ty to refine HF risk even among individuals with known risk factors is consistent with guidelines on the management of HF, which recommend risk stratification among both people at risk for HF and those with established structural heart disease.

[0245] ECG2HF may enable efficient prioritization of individuals for HF-focused preventive interventions in an ambulatory care setting. Although risk factor scores like PCP-HF and others are widely available, model complexity7and potential misclassification of inputs have limited clinical implementation and guideline endorsement. Conversely, biomarkers such as NTproBNP possess predictive value for heart failure but are challenging to use for ambulatory screening given need for blood sample collection. To this end, ECG2HF possesses a substantial advantage by offering risk estimation using an inexpensive diagnostic test routinely performed in the clinic and obtainable within seconds. In the MGH and BWH ambulatory7care samples, individuals in the highest quintile of ECG2HF had a 10-year risk of incident HF approaching 20%. The ability to identify such individuals may provide the opportunity for intensive modification of risk factors (e.g., blood pressure, weight, lipid levels), or targeted testing (e.g., echocardiography, exercise testing) to identify undiagnosed structural or functional abnormalities. Given evolving medical therapies that not only treat but may prevent HF, future studies are warranted to assess whether early administration of agents such as SGLT2 inhibitors in individuals at highest risk may improve outcomes.

[0246] In this w ay, ECG2HF is a deep learning model that predicts 10-year risk of incident HF using only a single 12-lead ECG. ECG2HF provided consistent performance across three test sets including nearly 100,000 individuals, and offered favorable predictive utility compared to the validated PCP-HF clinical risk scores. ECG2HF may facilitate automated and efficient HFFIG. 1 Patents 66 Docket No. : BI-11127-PCTrisk stratification at scale, to prioritize susceptible individuals for potential preventive interventions.Conclusion

[0247] Although the invention has been described in language specific to structural features and / or methodological acts, it is to be understood that the invention defined in the appended claims is not necessarily limited to the specific features or acts described. Rather, the specific features and acts are disclosed as example forms of implementing the claimed invention.

[0248] Features described above as well as those claimed below may be combined in various ways without departing from the scope hereof. The following disclosed items illustrate some possible, non-limiting combinations:

[0249] (Al) A system for electrocardiogram-based cardiac prediction, comprising: an electrocardiogram analysis module implemented in a transitory or non-transitory computer- readable storage medium, the electrocardiogram analysis module comprising: a data preprocessor configured to normalize an electrocardiogram to generate a standardized input for the electrocardiogram-based cardiac prediction; and a deep learning model including a neural network and at least one dense layer, the deep learning model trained to: identify, via the neural network, features associated with a cardiac condition that reflect underlying cardiac changes that result from or predispose development of the cardiac condition; and generate, via the at least one dense layer, a cardiac prediction based on the identified features.

[0250] For example, as described herein, the deep learning model may be configured and / or trained to generate the cardiac prediction as an output.

[0251] The system may comprise one or more processors configured to perform operations, the operations comprising using a deep learning model to generate a standardized input for an input electrocardiogram, the deep learning model comprising a convolutional neural network and at least one dense layer; and using the deep learning model to generate a cardiac prediction based on the standardized input. Optionally, as described herein, the input electrocardiogram may comprise a signal obtained by a plurality of leads. Further optionally, the signal obtained by each of the plurality of leads may comprise a sequence of voltage measurements obtained during a single acquisition process.

[0252] In this way, information is extracted from the input electrocardiogram, including, optionally, from portions that have previously been considered suboptimal or non-diagnostic for human analysis or which are not typically used during a manual clinical analysis workflow for the cardiac condition. By leveraging this information, as described in greater detailFIG. 1 Patents 67 Docket No. : BI-11127-PCTelsewhere herein, the cardiac predictions generated by the deep learning model are made more accurate.

[0253] (A2) For the system denoted as (Al), the neural network comprises a convolutional neural network.

[0254] (A3) For the system denoted as (A2), the convolutional neural network comprises multiple convolutional blocks, each convolutional block including a plurality of convolutional layers, and wherein outputs of at least two convolutional layers of the plurality of convolutional layers are combined via a concatenation.

[0255] (A4) For the system denoted as any one of (Al) through (A3), the cardiac condition is heart failure, and the cardiac prediction comprises a probability score indicating a likelihood of developing heart failure within a period of time.

[0256] (A5) For the system denoted as any one of (Al) through (A4), the cardiac prediction further comprises a plurality' of probability scores indicating respective likelihoods of developing heart failure for different time bins within the period of time.

[0257] (A6) For the system denoted as any one of (Al) through (A5). the at least one dense layer is further trained to generate auxiliary predictions based on the identified features, the auxiliary predictions comprising at least one of an all-cause mortality prediction, an age prediction, a sex prediction, or a diagnostic code prediction.

[0258] (A7) For the system denoted as any one of (Al) through (A6), the cardiac condition is a low left ventricular ejection fraction, and the cardiac prediction comprises a left ventricular ejection fraction estimate.

[0259] (A8) For the system denoted as any one of (Al) through (A7), the data preprocessor is further configured to upsample and zero-pad the electrocardiogram to generate the standardized input for the electrocardiogram-based cardiac prediction.

[0260] (A9) For the system denoted as any one of (Al ) through (A8), wherein the standardized input comprises full electrocardiogram waveform data as a time-series of voltage measurements for each lead of the electrocardiogram, wherein the time-series of voltage measurements are sampled at a standardized frequency over a predetermined time period.FIG. 1 Patents 68 Docket No. : BI-11127-PCT

[0261] (A10) For the system denoted as any one of (Al) through (A9), the system further comprises a training module configured to: train the deep learning model using a training sample comprising a first portion of a first subset of electrocardiogram training data, the electrocardiogram training data comprising electrocardiograms obtained from healthy patients and from patients diagnosed with heart failure; and refine the trained deep learning model using a validation sample comprising a second portion of the first subset of the electrocardiogram training data.

[0262] (Al l) For the system denoted as (A10), the training module is further configured to internally validate the trained and refined deep learning model using an internal test sample comprising a third portion of the first subset of the electrocardiogram training data.

[0263] (Al 2) For the system denoted as (A10) or (Al l), the training module is further configured to externally validate the deep learning model using an external test sample comprising a second subset of the electrocardiogram training data, wherein the first subset of the electrocardiogram training data and the second subset of the electrocardiogram training data comprise electrocardiogram recordings from different data sources.

[0264] There are several technical effects of the system for electrocardiogram-based cardiac prediction of the present disclosure. First, the integration of the convolutional neural network with multiple convolutional blocks enables the extraction of complex features from ECG waveforms that are not readily interpretable by human observers, which may improve the accuracy of the cardiac prediction compared to traditional clinical assessment methods. The data preprocessor provides standardized input processing to facilitate consistent model performance across different ECG acquisition systems and recording conditions, which enhances the generalizability' of the deep learning model across diverse healthcare settings. Additionally, the multi-task learning architecture that simultaneously generates the cardiac prediction and the auxiliary predictions leverages shared representations from the ECG data, which improves overall model performance while providing additional clinical value through, for example, age prediction, sex prediction, and mortality prediction capabilities. The electrocardiogram-based deep learning system for cardiac prediction offers a scalable solution that can process ECG data in real-time during clinical workflows, reducing the time required for cardiovascular risk assessment from hours or days to seconds. Furthermore, a survival analysis framework with explicit censoring handling enables accurate time-to-event predictions across multiple time bins, providing clinicians with both short-term and long-term risk stratification capabilities. The automated nature of the ECG analysis module reduces dependency on specialist interpretation while maintaining high accuracy, which allowsFIG. 1 Patents 69 Docket No. : BI-11127-PCThealthcare providers to efficiently screen large patient populations for heart failure risk. These technical advantages collectively contribute to a comprehensive solution for cardiovascular risk assessment that is both clinically effective and computationally efficient, representing a significant advancement in the application of artificial intelligence to electrocardiographic analysis.

[0265] (B 1) A method for generating a cardiac prediction, said method comprising: generating, by a data preprocessor, a standardized input for an electrocardiogram that is to be processed by a deep learning model trained to output the cardiac prediction; extracting, by a neural network of the deep learning model, features of the standardized input; outputting, by the neural network, an electrocardiogram feature map representing a high-dimensional abstraction of the features of the standardized input; and generating, by an output layer of the deep learning model, the cardiac prediction based at least in part on the electrocardiogram feature map, wherein the cardiac prediction comprises a prediction associated with developing heart failure.

[0266] (B2) For the method denoted as (Bl), generating the standardized input comprises at least one of normalizing, upsampling, or zero-padding the electrocardiogram, and wherein the standardized input comprises a time-series of voltage measurements for each lead of the electrocardiogram that are sampled at a predetermined frequency over a predetermined time period.

[0267] (B3) For the method denoted as (Bl) or (B2), wherein the neural network comprises a convolutional neural network having a plurality of convolutional blocks, each convolutional block of the plurality of convolutional blocks comprising at least one convolutional layer and at least one concatenation, and wherein the output layer comprises one or more dense layers.

[0268] (B4) For the method denoted as any one of (Bl) through (B3), wherein the prediction associated with developing heart failure is at least one probability score indicating a likelihood of developing heart failure within a specified time frame.FIG. 1 Patents 70 Docket No. : BI-11127-PCT

[0269] (Cl) A method for a cardiac prediction associated with heart failure, comprising: training a deep learning model to output the cardiac prediction, the training comprising: initially training the deep learning model using a training sample subset of a first portion of electrocardiogram training data by adjusting weights and biases of the deep learning model based on a difference between an output of the deep learning model for the cardiac prediction and a ground truth label; and refining the initially trained deep learning model using a validation sample subset of the first portion of the electrocardiogram training data, the refining including adjusting hyperparameters of the deep learning model; and generating the cardiac prediction for an individual using the trained deep learning model, the generating comprising: generating, by a data preprocessor operatively connected to the trained deep learning model, a standardized input of an electrocardiogram obtained from the individual by preprocessing the electrocardiogram; extracting, by the trained deep learning model, features of the standardized input; outputting, by at least a portion of the deep learning model, an electrocardiogram feature map summarizing the features of the standardized input; and generating the cardiac prediction based on the electrocardiogram feature map.

[0270] (C2) For the method denoted as (Cl), the training further comprises: internally validating the refined deep learning model using an internal test sample subset of the first portion of the electrocardiogram training data; and externally validating the internally validated deep learning model using an external test sample comprising a second portion of the electrocardiogram training data.

[0271] (C3) For the method denoted as (Cl ) or (C2), the cardiac prediction comprises at least one probability score indicating a likelihood of the individual having or developing heart failure within a specified time period.

[0272] (C4) For the method denoted as any one of (Cl) through (C3). the cardiac prediction comprises an estimate of left ventricular ejection fraction.

[0273] (DI) A method for electrocardiogram-based cardiac prediction using a deep learning model including a neural network and at least one dense layer, the method comprising: normalizing, via a data preprocessor, an electrocardiogram to generate a standardized input for the electrocardiogram-based cardiac prediction; and identifying, via the neural network, features associated with a cardiac condition that reflect underlying cardiac changes that result from or predispose development of the cardiac condition; and generating, via the at least one dense layer, the cardiac prediction based on the identified features.

[0274] (D2) The method denoted as (DI), further comprising training the at least one dense layer to generate auxiliary predictions based on the identified features, the auxiliary predictionsFIG. 1 Patents 71 Docket No. : BI-11127-PCTcomprising at least one of an all-cause mortality prediction, an age prediction, a sex prediction, or a diagnostic code prediction.

[0275] (D3) The method denoted as (DI) or (D2), further comprising upsampling and zeropadding the electrocardiogram to generate the standardized input for electrocardiogram-based cardiac prediction.

[0276] (D4) The method denoted as any one of (DI) through (D3), further comprising: training the deep learning model using a training sample comprising a first portion of a first subset of electrocardiogram training data, the electrocardiogram training data comprising electrocardiograms obtained from healthy patients and from patients diagnosed with heart failure; and refining the trained deep learning model using a validation sample comprising a second portion of the first subset of the electrocardiogram training data.

[0277] (D5) The method denoted as any one of (DI) through (D4), further comprising internally validating the trained and refined deep learning model using an internal test sample comprising a third portion of the first subset of the electrocardiogram training data.

[0278] (El) A computer-readable storage medium comprising instructions which, when executed, cause one or more processors to perform a method as defined in any preceding item, or any of the methods or method steps described herein.

[0279] (Fl) A system comprising the computer-readable storage medium denoted as (El), and the one or more processors.

[0280] (Gl) A computer program comprising instructions which, when the program is executed by one or more computing devices, cause the one or more computing devices to perform a method as defined in any preceding item, or any of the methods or method steps described herein.FIG. 1 Patents 72 Docket No. : BI-11127-PCT

Claims

CLAIMSWhat is claimed is:

1. A system for electrocardiogram-based cardiac prediction, comprising: an electrocardiogram analysis module implemented in a non-transitory computer- readable storage medium, the electrocardiogram analysis module comprising: a data preprocessor configured to normalize an electrocardiogram to generate a standardized input for the electrocardiogram-based cardiac prediction; and a deep learning model including a neural network and at least one dense layer, the deep learning model trained to: identify, via the neural network, features associated with a cardiac condition that reflect underlying cardiac changes that result from or predispose development of the cardiac condition; and generate, via the at least one dense layer, a cardiac prediction based on the identified features.

2. The system of claim 1, wherein the neural network comprises a convolutional neural network.

3. The system of claim 2, wherein the convolutional neural network comprises multiple convolutional blocks, each convolutional block including a plurality of convolutional layers, and wherein outputs of at least two convolutional layers of the plurality of convolutional layers are combined via a concatenation.

4. The system of any one of claims 1 to 3, wherein the cardiac condition is heart failure, and the cardiac prediction comprises a probability score indicating a likelihood of developing heart failure within a period of time.

5. The system of claim 4, wherein the cardiac prediction further comprises a plurality of probability scores indicating respective likelihoods of developing heart failure for different time bins within the period of time.FIG. 1 Patents 73 Docket No. : BI-11127-PCT6. The system of any one of claims 1 to 5, wherein the at least one dense layer is further trained to generate auxiliary predictions based on the identified features, the auxiliary predictions comprising at least one of: an all-cause mortality prediction, an age prediction, a sex prediction, or a diagnostic code prediction.

7. The system of any one of claims 1 to 6, wherein the cardiac condition is a low left ventricular ejection fraction, and the cardiac prediction comprises a left ventricular ejection fraction estimate.

8. The system of any one of claims 1 to 7. wherein the data preprocessor is further configured to upsample and zero-pad the electrocardiogram to generate the standardized input for the electrocardiogram-based cardiac prediction.

9. The system of any one of claims 1 to 8, wherein the standardized input comprises full electrocardiogram waveform data as a time-series of voltage measurements for each lead of the electrocardiogram, wherein the time-series of voltage measurements are sampled at a standardized frequency over a predetermined time period.

10. The system of any one of claims 1 to 9, further comprising a training module configured to: train the deep learning model using a training sample comprising a first portion of a first subset of electrocardiogram training data, the electrocardiogram training data comprising electrocardiograms obtained from healthy patients and from patients diagnosed with heart failure; and refine the trained deep learning model using a validation sample comprising a second portion of the first subset of the electrocardiogram training data.

11. The system of claim 10, wherein the training module is further configured to internally validate the trained and refined deep learning model using an internal test sample comprising a third portion of the first subset of the electrocardiogram training data.FIG. 1 Patents 74 Docket No. : BI-11127-PCT12. The system of claim 10 or claim 11, wherein the training module is further configured to externally validate the deep learning model using an external test sample comprising a second subset of the electrocardiogram training data, wherein the first subset of the electrocardiogram training data and the second subset of the electrocardiogram training data comprise electrocardiogram recordings from different data sources.

13. A method for generating a cardiac prediction, said method comprising: generating, by a data preprocessor, a standardized input for an electrocardiogram that is to be processed by a deep learning model trained to output the cardiac prediction; extracting, by a neural network of the deep learning model, features of the standardized input; outputting, by the neural network, an electrocardiogram feature map representing a high-dimensional abstraction of the features of the standardized input; and generating, by an output layer of the deep learning model, the cardiac prediction based at least in part on the electrocardiogram feature map, wherein the cardiac prediction comprises a prediction associated with developing heart failure.

14. The method of claim 13, wherein generating the standardized input comprises at least one of normalizing, upsampling, or zero-padding the electrocardiogram, and wherein the standardized input comprises a time-series of voltage measurements for each lead of the electrocardiogram that are sampled at a predetermined frequency over a predetermined time period.

15. The method of claim 13 or claim 14, wherein the neural network comprises a convolutional neural network having a plurality of convolutional blocks, each convolutional block of the plurality of convolutional blocks comprising at least one convolutional layer and at least one concatenation, and wherein the output layer comprises one or more dense layers.

16. The method of any one of claims 13 to 15, wherein the prediction associated with developing heart failure is at least one probability score indicating a likelihood of developing heart failure within a specified time frame.FIG. 1 Patents 75 Docket No. : BI-11127-PCT17. A method for a cardiac prediction associated with heart failure, comprising: training a deep learning model to output the cardiac prediction, the training comprising: initially training the deep learning model using a training sample subset of a first portion of electrocardiogram training data by adjusting weights and biases of the deep learning model based on a difference between an output of the deep learning model for the cardiac prediction and a ground truth label; and refining the initially trained deep learning model using a validation sample subset of the first portion of the electrocardiogram training data, the refining including adjusting hyperparameters of the deep learning model; and generating the cardiac prediction for an individual using the trained deep learning model, the generating comprising: generating, by a data preprocessor operatively connected to the trained deep learning model, a standardized input of an electrocardiogram obtained from the individual by preprocessing the electrocardiogram; extracting, by the trained deep learning model, features of the standardized input; outputting, by at least a portion of the deep learning model, an electrocardiogram feature map summarizing the features of the standardized input; and generating the cardiac prediction based on the electrocardiogram feature map.

18. The method of claim 17, wherein the training further comprises: internally validating the refined deep learning model using an internal test sample subset of the first portion of the electrocardiogram training data; and externally validating the internally validated deep learning model using an external test sample comprising a second portion of the electrocardiogram training data.

19. The method of claim 17 or claim 18, wherein the cardiac prediction comprises at least one probability score indicating a likelihood of the individual having or developing heart failure within a specified time period.

20. The method of any one of claims 17 to 19, wherein the cardiac prediction comprises an estimate of left ventricular ejection fraction.FIG. 1 Patents 76 Docket No. : BI-11127-PCT

Images