A multi-modal video-based progression score for aortic stenosis using artificial intelligence

Abstract

Provided herein is a method of stratifying the risk of cardiovascular disease development and progression using multi-modal video-based modeling. The method includes providing a training set including a plurality of imaging sequences from normal subjects and subjects with the condition of interest, such as aortic stenosis, and training a machine-learning model to detect phenotypes associated with distinct severity levels of the condition and predict the risk of disease development or progression. Also provided herein is a computer-implemented method of detecting a heart condition using the machine learning model across distinct imaging modalities and an apparatus for executing the computer-implemented method.

Description

[0001] A MULTI-MODAL VIDEO-BASED PROGRESSION SCORE FOR AORTIC STENOSIS USING ARTIFICIAL INTELLIGENCE

[0002] CROSS-REFERENCE TO RELATED APPLICATIONS

[0003] The present application claims priority under 35 U.S.C. § 119(e) to U.S. Provisional Patent Application No. 63 / 619,241, filed January 9, 2024, which application is incorporated herein by reference in its entirety.

[0004] BACKGROUND OF THE INVENTION

[0005] With the expanding availability of transcatheter and surgical aortic valve replacement (AVR) procedures that effectively modify the natural history of aortic stenosis (AS), focus has shifted to the timely identification of patients at earlier stages of the disease who are at risk of rapid progression and worse clinical outcomes. Unfortunately, efforts to improve risk stratification of early AS have been limited by an incomplete understanding of the specific drivers of disease progression in each individual. Furthermore, longitudinal monitoring requires comprehensive Doppler echocardiography which may not be a cost-effective strategy for the large numbers of patients living with early aortic sclerosis or stenosis. Other risk factors, such as hypertension, hypercholesterolemia and diabetes lack specificity for AS.

[0006] Accordingly, there is a need in the art for articles and methods that improve on existing articles and methods for identifying AS and other heart conditions. The present invention addresses this need.

[0007] SUMMARY

[0008] In one aspect, a computer-implemented method of training a machine-learning model for multi-modal progression scores includes providing a training set including a plurality of imaging sequences from normal subjects and subjects with a heart condition; and training a machinelearning model to detect the heart condition on the training set. In some embodiments, the normal subjects include subjects without the heart condition or with a lower severity heart condition. In some embodiments, the heart condition includes cardiomyopathy or valvular disease. In some embodiments, the heart condition comprises aortic stenosis. In some embodiments, the normal subjects include at least one of: subjects without aortic stenosis, subjects with aortic sclerosis but no stenosis, subjects with mild aortic stenosis, or subjects with moderate aortic stenosis. In some embodiments, the subjects with the heart condition are divided into subjects with non-severe aortic stenosis and subjects with severe aortic stenosis. In some embodiments, the subjects with non-severe aortic stenosis include subjects with mild aortic stenosis, mild-moderate aortic stenosis, moderate, and moderate-severe aortic stenosis.

[0009] In some embodiments, the training comprises supervised learning with or without pretraining by self-supervised learning (SSL). In some embodiments, the SSL comprises pretexts forcing the machine-learning model to learn generalizable temporal and cardiac function-related features, the cardiac function-related features including at least one of a multi-instance contrastive learning task, a sequence re-ordering pretext task, a cross-modal contrastive learning task spanning videos, images of cardiac function, anatomy, and electrical activity, and a biometric learning task that identifies different videos drawn from the same subject to define subtle features from the different videos.

[0010] In another aspect, a computer-implemented method of detecting a heart condition includes receiving an input including a plurality of imaging sequences, each imaging sequence from the plurality of imaging sequences associated with a user from a plurality of users; extracting, for each imaging sequence, a clip from a plurality of clips; and executing the machine-learning model according to any of the embodiments disclosed herein using the plurality of clips to produce a phenotype score for a set of features associated with the heart condition and with a user from the plurality of users, the phenotype score used to generate a digital biomarker.

[0011] In some embodiments, the plurality of imaging sequences in the training set comprise a first input type; the plurality of imaging sequences associated with the user comprise a second input type; and the method further includes transforming the second input type to include an imaging view that resembles an imaging view from the first input type.

[0012] In some embodiments, the phenotype score indicates at least one of a level of severity or non-existence of aortic stenosis, the phenotype score further correlating with at least one of: traditional Doppler parameters of aortic stenosis presence, severity, and parameters of progressive diastolic dysfunction. In some embodiments, the phenotype score is a numerical score ranging from 0 to 1. In some embodiments, the phenotype score includes a multi-instance contrastive learning task, a sequence re-ordering pretext task, a cross-modal contrastive learning task spanning videos, images of cardiac function, anatomy, and electrical activity, or a biometric learning task that identifies different videos drawn from the same subject to define subtle features from the different videos.

[0013] In some embodiments, the machine-learning model includes at least one of: a supervised machine-learning model, an unsupervised machine-learning model, and a self-supervised machine-learning model. In some embodiments, the machine-learning model is trained using a training set that includes a plurality of imaging sequences correlated to a plurality of anatomical features of the heart condition. In some embodiments, the machine-learning model is an ensemble machine-learning model.

[0014] In some embodiments, the machine-learning model is configured to discern existence of the heart condition and produce digital biomarkers that predict severity of the heart condition, predict the development of the heart condition in patients without the heart condition, and predict progression of the heart condition in patients with a non-severe heart condition. In some embodiments, the machine learning model uses traditional cardiovascular risk factors to determine, based on the digital biomarker, an integrated risk score of aortic stenosis presence, future development, progression risk to guide appropriate clinical follow-up, or combinations thereof. In some embodiments, the machine-learning model is trained such that the machinelearning is configured to produce the plurality of digital biomarkers via zero-shot predictions. In some embodiments, executing the machine-learning model to produce the digital biomarkers includes executing the machine-learning model to produce the digital biomarkers from a plurality of digital biomarkers.

[0015] In another aspect, a computer-implemented method of detecting a heart condition includes receiving a first input including a plurality of imaging sequences, each imaging sequence from the plurality of imaging sequences associated with a user from a first plurality of users; extracting, for each imaging sequence, a clip from a first plurality of clips; executing a machine-learning model using the first plurality of clips to produce phenotype score for a set of features that is associated with the heart condition and with a user from the first plurality of users; receiving a second input including a plurality of datasets, each dataset from the plurality of datasets associated with a heart of a user from a second plurality of users; extracting, for each dataset, imaging data from a plurality of imaging data; executing the machine-learning model using the plurality of imaging data as an input to identify, for each imaging data from the plurality of imaging data, a plurality of imaging views with respect to a standardized view of the heart for that imaging data; compiling, for each imaging data, the plurality of imaging views to produce a compiled image sequence from a plurality of compiled image sequences; transforming each compiled image sequence from the plurality of compiled image sequences to generate a clip from a second plurality of clips, the clip from the second plurality of clips including an imaging view that resembles an imaging view from the first input; and refining the phenotype score based on a comparison between the second plurality of clips and the first plurality of clips to generate a digital biomarker that is used to detect the heart condition based on the set of features associated with the heart condition.

[0016] In some embodiments, each of the receiving, extracting, identifying, compiling, transforming, and comparing, is performed automatically. In some embodiments, the compiled image sequence is a cine video or still frame. In some embodiments, transforming each compiled image sequence includes rotating, cropping to a cardiac outline, and inverting to grayscale, that compiled image sequence to produce the clip from the second plurality of clips. In some embodiments, the method further includes confirming accuracy of the digital biomarker without Doppler imaging for phenotyping of aortic stenosis. In some embodiments, the method further includes training the machine-learning model using a training set that includes a plurality of imaging sequences correlated to a plurality of anatomical features associated with the heart condition.

[0017] In some embodiments, the digital biomarker is configured to stratify a risk of development of the heart condition based on inputs of data types that are different from those of the first input and the second input. In some embodiments, the computer-implemented method is executed on a desktop application, a cloud service, or a mobile device.

[0018] In another aspect, an apparatus includes a processor and a memory operatively coupled to the processor, the memory storing instructions to cause the processor to receive a first input including a plurality of imaging sequences, each imaging sequence from the plurality of imaging sequences associated with a user from a first plurality of users; extract, for each imaging sequence, a clip from a first plurality of clips; execute a machine-learning model using the first plurality of clips to produce phenotype score for a set of features that is associated with a heart condition and with a user from the first plurality of users; receive a second input including a plurality of datasets, each dataset from the plurality of datasets associated with a heart of a user from a second plurality of users; extract, for each dataset, imaging data from a plurality of imaging data; execute the machine-learning model using the plurality of imaging data as an input to identify, for each imaging data from the plurality of imaging data, a plurality of imaging views with respect to a standardized view of the heart for that imaging data; compile, for each imaging data, the plurality of imaging views to produce a compiled image sequence from a plurality of compiled image sequences; transform each compiled image sequence from the plurality of compiled image sequences to generate a clip from a second plurality of clips, the clip from the second plurality of clips including an imaging view that resembles an imaging from the first input; and refine the phenotype score based on a comparison between the second plurality of clips and the first plurality of clips to generate a digital biomarker that is used to detect a heart condition based on the set of features associated with the heart condition.

[0019] BRIEF DESCRIPTION OF THE DRAWINGS

[0020] For a fuller understanding of the nature and desired objects of the present invention, reference is made to the following detailed description taken in conjunction with the accompanying drawing figures wherein like reference characters denote corresponding parts throughout the several views.

[0021] FIG. l is a block diagram of a system for a multi-modal progression score for aortic stenosis, according to some embodiments.

[0022] FIG. 2 is a flow diagram of a method for training a machine-learning model for detecting aortic stenosis, according to some embodiments.

[0023] FIG. 3 is a flow diagram of a method for executing a machine-learning model to produce a phenotype score for a medical condition, according to some embodiments.

[0024] FIG. 4 is a flow diagram of a method for a producing a multi-modal progression score to generate a digital biomarker, according to some embodiments.

[0025] FIG. 5 is an illustrative diagram of process for producing a digital biomarker from a phenotype score using echocardiography data, according to some embodiments.

[0026] FIG. 6 is an illustrative diagram of a process for cross-modal validation of a phenotype score using MRI videos, according to some embodiments. FIG. 7 is an illustrative diagram of graphs depicting changes in the aortic valve, according to some embodiments.

[0027] FIG. 8 is an illustrative diagram depicting cross-modal validation of a phenotype score using MRI videos, according to some embodiments.

[0028] FIG. 9 is an illustrative diagram depicting saliency maps with phenotypic associations, according to some embodiments.

[0029] DETAILED DESCRIPTION

[0030] Definitions

[0031] The instant invention is most clearly understood with reference to the following definitions.

[0032] As used herein, the singular form “a,” “an,” and “the” include plural references unless the context clearly dictates otherwise.

[0033] Unless specifically stated or obvious from context, as used herein, the term “about” is understood as within a range of normal tolerance in the art, for example within 2 standard deviations of the mean. “About” can be understood as within 10%, 9%, 8%, 7%, 6%, 5%, 4%, 3%, 2%, 1%, 0.5%, 0.1%, 0.05%, or 0.01% of the stated value. Unless otherwise clear from context, all numerical values provided herein are modified by the term about.

[0034] As used in the specification and claims, the terms “comprises,” “comprising,” “containing,” “having,” and the like can have the meaning ascribed to them in U.S. patent law and can mean “includes,” “including,” and the like.

[0035] Unless specifically stated or obvious from context, the term “or,” as used herein, is understood to be inclusive.

[0036] Ranges provided herein are understood to be shorthand for all of the values within the range. For example, a range of 1 to 50 is understood to include any number, combination of numbers, or sub-range from the group consisting 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, or 50 (as well as fractions thereof unless the context clearly dictates otherwise). Detailed Description

[0037] Provided herein is a multi-modal approach to predict the progression risk of a heart condition using artificial intelligence (Al) algorithms applied to cardiovascular images and / or videos. The heart condition can include any suitable heart condition, such as, but not limited to, aortic valve stenosis (aortic stenosis, AS), aortic valve sclerosis, aortic valve calcification, aortic valve thickening or aortic valve degeneration. For example, in some embodiments the method includes predicting the risk of aortic stenosis progression using artificial intelligence algorithms applied to extreme phenotypes on echocardiographic videos that generalize to cardiovascular magnetic resonance through a novel cross-modal pipeline.

[0038] In some embodiments, as illustrated in FIG. 2, a method 200 of training a machinelearning model for detecting a heart condition includes providing a training set including a plurality of imaging sequences (205), and training the machine-learning model to detect the heart condition on the training set (210). In some implementations, the machine-learning model includes at least one of: a supervised machine-learning model, an unsupervised machine-learning model, and a self-supervised machine-learning model. For example, suitable machine-learning models include, but are not limited to, at least one of a deep neural network model (DNN), an artificial neural network (ANN) model, a fully connected neural network, a convolutional neural network (CNN), a residual network model, a region proposal network (RPN) model, a feature pyramid network (FPN) model, a generative adversarial network (GAN), a transformer-based model, a K-Nearest Neighbors (KNN) model, a Support Vector Machine (SVM), a decision tree, a random forest, an analysis of variation (ANOVA), boosting, a Naive Bayes classifier, and / or the like.

[0039] In some embodiments, the machine-learning model includes an ensemble machinelearning model. In some embodiments, the ensemble model includes an ensemble of multiple machine-learning models with different initializations. For example, in some embodiments, the ensemble model includes an ensemble of three models with a combination of three initializations (e.g., random, Kinetics-400, and self-supervised learning). In some such embodiments, the output from the ensemble model includes an average of the study-level predictions from the individual models that form the ensemble.

[0040] The training set includes any suitable images for training the machine-learning model, such as, but not limited to, imaging sequences from normal subjects and subjects with a medical condition (e.g., heart condition). In some embodiments, the training set includes a plurality of imaging sequences correlated to a plurality of anatomical features of the heart condition. Suitable heart conditions include, but are not limited to, aortic stenosis, cardiomyopathy, valvular disease, and / or the like. In some embodiments, for example, the heart condition is aortic stenosis. In some instances, the normal subjects can include subjects without the heart condition or with a lower severity heart condition. For example, in some cases, the normal subjects can include at least one of: subjects without aortic stenosis, subjects with aortic sclerosis but no stenosis, subjects with mild aortic stenosis, or subjects with moderate aortic stenosis. In some cases, subjects with the heart condition can be divided into subjects with non-severe aortic stenosis and subjects with severe aortic stenosis. In some cases, subjects with non-severe aortic stenosis can include subjects with mild aortic stenosis, mild-moderate aortic stenosis, moderate, and moderate-severe aortic stenosis.

[0041] In some embodiments, the training includes supervised learning with or without pretraining by self-supervised learning (SSL). In some cases, SSL can include pretexts forcing the machine-learning model to learn generalizable temporal and cardiac function-related features. In some cases, the pretexts for the cardiac function-related features can include, for example, at least one of: a multi-instance contrastive learning task, a sequence re-ordering pretext task, a cross-modal contrastive learning task spanning videos, images of cardiac function, anatomy, and electrical activity, a biometric learning task that identifies different videos drawn from the same subject to define subtle features from the different videos, and / or the like. In some embodiments, the pretexts include identification of a medical condition (e.g., heart condition) in the training set.

[0042] In some embodiments, the machine-learning model includes a set of model parameters such as weights, biases, or activation functions that can be executed to annotate and / or classify features of an organ in an image. In some embodiments, the training phase includes inputting the training set to the machine-learning model and refining the set of model parameters of the machine-learning model. The set of model parameters are refined such that features in images and / or videos in the training set can be annotated and / or classified correctly with a certain likelihood of correctness (e.g., a pre-set likelihood of correctness).

[0043] In some instances, the training set can be randomly divided into a training subset and a testing subset. For example, the training set can be randomly divided into 60% training subset and 40% testing subset. Although discussed herein with respect to a specific division of the training set, as will be appreciated by those skilled in the art, the disclosure is not so limited and may include any other suitable division between the training and testing subsets. In such embodiments, the machine-learning model can be iteratively refined based on the training subset while being tested on the testing subset set to avoid overfitting and / or underfitting of the training set. Once the machine-learning model is trained based on the training set, a performance of the machine-learning model can be further verified.

[0044] In some embodiments, following the training phase, the trained machine-learning model is configured to detect the medical condition in one or more datasets from a subject. For example, in some embodiments, the trained machine-learning model is configured to discern the existence of the heart condition and produce digital biomarkers that predict the severity of the heart condition, predict the development of the heart condition in patients without the heart condition, and / or predict the progression of the heart condition in patients with a non-severe heart condition. In some cases, the machine learning model can use traditional cardiovascular risk factors to determine, based on the digital biomarker, an integrated risk score of aortic stenosis presence, future development, and / or progression risk to guide appropriate clinical follow-up. In some cases, the machine-learning model can be trained such that the machinelearning is configured to produce the plurality of digital biomarkers via zero-shot predictions.

[0045] In some embodiments, the digital biomarkers (e.g., imaging biomarkers) are confined within a single modality. Alternatively, in some embodiments, the machine-learning model is trained to detect pertinent anatomical and functional features with modality-invariance. In some such embodiments, during training, the machine-learning model learns key representations of the disease, rather than technical confounders specific to the acquisition process to generate biomarkers that provide prognostic value across two or more modalities. Additionally or alternatively, in some embodiments, the generation of multi-modality biomarkers enables zeroshot predictions in new cohorts and clinical settings and / or augments the ability to opportunistically risk stratify for AS using existing data streams.

[0046] Also provided herein are methods of detecting a medical condition using the trained machine-learning model according to any of the embodiments disclosed here. In such embodiments, at least one dataset from at least one subject is input to the trained machinelearning model, and the machine-learning model annotates and / or classifies the at least one dataset with respect to the medical condition. In some embodiments, the trained machine learning model generates a phenotype score based upon the annotating and / or classifying of the at least one dataset. In some cases, the phenotype score can be a numerical score ranging from 0 to 1. In some cases, the phenotype score can be a Digital AS Severity index (DASSi). In some implementations, the phenotype score can effectively integrate structural and functional features linked to phenotypes of a medical condition such as, for example, AS, that spans across all disease stages, and further correlate with parameters of progressive diastolic dysfunction. Because the execution phase is performed using the set model parameters refined during the training phase, the execution phase can be computationally quick.

[0047] In some embodiments, a machine-learning model and / or deep learning algorithm can be applied to severe phenotypes of heart disease (e.g., AS), and in which the machine-learning model can output a continuous, observer-independent metric of phenotypes, applied across the spectrum of the disease, including patients without heart disease, or with mild or moderate heart disease. In some embodiments, the algorithm provides independent prognostic value for both the echocardiographic and clinical progression of heart disease, defined based on longitudinal changes in echo indices of heart disease severity and the incidence of all-cause mortality or AVR, respectively. In some implementations, the machine-learning model can maintain its ability to discriminate existing AS, but also predict worse outcomes, independent of traditional risk factors.

[0048] In some embodiments, the machine-learning model (or algorithm) can be applied to raw echocardiographic (as well as cardiac magnetic resonance) videos, rather than processed images, or reports. As such, it can be integrated into any point-of-care ultrasound or traditional echocardiographic exam, both retrospectively and prospectively. In some cases, the machinelearning model does not require Doppler measurements which are time-consuming and current gold-standard for the diagnosis of AS. The machine-learning model can exhibit excellent predictive performance across several geographically and temporally distinct datasets and generalize across imaging modalities, with identical results across multinational, multimodal imaging cohorts. This highlights its value as a modality-invariant video-based tool to predict the risk of AS progression.

[0049] For instance, aortic sclerosis can be recognized as a precursor of AS that is present in approximately 26% and 50% of individuals over 65 years and 85 years, respectively. In some implementations, the machine-learning model can produce and / or determine a digital biomarker that offers a robust yet scalable approach to clinically actionable risk stratification of the millions of echocardiographic or portable cardiac ultrasound studies performed yearly in the community. In some cases, the machine-learning model can require no modifications to the acquisition protocol, and thus maximizes the inference from images that are already acquired as part of standard practice. In some implementations, the machine-learning model can be portable across different modalities (both echo and cardiac magnetic resonance imaging), enabling a deeper multimodal phenotyping of the risk of AS in several clinical contexts.

[0050] The training set and the subject dataset can include any suitable images associated with an organ corresponding to the medical condition, such as, but not limited to, the heart. In some embodiments, the training set and / or the subject dataset include images from multiple modalities. For example, in some embodiments, the images include a first input type from a first modality (e.g., echocardiographic images / reports) and a second input type from a second modality (e.g, cardiac magnetic resonance (CMR) data). In some embodiments, the training set includes the first input type and the subject dataset includes the first and / or second input type. For example, in some embodiments, the machine-learning model can be trained using the first input type (e.g, echocardiographic images), after which the trained machine-learning model can be used to detect a medical condition using the first and / or second input type from the subject (e.g, echocardiographic and / or CMR images). Additionally or alternatively, in some embodiments, the machine-learning model is trained and / or configured (e.g., via training, self-training, etc.) to refine the phenotype score based on a comparison between the second input type and the first input type. Accordingly, in some embodiments, the method includes generating a multi-modality digital biomarker for detecting the medical condition (e.g., heart condition) based on the features associated with the medical condition that can be detected in different data types.

[0051] In some embodiments, the method further includes extracting imaging data from the second input e.g., CMR data). For example, the machine-learning model can be further trained to identify imaging views from the CMR data. The imaging views can include, for example, a standardized view (e.g., long axis cine views) of the imaging dataset. In some implementations, the machine-learning model compiles the imaging views to produce a compiled image sequences. In some cases, the compiled image sequences can be or include cine videos (resampled to 112 x 112 pixels) or still frames used for direct DASSi inference. In some implementations, the compiled image sequences from the CMR data are further transformed into echo-like visualizations that match the orientation of a PLAX view that is similar to the echocardiographic images / reports (i.e., the second input type is transformed into imaging sequences similar to the first input type). That is, in some implementations, the machine-learning model is implemented in a pipeline that directly transforms videos derived from an independent imaging modality, cardiovascular magnetic resonance (CMR), to echo-like visualizations, enabling the portability and generalization of the algorithm across cohorts and modalities. Although described herein primarily with respect to echocardiographic and CMR data, as will be appreciated by those skilled in the art, the disclosure is not so limited and includes training and / or detection using any other suitable data type.

[0052] FIG. 3 is a flow diagram of a method 300 for executing a machine-learning model to produce a phenotype score for a medical condition, according to some embodiments. In some implementations, the method 300 can be performed by a processor of a compute device. At 305, the method 300 includes receiving an input including a plurality of imaging sequences, each imaging sequence from the plurality of imaging sequences associated with a user from a plurality of users.

[0053] At 310, the method 300 can include extracting, for each imaging sequence, a clip from a plurality of clips.

[0054] At 315, the method 300 can include executing the machine-learning model using the plurality of clips to produce a phenotype score for a set of features associated with the heart condition and with a user from the plurality of users, the phenotype score used to generate a digital biomarker. In some implementations, the phenotype score can be consistent with any phenotype score as described herein. In some cases, the phenotype score can indicate at least one of a level of severity or non-existence of aortic stenosis, the phenotype score further correlating with at least one of: traditional Doppler parameters of aortic stenosis presence, severity, and parameters of progressive diastolic dysfunction. The phenotype score can be a numerical score ranging from 0 to 1. In some cases, the phenotype score can be assigned to features that represent, for example, a multi-instance contrastive learning task, a sequence re-ordering pretext task, a cross-modal contrastive learning task spanning videos, images of cardiac function, anatomy, and electrical activity, and / or a biometric learning task that identifies different videos drawn from the same subject to define subtle features from the different videos. FIG. 4 is a flow diagram of a method 400 for producing a multi-modal progression score to generate a digital biomarker, according to some embodiments. In some embodiments, the method 400 can be performed by a processor of a compute device and / or automatically. In some cases, the method 400 can be executed on a desktop application, a cloud service, or a mobile device.

[0055] At 405, the method 400 includes receiving a first input including a plurality of imaging sequences, each imaging sequence from the plurality of imaging sequences associated with a user from a first plurality of users.

[0056] At 410, the method 400 includes extracting, for each imaging sequence, a clip from a first plurality of clips.

[0057] At 415, the method 400 includes executing a machine-learning model using the first plurality of clips to produce phenotype score for a set of features that is associated with a heart condition and with a user from the first plurality of users.

[0058] At 420, the method 400 includes receiving a second input including a plurality of datasets, each dataset from the plurality of datasets associated with a heart of a user from a second plurality of users.

[0059] At 425, the method 400 includes extracting, for each dataset, imaging data from a plurality of imaging data;

[0060] At 430, the method 400 includes executing the machine-learning model using the plurality of imaging data as an input to identify, for each imaging data from the plurality of imaging data, a plurality of imaging views with respect to a standardized view of the heart for that imaging data.

[0061] At 435, the method 400 includes compiling, for each imaging data, the plurality of imaging views to produce a compiled image sequence from a plurality of compiled image sequences. In some cases, the compiled image sequence can be a cine video or still frame.

[0062] At 440, the method 400 includes transforming each compiled image sequence from the plurality of compiled image sequences to generate a clip from a second plurality of clips, the clip from the second plurality of clips including an imaging view that resembles an imaging view from the first input. For example, in some embodiments, the method 400 includes transforming CMR imaging to resemble echocardiographic imaging. In some cases, transforming each compiled image sequence can include rotating, cropping to a cardiac outline, and inverting to grayscale.

[0063] In some embodiments, the method 400 can include translating the DASSi to CMR imaging (z.e., applying the DASSi trained on echocardiographic imaging to CMR imaging). For instance, images used to translate can include long axis cines for cardiac function through the left ventricular outflow tract (LVOT) view, which can include the same anatomical structures as the echocardiographic PLAX view used for the development of DASSi on echocardiography. An automatic pipeline can be defined that extracts individual .DICOM files from each study-specific folder, identifies long axis cine views of the LVOT tract using the available .DICOM headers that suggest a 3 -chamber view (CINE-segmented-LAX-3Ch), windows the grayscale according to the default center and width of each study, converts the windowed data to 8-bit .avi files while selecting every other frame (thus creating 25-frame-long clips, by skipping every other clip in the original 50 frames of each view-specific cine) and down-samples to 112 x 112 pixels. The sagittal clips can then be rotated by 90 degrees to match the orientation of a PLAX view, cropped (removing 30 pixels from the new left / right side, 20 pixels from the top and 30 pixels from the bottom) to remove structures traditionally not seen on echocardiography, and the grayscale can then be inversed to ensure that the myocardial wall appears brighter than its cavity. This can enable an MRI2echo® transition that does not modify the composition of the underlying signal. The final clips can be resampled to 112 x 112 pixels and can then be used for direct DASSi inference.

[0064] At 445, the method includes refining the phenotype score based on a comparison between the second plurality of clips and the first plurality of clips to generate a digital biomarker that is used to detect the heart condition based on the set of features associated with the heart condition. In some cases, the digital biomarker can be used to stratify a risk of development of the heart condition based on inputs of data types that are different from those of the first input and the second input

[0065] In some implementations, the method 400 can further include confirming the accuracy of the digital biomarker without Doppler imaging for phenotyping of aortic stenosis. In some implementations, the method 400 can include training the machine-learning model using a training set that includes a plurality of imaging sequences correlated to a plurality of anatomical features associated with the heart condition. In some implementations, the machine-learning model can be deployed and can receive inputs that are de-identified, down-sampled, and processed for automated view classification to identify specific videos from the input that correspond to PLAX views (similar to the first input). In some cases, down-sampled 16-frame clips extracted from 2D PLAX videos can be further processed. The machine-learning model can be trained to detect severe AS, and predictions can be based on an ensemble of any combination of three models with a combination of weight initializations: random, pre-trained (e.g., Kinetics-400), and / or self-supervised learning for echocardiograms.

[0066] FIG. 5 is an illustrative diagram of process for producing a digital biomarker from a phenotype score using echocardiography data, according to some embodiments. FIG. 5 can also represent a study design, for AV Vmax: peak aortic valve velocity; AVR: aortic valve replacement; DASSi: digital aortic stenosis severity index; MRI: magnetic resonance imaging; PLAX: parasternal long axis view. Users for the study were independently drawn from New England (5 hospitals affiliated with Yale-New Haven Health) with eligible users used to define two nested sub-cohorts: (1) A longitudinal echocardiography cohort of individuals who underwent transthoracic echocardiography at two or more timepoints; and (2) a clinical outcomes cohort which included all individuals who had longitudinal follow-up for a composite clinical outcome of death and / or AVR. Eligible individuals had i) baseline peak aortic valve velocity (AV Vmax) of less than 4 m / sec, ii) no prior history of AVR, and iii) available PLAX videos available for processing. To avoid bias, none of the patients who contributed to the training set of the original model development were included in the study.

[0067] FIG. 6 is an illustrative diagram of a process for cross-modal validation of a phenotype score using MRI videos, according to some embodiments. An analysis was performed on 45,470 individuals who enrolled in a CMR sub-study, after excluding individuals who had withdrawn consent, those with prior AVR and those for whom the files could not be processed to generate reliable videos.

[0068] FIG. 7 is an illustrative diagram of graphs depicting changes in aortic valve, according to some embodiments. FIG. 7 also depicts results of AV Vmax progression. For instance, higher DASSi was an independent predictor of AV Vmax progression, with each 0.1 increment associated with a +0.033 m / s / year [95% CI: 0.028-0.038, p<0.001 ] increase in AV Vmax, adjusting for the patient’s age, sex, race, ethnicity, baseline AV Vmax, and LVEF (see 702). There was a graded association, ranging from 0.04±0.01 (SEM) m / sec / year for DASSi values <0.2 at baseline, to 0.21±0.01 for baseline values >0.6, which persisted within each distinct baseline AS stenosis group (see 704). There was evidence of interaction between baseline DASSi and the flow-corrected peak velocity ratio (pinteraction=0.002), but not with AV Vmax (pinteraction=0.26), with higher DASSi associated with higher rates of progression for lower baseline peak velocity ratios (see 706 and 708).

[0069] FIG. 8 is an illustrative diagram depicting cross-modal validation of a phenotype score using MRI videos, according to some embodiments. FIG. 8 also depicts a cross-modal validation of DASSi using CMR videos. For instance, by directly applying the original DASSi algorithm to the preprocessed cine CMR videos to create pseudo-PLAX echo views (1 video / patient) (see 802), the algorithm was able to successfully discriminate between patients with vs without diagnosed AS (DASSi: 0.32 [IQR 0.25-0.38], n=182, vs 0.19 [0.12-0.26], n=45,292, respectively, p<0.001) (see 804). Of note, in this relatively healthy community cohort, no individual had a DASSi value >0.607, consistent with severe AS. Finally, among 45,470 individuals followed over 2.5 [IQR 1.6-3.9] years, 57 participants underwent AVR and 376 died. Higher DASSi at baseline was associated with a higher risk of all-cause mortality or AVR (per 0.1 increments: adj. HR 1.20 [95% CI 1.08-1.33], p<0.001) (see 806), independent of age, sex, ethnic background, history of AS, and baseline LVEF. Compared to the reference group of DASSi <0.2, those with values >0.4 had a 3.2-fold higher adjusted risk of the composite endpoint, which persisted even in the absence of previously known AS (see 808-810), and an up to 20-fold higher adjusted risk of future AVR.

[0070] FIG. 9 is an illustrative diagram depicting saliency maps with phenotype score predictions, according to some embodiments. For instance, the pertinent saliency maps for two of the patients with the highest DASSi predictions (0.5-0.6) are shown (see 902 and 904), highlighting the structure / motion of the right ventricle and left atrium as relevant areas. In an age- and sex-adjusted PheWAS analysis, DASSi was linked to cardiovascular-specific risk factors closely linked with diastolic dysfunction, such as hypertension, atrial fibrillation, obesity, diabetes mellitus and hypercholesterolemia (see 906). Age and sex explained 19% of the variation in DASSi, whereas the addition of the top 10 hits only increased this to 21% (adj. R2=0.21, based on multivariable linear regression). Further adjustment for these phenotypes did not impact the prognostic value of DASSi for death / AVR (adj. HR 1.16 [95%CT 1 .04-1 .29], p=0.006).

[0071] Any of the methods disclosed herein can be performed by a suitable processor of a compute device. For example, FIG. 1 illustrates a block diagram of a system 100 for a multimodal progression score for aortic stenosis, according to some embodiments. The system can include a compute device 101. The compute device 101 can include a processor 102 and a memory 103 that communicate with each other, and with other components (e.g., database 105, network interface 106, VO interfaces 107, etc.), via a bus 104. The bus 104 can include any of several types of bus structures including, but not limited to, a memory bus, a memory controller, a peripheral bus, a local bus, and any combinations thereof, using any of a variety of bus architectures. The compute device 101 can be or include, for example, a computer workstation, a terminal computer, a server computer, a handheld device (e g., a tablet computer, a smartphone, etc.), a web appliance, a network router, a network switch, a network bridge, any machine capable of executing a sequence of instructions that specify an action to be taken by that machine, and any combinations thereof. The compute device 101 can also include multiple compute devices that can be used to implement a specially configured set of instructions for causing one or more of the compute devices to perform any one or more of the aspects and / or methodologies described herein.

[0072] In some implementations, the compute device 101 can include a network interface 106. The network interface 106, can be utilized for connecting the compute device 101 to one or more of a variety of networks (e.g., a network) and one or more remote devices connected thereto. In other words, although not shown in FIG. 1, the various devices including compute device 101 can communicate with other devices via a network(s). The network can include, for example, private network, a Virtual Private Network (VPN), a Multiprotocol Label Switching (MPLS) circuit, the Internet, an intranet, a local area network (LAN), a wide area network (WAN), a metropolitan area network (MAN), a worldwide interoperability for microwave access network (WiMAX®), an optical fiber (or fiber optic)-based network, a Bluetooth® network, a virtual network, and / or any combination thereof. In some instances, the network can be a wireless network such as, for example, a Wi-Fi or wireless local area network (“WLAN”), a wireless wide area network (“WWAN”), and / or a cellular network. In other instances, the network can be a wired network such as, for example, an Ethernet network, a digital subscription line (“DSL”) network, a broadband network, and / or a fiber-optic network. In some instances, the compute device 101 can use Application Programming Interfaces (APIs) and / or data interchange formats (e.g., Representational State Transfer (REST), JavaScript Object Notation (JSON), Extensible Markup Language (XML), Simple Object Access Protocol (SOAP), and / or Java Message Service (JMS)). The communications sent via the network can be encrypted or unencrypted. In some instances, the network can include multiple networks or subnetworks operatively coupled to one another by, for example, network bridges, routers, switches, gateways and / or the like.

[0073] The processor 102 can be or include, for example, a hardware based integrated circuit (IC), or any other suitable processing device configured to run and / or execute a set of instructions or code. For example, the processor 102 can be a general-purpose processor, a central processing unit (CPU), an accelerated processing unit (APU), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA), a programmable logic array (PLA), a complex programmable logic device (CPLD), a programmable logic controller (PLC) and / or the like. In some implementations, the processor 102 can be configured to run any of the methods and / or portions of methods discussed herein.

[0074] The database 105 can store information generated by the processor 102 and / or received at the processor 102. In some implementations, the database 105 can include, for example, hard disk drives (HDDs), solid-state drives (SSDs), USB flash drives, memory cards, optical discs such as CDs and DVDs, and / or the like. In some implementations, the database 105 can include a database (e g., a cloud database, a local database, etc.) that can be different from the memory 103. For example, the memory 103 can be volatile, meaning that its contents can be lost when the compute device 101 is turned off. The database 105 can be configured to be persistent, meaning that its contents can be retained even when the compute device 101 is turned off. In some implementations, the database 105 can be configured to organize and manage large amounts of data, whereas the memory 103 can be configured to be used for temporary storage of data and program instructions. In some implementations, the database 105 can be configured to provide efficient and reliable storage and retrieval of data and can include features such as, for example, indexing, querying, and transaction management, while the memory 103 can be configured for rapid access and manipulation of data.

[0075] The memory 103 can be or include, for example, a random-access memory (RAM), a memory buffer, a hard drive, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), and / or the like. In some instances, the memory can store, for example, one or more software programs and / or code that can include instructions to cause the processor 102 to perform one or more processes, functions, and / or the like. In some implementations, the memory 103 can include extendable storage units that can be added and used incrementally. In some implementations, the memory 103 can be a portable memory (e.g., a flash drive, a portable hard disk, and / or the like) that can be operatively coupled to the processor 102. In some instances, the memory 103 can be remotely operatively coupled with a compute device (not shown); for example, a remote database device can serve as a memory and be operatively coupled to the compute device. The memory 103 can include various components (e.g., machine- readable media) including, but not limited to, a random-access memory component, a read only component, and any combinations thereof. In one example, a basic input / output system (BIOS), including basic routines that help to transfer information between components within the compute device 101, such as during start-up, can be stored in memory 103. The memory 103 can further include any number of program modules including, for example, an operating system, one or more application programs, other program modules, program data, and any combinations thereof.

[0076] In some implementations, the memory 103 can store data such as, for example, a first input 110, first clips 111, a machine-learning model 112, a training set 113, a phenotype score 114, a second input 115 that is different from the first input 110, imaging views 116, compiled image sequences 117, second clips 118, and / or a digital biomarker 119. The first input 110 can include, for example, imaging sequences or videos of two-dimensional echocardiographic videos of PLAX view. The processor 102 can be caused to extract the first clips 111 from the first input 110 which includes a collection of videos or image sequences of multiple different users and their organ such as, for example, a heart. The second input 115 can include imaging from a different modality as compared to the first input 110, such as, but not limited to, CMR data. The processor 102 can be caused to extract the second clips 118 from the second input 115 which includes a collection of videos or image sequences of multiple different users and their organ such as, for example, a heart.

[0077] In some instances, the training set can be divided into batches of data based on a memory size, a memory type, a processor type, and / or the like. In some instances, the images and / or videos can be divided into batches of data based on a type of the processor 102 (e.g., CPU, GPU, and / or the like), number of cores of the processor, and / or other characteristic of the memory or the processor.

[0078] The I / O interfaces 107 of the compute device 101 can be used to connect the compute device 101 to one or more of a variety of networks and one or more remote devices connected thereto. In other words, the compute device 101 can communicate with other devices via a network. I / O interfaces 107 can be any suitable component(s) that enable communication between internal components of the compute device 101 and external devices, such as, for example, a mobile device, a remote compute device, and / or the like.

[0079] The database 105 can store information generated by the processor 102 and / or received at the processor 102. In some implementations, the database 105 can include, for example, hard disk drives (HDDs), solid-state drives (SSDs), USB flash drives, memory cards, optical discs such as CDs and DVDs, and / or the like. In some implementations, the database 105 can include a database (e g., a cloud database, a local database, etc.) that can be different from the memory 103. For example, the memory 103 can be volatile, meaning that its contents can be lost when the compute device 101 is turned off. The database 105 can be configured to be persistent, meaning that its contents can be retained even when the compute device 101 is turned off. In some implementations, the database 105 can be configured to organize and manage large amounts of data, whereas the memory 103 can be configured to be used for temporary storage of data and program instructions. In some implementations, the database 105 can be configured to provide efficient and reliable storage and retrieval of data and can include features such as, for example, indexing, querying, and transaction management, while the memory 103 can be configured for rapid access and manipulation of data.

[0080] The following example(s) further illustrate aspects of the present invention. However, they are in no way a limitation of the teachings or disclosure of the present invention as set forth herein.

[0081] EXAMPLES

[0082] EXAMPLE 1

[0083] Introduction

[0084] Aortic stenosis (AS) is a major public health challenge that requires frequent testing for diagnosis but lacks a reliable prognostic biomarker that allows personalized screening and follow-up. With the expanding availability of transcatheter and surgical aortic valve replacement (AVR) procedures that effectively modify the natural history of AS, there has been a focus on timely identification of patients at earlier stages of the disease at risk of rapid progression and worse clinical outcomes. However, AS is often diagnosed during its late, symptomatic stage, despite a long asymptomatic course during which timely identification is likely to have the most impact on patient outcomes and prevent adverse myocardial remodeling. This can stem from challenges with risk stratification of AS development and progression, wherein there has been limited ability to define measurable features of disease progression, given the large heterogeneity in the condition and its drivers. In this context, the only way to currently identify AS and track its progression is to pursue serial Doppler echocardiography, which is neither feasible nor cost- effective given the high prevalence of early aortic sclerosis and stenosis, and a highly variable rate of progression to more advanced stages of the disease.

[0085] This Example describes a cross-modal video-based artificial intelligence (Al) biomarker, and explores its prognostic value for AS development and progression across multinational cohorts and two distinct modalities - namely echocardiography and cardiac magnetic resonance (CMR). A deep learning (DL) strategy was developed that uses a self-supervised, contrastive learning approach to learn key representations of severe AS on single-view, two-dimensional echocardiographic videos of the parasternal long axis (PLAX), a standard and easy-to-obtain echocardiographic view without the need for Doppler imaging. The predicted phenotype score from this model, the Digital AS Severity index (DASSi), demonstrates excellent performance across temporally and geographically distinct cohorts, effectively integrating structural and functional features linked to the severe AS phenotype that spanned all disease stages, and further correlating with parameters of progressive diastolic dysfunction.

[0086] This Example also explores whether as DASSi identifies the echocardiographic signature of the severe AS phenotype, it would carry prognostic value among individuals without severe AS, stratifying the risk of echocardiographic and clinical disease progression independent of traditional Doppler parameters. Furthermore, this Example explores whether the anatomical and temporal information learned by DASSi during its self-supervised pre-training would generalize across imaging modalities, detecting a high-risk phenotype for AS development and progression that is dissociated from the specific technical characteristics of the acquisition method. These represent important pieces for the comprehensive evaluation of Al-guided imaging biomarkers and serve to demonstrate that the representations learned by the model are independent of modality- or population-specific confounders, thus supporting their generalizability and maximizing their application across the range of cardiovascular diagnostics.

[0087] To this end, data was used to demonstrate DASSi in a health system-based echocardiography cohort as well as in a cardiac magnetic resonance (CMR) cohort from UK, drawn from the UK Biobank, a population-based cohort of protocolized CMR imaging. The rates of AS development and progression by echocardiography were compared, as well as adverse clinic events, namely death or AVR, across DASSi strata defined on both echocardiography and CMR videos, spanning a range of geographical and temporal settings and patient phenotypes from no to early, mild, and moderate AS.

[0088] Across two distinct cohorts of thousands of patients undergoing echocardiography, higher baseline DASSi (in 0.1 increments; from 0 to 1) was associated with faster progression in Doppler parameters of AS (0.03 to 0.08 m / s / year increase in peak aortic valve velocity), a 16 to 19% higher risk of developing new AS, 14 to 23% higher risk of progressing to the next stage, and a 10 to 14% higher of future all-cause mortality or aortic valve replacement ( / ?<0.001 for all). In a cross-modal experiment using a novel pipeline that transforms CMR data to echocardiography-like videos, zero-shot estimation of DASSi in thousands of participants undergoing protocolized assessment in the UK Biobank replicated a 20% higher adjusted risk of death or Aortic valve replacement (AVR) or valvuloplasty procedure (per 0.1 DASSi increments). Saliency maps and phenome-wide association studies identified close links between DASSi and traditional cardiovascular risk factors and diastolic dysfunction, yet these parameters failed to explain its variation and prognostic value. Results were consistent across severity strata, including those without AS at baseline. In conclusion, this Example describes a unified videobased imaging biomarker of cardiac structure and function that can efficiently stratify the risk of AS development and progression on both two-dimensional echocardiography and CMR, enabling efficient risk stratification and multimodal phenotyping of AS in the community, and maximizing the value of two of the most widely performed cardiovascular tests with no change in their acquisition protocols.

[0089] Results

[0090] Study Overview and Population Aortic stenosis is widely recognized as a disease of both the valve and the underlying myocardium. Acknowledging the need for multiparametric phenotyping, DASSi was trained using a self-supervised framework that enables the learning of key representations of valvular and myocardial structure linked to the severe AS phenotype. This multi-arm study was designed to comprehensively assess the prognostic value of DASSi for AS development and progression by both serial imaging and clinical outcomes, across two independent multi -hospital networks in the United States, with further international and cross-modal validation in the CMR cohort (as shown in FIG. 5 or FIG. 6)

[0091] Echocardiography Study

[0092] Echocardiography is the gold-standard modality for the diagnosis and monitoring of AS. Clinical echocardiographic studies were retrieved by querying the local database of the two participating hospital networks (Yale-New Haven Health-affiliated hospitals in Connecticut and Rhode Island; New England cohort) for studies fulfdling the following criteria: i) no AS, sclerosis without stenosis, mild, or moderate AS at baseline (per the corresponding report), ii) presence of at least one two-dimensional video of the parasternal long axis view (PLAX), and iii) baseline peak aortic valve velocity (AV Vmax) of <4 m / s. The New England cohort consisted of 8,798 patients (n=4,250 [48.3%] women) with a median age of 71 [IQR 60-80] years, of whom 613 (8.2%) reported Hispanic / Latino ethnicity and 737 (9.6%) were Black. None of these patients had contributed data to the original model development. At the time of the baseline echocardiographic assessment, 1,047 (13.1%) participants had aortic sclerosis without stenosis, 2,017 (25.3%) had mild AS and 979 (12.3%) had moderate AS. Participants were followed to assess the progression of AS through longitudinal changes in AV Vmax, as well as all-cause mortality or AVR >90 days after the index scan (see FIG. 5).

[0093] Cardiac MR.

[0094] The UK Biobank is a prospective cohort study of half a million 40-69 year-old participants from the United Kingdom. Individuals who underwent cardiac MRI imaging were included as part of a follow-up comprehensive imaging visit, using a detailed protocol that has been previously described. This allowed expansion of the analysis to individuals who underwent imaging independent of symptoms, thus minimizing confounding by indication, while alleviating potential confounding effects from signals that overfit to technical aspects of echocardiography. After excluding individuals who had (i) withdrawn consent for use of their data, ii) those with a history of AVR before or within 90 days of CMR, iii) and those with videos that could not be processed due to technical reasons, as there were 45,470 individuals included in the study (65 [59-71] years, n=23,558 [51.8%] women), 178 (0.4%) of whom had diagnosed AS (by ICD codes) at the time of enrollment (see FIG. 6).

[0095] Baseline DASSi Phenotyping

[0096] DASSi can be directly quantified using a simple two-dimensional PLAX echo video as input in less than 2 seconds, enabling direct inference from videos that can be acquired by individuals with no extensive training. In the New England cohort, the median DASSi was 0.24 [IQR 0.10-0.47], For reference, the optimal cut-off for severe AS in the general population was previously identified at 0.607. Higher baseline DASSi was strongly associated with greater AS severity by all traditional Doppler-derived parameters, including AV Vmax, and mean AV gradients (p=0.63, n=8,798; and 0.64, n=6,220, respectively, both / ?<0.001), calculated AV area (p=-0.53, n=5,410, ><0.001) and peak velocity ratio (p=-0.63, n=8, 163, / ;<0.001 ). There was also a modest correlation with diastolic dysfunction, as assessed by greater E / e’ (maximum velocity of the early trans-mitral filling flow at diastole divided by the maximum velocity of the septal mitral annulus at early diastole; p=0.36, n=7,079, / ?<0.001), left atrial volume index values (p=0.31, n=7,421, / ?<0.001), and right ventricular systolic pressure (p=0.18, n=6,312, / ?<0.001). Confirming its lack of dependence on systolic function and flow parameters for defining future AS-related risk, DASSi was independent of left ventricular ejection fraction (p=-0.01, n=8,608,?=0.39) and estimated stroke volume (p=-0.02, n=7,073, / ?=0.11).

[0097] DASSi and Echocardiographic Progression of AS

[0098] Longitudinal monitoring for AS progression can rely on serial Doppler echocardiography, yet this is not recommended for those without hemodynamically significant AS at baseline; furthermore, rates of progression can be highly variable precluding precise course estimation using traditional criteria. In this Example, 5,483 of 8,798 patients (62.3%) in the New England cohort had at least one follow-up study (median 4 [IQR 2-5] studies / patient) within a median of 3.5 [IQR 2.3-4.6] years. Greater AS severity correlated with both higher baseline DASSi as well as more rapid change in AV Vmax. DASSi ranged from 0.15 [IQR 0.06- 0.26] among patients without AS or sclerosis to 0.61 [0.48-0.72] among patients with moderate AS (see FIG. 7), whereas the change in AV Vmax ranged from 0.01 [IQR -0.11 to 0.13] m / sec / year among patients without AS to 0.18 [0.03-0.38] m / sec / year among patients with moderate AS (see FIG. 7).

[0099] However, regardless of baseline severity, higher DASSi was an independent predictor of AV Vmax progression, with each 0.1 increment associated with a +0.033 m / s / year [95% CI: 0.028-0.038, p<0.001] increase in AV Vmax, adjusting for the patient’s age, sex, race, ethnicity, baseline AV Vmax, and LVEF (see FIG. 8). There was a graded association, ranging from 0.04+0.01 (SEM) m / sec / year for DASSi values <0.2 at baseline, to 0.21+0.01 for baseline values >0.6), which persisted within each distinct baseline AS stenosis group (see FIG. 7). There was evidence of possible interaction between baseline DASSi and the flow-corrected peak velocity ratio (^interaction 0.002), but not with AV V max pinteraction 0.26), with higher DASSi associated with higher rates of progression for lower baseline peak velocity ratios (see FIG. 7). The association remained consistent across demographic subgroups, impaired or preserved left ventricular function (LVEF > vs <50%) and baseline AV Vmax strata.

[0100] To assess whether changes on a continuous scale translate into clinically actionable progression, the risk of progressing to a higher severity stage based on established criteria was further examined, as well as the risk of developing new-onset AS among those without stenosis at baseline. Over a median follow-up of 3.1 [1.5-4.0] years, 2,037 (37.2%) participants had a follow-up echocardiographic report describing a higher AS severity grade than their baseline study. Greater DASSi was associated with a higher adjusted risk of progressing to a higher severity stage (adj. HR 1.14 [95% CI 1.12-1.17], p<0.001, per 0.1 increments) after adjusting for the above-mentioned covariates, including the baseline AS severity stage (sclerosis, mild or moderate AS). Furthermore, DASSi can predict the future development of any AS in a subset of 2,091 patients without AS at baseline (per 0.1 incr.; adj. HR 1.16 [1.09-1.23], p<0.001, n=325 new AS cases).

[0101] DASSi and Future Risk of Mortality or Aortic Valve Replacement

[0102] The availability of echocardiographic follow-up is not random and it is known that AS often follows an insidious course that contributes to increased morbidity and mortality. To explore this, the 8,798 patients in the New England cohort were followed for 4.1 [IQR 2.3-5.4] years, during which 1,302 patients died and 736 underwent AVR (1,964 patients had AVR or died during follow-up). For every 0.1 increment in the baseline DASSi, there was a higher adjusted risk of death or AVR (HR 1.10 [95% CI 1.08-1.13, p<0.001), which persisted in competing risk analysis (all-cause mortality: HR 1.06 [ 1.02-1.09], p=0.002, & AVR: 1.21 [1.16- 1.26], p<0.001), adjusted for the patient’s age, sex, race / ethnicity, baseline AV Vmaxand LVEF (see FIG. 9). Specifically, patients in the highest group (DASSi >0.6) had an 80% higher adjusted risk of the composite endpoint compared to those in the lowest group (DASSi <0.2). The prognostic value of DASSi was consistent across men and women, and different age, race / ethnicity, LVEF, and AV Vmax strata.

[0103] Cross-modal Translation of DASSi to CMR Imaging

[0104] Imaging biomarkers are often developed and confined within a single modality, however those with diagnostic and prognostic value that relies on detecting pertinent anatomical and functional features should maintain their performance across different, related modalities. Proving the prognostic value of a biomarker across two or more modalities is important, since i) it supports that notion the algorithm learns key representations of the disease, rather than technical confounders specific to the acquisition process; ii) it maximizes the value of Al algorithms by enabling zero-shot predictions in new cohorts and clinical settings; iii) it augments our ability to opportunistically risk stratify for AS using existing data streams. This hypothesis was tested by translating DASSi to CMR imaging using the UK Biobank registry. For this, we defined an automated pipeline that extracts individual CMR files, identifies long axis cine views of the left ventricular outflow tract, and ultimately compiles these into cine videos or still frames. These are then rotated, cropped to cardiac outline, and inverted to grayscale to create clips that loosely mimic the acquisition of a PLAX view on echocardiography, essentially enabling an MRI2echo® transition that does not modify the composition of the underlying signal.

[0105] Cross-modal Testing of DASSi using CMR Videos (Cross-modal Validation)

[0106] By directly applying the original DASSi algorithm to the preprocessed cine CMR videos to create pseudo-PLAX echo views (1 video / patient), the present algorithm was able to successfully discriminate between patients with vs without diagnosed AS (DASSi: 0.32 [IQR 0.25-0.38], n=182, vs 0.19 [0.12-0.26], n=45,292, respectively, <0.001) (see FIG. 9). Of note, in this relatively healthy community cohort, no individual had a DASSi value >0.607, consistent with severe AS. Finally, among 45,470 individuals followed over 2.5 [IQR 1.6-3.9] years, 57 participants underwent AVR and 376 died. Higher DASSi at baseline was associated with a higher risk of all-cause mortality or AVR (per 0.1 increments: adj. HR 1.20 [95% CI 1.08-1.33],?<0.001) (see FIG. 9), independent of age, sex, ethnic background, history of AS, and baseline LVEF. Compared to the reference group of DASSi <0.2, those with values >=0.4 had a 3.2-fold higher adjusted risk of the composite endpoint, which persisted even in the absence of previously known AS, and an up to 20-fold higher adjusted risk of future AVR.

[0107] Phenotypic Correlates of DASSi

[0108] The explainability of DASSi was assessed by computing gradient-weighted class activation maps on CMR and performing a phenome-wide association study (PheWAS). Pertinent saliency maps for two of the patients with the highest DASSi predictions (0.5-0.6) , highlighting the structure / motion of the right ventricle and left atrium as relevant areas. Finally, in PheWAS analysis that included all 45,740 patients in the UK Biobank, DASSi was associated predominantly with cardiovascular-specific risk factors associated with the risk of AS progression as well as diastolic dysfunction, such as hypertension, atrial fibrillation, obesity, diabetes mellitus and hypercholesterolemia. Age and sex explained 19% of the variation in DASSi, whereas the addition of the top 10 hits only increased this to 21% (adj. R2=0.21, based on multivariable linear regression). Further adjustment for these phenotypes did not impact the prognostic value of DASSi for death / AVR (adj. HR 1.16 [95%CI 1.04-1.29], p=0.006).

[0109] Discussion

[0110] In this multinational cohort study of patients with no or early AS at baseline, a deep learning-derived video-based biomarker of AS - DASSi - was a strong and independent predictor of future AS development and progression independent of key clinical parameters and baseline severity defined by traditional Doppler criteria. The prognostic value of DASSi was shown across three geographically, temporally, and phenotypically distinct multinational cohorts, including 8,798 patients undergoing clinical echocardiography, the original modality used for its training, as well as 45,470 patients undergoing protocolized CMR phenotyping in the UK Biobank using a zero-shot prediction approach. These findings lend significant support to the use of DASSi, a first-of-its-kind, cross-modal Al biomarker that can be directly applied to two- dimensional echocardiography without Doppler and standard CMR videos, as a standardized solution for deeper phenotyping of AS and screening in the community. Moreover, from a methodological standpoint, our work describes a novel zero-shot, cross-modal approach that can maximize the inference of novel Al digital biomarkers by enabling compatibility across distinct imaging modalities. Interpretability methods and PheWAS studies further confirm that DASSi learns patterns of early diastolic dysfunction and cardiovascular risk but cannot be replaced by such structured metrics. In summary, these findings highlight the promise of deep learning- enhanced two-dimensional echocardiography in the phenotyping of complex valvular disease by maximizing the diagnostic yield of existing clinical protocols.

[0111] Several epidemiological studies have highlighted the morbidity and health economic impact of undiagnosed AS, and the importance of early detection and risk stratification. Efforts to identify individuals at risk of progression are limited by the large burden of milder forms of aortic valve disease, such as aortic sclerosis, which is present in approximately 26% and 50% of individuals over 65 years and 85 years, respectively. Despite the lack of flow limitation in these valves, aortic sclerosis is associated with an up to 50% increase in the risk of cardiovascular mortality or myocardial infarction, and higher risk of progression to severe AS. However, a key challenge in personalizing the management of these patients is the marked variability in the progression rates of patients within similar Doppler-adjudicated severity stages. Prior efforts in this space have focused on identifying a broader range of risk factors that are independently associated with AS progression, including traditional risk factors such as hypercholesterolemia, smoking, renal dysfunction, and elevated natriuretic peptide levels, which lack specificity for AS. Alternative Doppler-derived indices require skilled acquisition and modifications to the scanning and reporting protocols, whereas imaging by positron-emission tomography using radiotracers specific for active microcalcification can be costly and is not readily available.

[0112] On the echocardiography front, deep learning-enhanced, two-dimensional echocardiography with DASSi aims to bridge this gap by providing a Doppler-independent AS severity metric that can be computed from any portable or standard transthoracic echocardiogram. Trained to detect generalizable features associated with the severe AS phenotype, DASSi can be computed on any routine echocardiogram, and maintains its prognostic value across the spectrum of AS stages, identifying individuals who do not meet traditional criteria for severe AS, yet exhibit faster rates of progression similar to those with moderate AS. Critically, DAS Si has several features that make it generalizable and scalable. Unlike prior methods that have utilized structured echo reports and measurements, Doppler images, or still images of the aortic valve, DASSi can be directly applied to unprocessed, standard PLAX videos, without the need for any Doppler or two-dimensional measurements. This minimizes potential information loss and provides a quality-controlled, reader-independent metric to supplement a trained echocar di ographer’s impression. More importantly, however, the extensive multinational and multimodal evaluation of DASSi with consistent and robust effect sizes for both intermediate echocardiographic as well as long-term clinical outcomes, reproduced in participants undergoing both clinically-indicated and protocolized imaging, strongly support its integration into existing echocardiographic and CMR protocols, thus maximizing our ability to risk stratify for AS development and progression at no extra cost.

[0113] On the methodological front, this Example describes a paradigm to maximize the clinical value of novel Al-derived echocardiographic biomarkers. Indeed, DASSi was trained using a self-supervised, contrasting learning pre-text, which enabled the model to learn key representations of myocardial function and anatomy. Furthermore, by fine-tuning this against a severe AS phenotype, DASSi learned structural and functional parameters on two-dimensional PLAX views that generalized across lower severity stages, and successfully stratified the future risk of both AS development (among patients with no AS or sclerosis without stenosis) and AS progression. The independent confirmation of these associations in a distinct modality and a population that underwent protocolized as opposed to clinically-indicated imaging, supports that the prognostic value of DASSi is linked to the underlying myocardial / valvular phenotypic signature, rather than modality- or population-specific confounders. This is further supported by saliency maps and phenome-wide association studies confirming strong associations with traditional risk factors and links to diastolic dysfunction. It is notable however, that these only explain a small amount of the variability in DASSi, highlighting the need for multiparametric phenotyping using ALenhanced video interpretation. This framework represents a novel way to test the validity and generalizability of new Al-derived echocardiographic biomarkers.

[0114] In some cases, in the echocardiography study, the decision to pursue repeat imaging was based on clinical grounds rather than a study protocol. In some implementations, analyses revealed overall consistent results across varying levels of AS severity at baseline, for both echocardiographic and clinical outcomes, with the overall findings reproduced in the prospectively enrolled population of the UK Biobank.

[0115] Conclusion

[0116] This Example demonstrates the ability of deep learning-enhanced video-based phenotyping of cardiac anatomy and function to detect distinct clinical phenotypes and trajectories among patients with non-severe AS, which generalize across multi-national cohorts with varying risk profiles, as well as distinct non-invasive modalities, namely echocardiography and cardiac magnetic resonance. DASSi can be directly calculated through both standard, singleview, echocardiography by operators with minimal experience without the need for Doppler imaging, and long axis cine CMR views, enabling timely risk stratification for the most common valvular disorder without any changes in the image acquisition protocols. The proposed paradigm effectively integrates structural and temporal information into a unified index that determine AS progression and its associated morbidity and mortality.

[0117] Methods

[0118] Study Population and Data Source Echocardiography Study

[0119] This was a retrospective cohort study of patients without severe AS (no, mild or moderate AS) who underwent clinically indicated echocardiography for any indication and were followed longitudinally within their respective health systems. Eligible participants were drawn from centers affiliated with a hospital-based health system in New England (Yale-New Haven Health) and used to define two nested cohorts: (1) A longitudinal echocardiography cohort of individuals who underwent transthoracic echocardiography at two or more timepoints to assess the correlation of DASSi at baseline with longitudinal changes in Doppler-defined AS severity, and (2) a clinical outcomes cohort which included all individuals who had longitudinal follow-up for a composite clinical outcome of all-cause mortality and / or AVR.

[0120] Study Population and Data Source: CMR Study

[0121] UK Biohank The UK Biobank is a prospective observational study of 502,468 participants aged 40-69 years at the time of who were recruited between 2006 and 2010 which continues to collect extensive phenotypic and genotypic details using multimodal data capture. Though the UK Biobank is not perfectly representative of the broader UK population, its size, accuracy, depth of phenotypic and genomic characterization, and prospective nature have identified it as a valuable source for epidemiological research and validation of risk stratification tools in overall healthy community-dwelling individuals, that are often under-represented in hospital-based cohorts. 45,470 eligible individuals with a median age of 65 [IQR 59-71] years, 23,558 (51.8%) of whom were women were included in the study.

[0122] Echocardiogram Interpretation

[0123] All echocardiographic studies were performed by trained sonographers or cardiologists and reported by cardiologists board-certified in echocardiography. These reports were a part of routine clinical care in accordance with the recommendations of the American Society of Echocardiography. The presence of AS severity was adjudicated based on the original echocardiographic report. Further details on the measurements obtained are presented in the Supplement.

[0124] Magnetic Resonance Imaging (MRI) Pre-processing

[0125] In the UK Biobank, participants underwent a 20-minute cardiac magnetic resonance (CMR) protocol without a pharmacological stressor or contrast agent, which was integrated into a 30-minute combined CMR and abdominal MRI protocol performed using a clinical wide bore 1.5 Tesla scanner (MAGNETOM Aera®, Syngo Platform VD13A®, Siemens Healthcare®, Erlangen, Germany). Analysis was restricted to long axis cines for cardiac function through the left ventricular outflow tract view, which includes the same anatomical structures as the echocardiographic PLAX view which was used for the development of DAS Si on echocardiography. Next, an automatic pipeline was defined that extracts individual .DICOM files from each study-specific folder, identifies long axis cine views of the LVOT tract using the available .DICOM headers that suggest a 3 -chamber view (CINE-segmented-LAX-3Ch), windows the grayscale according to the default center and width of each study, converts the windowed data to 8-bit .avi files while selecting every other frame (thus creating 25-frame-long clips, by skipping every other clip in the original 50 frames of each view-specific cine) and down samples to 112 x 112 pixels. The sagittal clips are then rotated by 90 degrees to match the orientation of a PLAX view, cropped (removing 30 pixels from the new left / right side, 20 pixels from the top and 30 pixels from the bottom) to remove structures traditionally not seen on echocardiography, and the grayscale is then inversed to ensure that the myocardial wall appears brighter than its cavity. This enables an MRI2echo® transition that does not modify the composition of the underlying signal. The final clips, resampled to 112 x 112 pixels, can the be used for direct DASSi inference.

[0126] DASSi Calculation

[0127] The algorithm described in this Example provides a numerical probability of severe AS phenotype ranging from 0 (lowest probability of severe AS phenotype) to 1 (highest probability of severe AS phenotype). In echocardiography, deployment of the model involves the input of a full study, which is de-identified, down-sampled, and then processed for automated view classification to identify the specific videos from each study that correspond to PLAX views. The down-sampled 16-frame clips extracted from 2D PLAX videos are processed in a 3D- ResNetl8® network architecture trained to detect severe AS, and predictions are based on an ensemble of three models with a combination of three initializations: random, Kinetics-400, and self-supervised learning for echocardiograms. Model-specific study-level predictions represent the average predictions across all PLAX videos in a study for a given model. Finally, study-level predictions are averaged to form an ensemble, with the output (DASSi) reflecting the probability (from 0 to 1) of a severe AS phenotype across all videos of a given study. DASSi was computed for those echocardiograms without evidence of severe AS at baseline, spanning cases without AS, aortic sclerosis but no stenosis, as well as mild and moderate AS. Of note, DASSi can be calculated using any PLAX video as input independent of the vendor and hardware used to acquire the images. For CMR, the model is applied directly to the pre-processed PLAX-like video, with one study / video for each individual patient.

[0128] Definition of Outcomes

[0129] The primary echocardiographic outcome of the study was defined as the annualized rate of change in the AV Vmax, reported in m / s / year. For this, we calculated the rate of change for each pair of consecutive studies for the same patient. If three or more studies were present, we calculated the rate of change as the coefficient of a univariate ordinary least squares regression model of time against AV Vmax. To avoid the effect of extreme outliers no cases were excluded, but rather the average rate of change was winsorized to no less than -1 m / sec / year and no more than +2 m / sec / year, based on previously reported ranges. We purposefully chose this over the aortic valve area or mean gradient, given that the latter two indices are not consistently reported in patients with no or borderline Doppler findings for AS (missing in 36.8% and 29.0% of our cohort, respectively). To account for variability in the Doppler angle or flow states across studies, we also calculated the peak velocity ratio, defined as the ratio of the peak velocity in the left ventricular outflow tract (LVOT Vmax) to the AV Vmax. A secondary echocardiographic outcome was defined as the time to the next or higher severity stage on follow-up echocardiography (e.g., no AS to mild / moderate / severe, mild to moderate / severe and moderate to severe).

[0130] The primary clinical outcome of time-to-all-cause mortality or AVR was adjudicated by reviewing the linked institutional electronic health records which included the date of death and dates of relevant procedures. For the New England cohort, outcomes were assessed until April 18, 2023. AVR was defined based on procedure codes corresponding to percutaneous or open AVR with any valve type or valvuloplasty / aortic valve dilation, excluding procedures done within 90 days of the baseline TTE. Both in-hospital and out-of-hospital death reports were obtained from the vital statistics log maintained by the health system, drawn from social security administration and state vital statistic records.

[0131] Model Explanation

[0132] The explainability of the DAS Si model was further assessed in the context of its predictive assessment among patients without severe AS in a cross-modal setting using both saliency maps and phenome-wide association studies in the UK Biobank.

[0133] Saliency maps

[0134] Saliency maps were generated for the self-supervised part of the ensemble model using the Gradient-weighted Class Activation Mapping (Grad-CAM) method, a method that produces frame-by-frame “visual explanations” of where the model is focusing to make its predictions. To generate a single 2-dimensional heatmap for a given echo clip, the pixelwise maximum along the temporal axis was taken to capture the most salient regions across all timepoints.

[0135] Phenome-wide association study

[0136] A phenome-wide association study was performed for DAS Si, measured on CMR, against baseline phenotypes as assessed by ICD-10 codes in the UK Biobank. The PheWAS R package was used to automatically map ICD codes to 1,572 unique phenomic concepts (e.g., hypertension, diabetes mellitus, diverticulosis), adjusts estimates for relevant covariates (age, and sex), applies Bonferroni correction, and maps the results to a Manhattan plot.

[0137] Statistical Methods

[0138] Categorical variables are summarized as counts (valid percentages), whereas continuous variables are presented as mean ± standard deviation, or median [25th-75thpercentile]. Correlations between continuous variables are visualized using Loess regression plots that use local weighted regression to fit smooth curves through points of a scatterplot. Before inclusion in multivariable regression models, predictors with missing values were imputed using nonparametric chained equation imputation with random forests and n=5 iterations. Pairwise comparisons between continuous variables or an ordinal and a continuous variable were performed using Spearman’s rho (p) coefficient. For the rate of change in AV Vmax, we fit a generalized linear model adjusting for the patient’s age, sex, race, ethnicity, as well as AV Vmax and LVEF at baseline. Interactions between continuous covariates were modeled using an interaction term (i.e., AV Vmax x DASSi) and presented graphically using contour plots.

[0139] EQUIVALENTS

[0140] Although preferred embodiments of the invention have been described using specific terms, such description is for illustrative purposes only, and it is to be understood that changes and variations may be made without departing from the spirit or scope of the following claims.

Claims

CLAIMS1. A computer-implemented method of training a machine-learning model for multi-modal progression scores comprising: providing a training set including a plurality of imaging sequences from normal subjects and subjects with a heart condition; and training a machine-learning model to detect the heart condition on the training set.

2. The method of claim 1, wherein the normal subjects include subjects without the heart condition or with a lower severity heart condition.

3. The method of claim 1, wherein the heart condition includes cardiomyopathy or valvular disease.

4. The method of claim 3, wherein the heart condition comprises aortic stenosis.

5. The method of claim 4, wherein the normal subjects include at least one of: subjects without aortic stenosis, subjects with aortic sclerosis but no stenosis, subjects with mild aortic stenosis, or subjects with moderate aortic stenosis.

6. The method of claim 4, wherein the subjects with the heart condition are divided into subjects with non-severe aortic stenosis and subjects with severe aortic stenosis.

7. The method of claim 6, wherein the subjects with non-severe aortic stenosis include subjects with mild aortic stenosis, mild-moderate aortic stenosis, moderate, and moderate-severe aortic stenosis.

8. The method of claim 1, wherein the training comprises supervised learning with or without pre-training by self-supervised learning (SSL).

9. The method of claim 8, wherein the SSL comprises pretexts forcing the machine-learning model to learn generalizable temporal and cardiac function-related features, the pretexts including at least one of: a multi-instance contrastive learning task, a sequence re-ordering pretext task, a cross-modal contrastive learning task spanning videos, images of cardiac function, anatomy, and electrical activity, and a biometric learning task that identifies different videos drawn from the same subject to define subtle features from the different videos.

10. A computer-implemented method of detecting a heart condition comprising: receiving an input including a plurality of imaging sequences, each imaging sequence from the plurality of imaging sequences associated with a user from a plurality of users; extracting a clip for each imaging sequence, generating a plurality of clips; and inputting the plurality of clips to the machine-learning model according to any one of claims 1-9 to produce a phenotype score for a user from the plurality of users.

11. The computer-implemented method of claim 10, wherein: the plurality of imaging sequences in the training set comprise a first input type; the plurality of imaging sequences associated with the user comprise a second input type; and the method further comprises transforming the second input type to include an imaging view that resembles an imaging view from the first input type.

12. The computer-implemented method of claim 10, wherein the phenotype score indicates at least one of a level of severity or non-existence of aortic stenosis, the phenotype score further correlating with at least one of: traditional Doppler parameters of aortic stenosis presence, severity, and parameters of progressive diastolic dysfunction.

13. The computer-implemented method of claim 10, wherein the phenotype score is a numerical score ranging from 0 to 1.

14. The computer-implemented method of claim 10, wherein the phenotype score includes features comprising: a multi-instance contrastive learning task, a sequence re-ordering pretext task, a cross-modal contrastive learning task spanning videos, images of cardiac function, anatomy, and electrical activity, or a biometric learning task that identifies different videos drawn from the same subject to define subtle features from the different videos.

15. The computer-implemented method of claim 10, wherein the machine-learning model includes at least one of: a supervised machine-learning model, an unsupervised machine-learning model, and a self-supervised machine-learning model.

16. The computer-implemented method of claim 10, wherein the machine-learning model is trained using a training set that includes a plurality of imaging sequences correlated to a plurality of anatomical features of the heart condition.

17. The computer-implemented method of claim 10, wherein the machine-learning model is an ensemble machine-learning model.

18. The computer-implemented method of claim 10, wherein the machine-learning model is configured to: discern existence of the heart condition and produce digital biomarkers that predict severity of the heart condition, predict the development of the heart condition in patients without the heart condition, and predict progression of the heart condition in patients with a non-severe heart condition.

19. The computer-implemented method of claim 10, wherein the machine learning model uses traditional cardiovascular risk factors to determine, based on the digital biomarker, anintegrated risk score of aortic stenosis presence, future development, progression risk to guide appropriate clinical follow-up, or combinations thereof.

20. The computer-implemented method of claim 10, wherein the machine-learning model is trained such that the machine-learning is configured to produce the plurality of digital biomarkers via zero-shot predictions.

21. The computer-implemented method of claim 10, wherein executing the machine-learning model to produce the digital biomarkers includes executing the machine-learning model to produce the digital biomarkers from a plurality of digital biomarkers.

22. A computer-implemented method of detecting a heart condition comprising: receiving a first input including a plurality of imaging sequences, each imaging sequence from the plurality of imaging sequences associated with a user from a first plurality of users; extracting, for each imaging sequence, a clip from a first plurality of clips; executing a machine-learning model using the first plurality of clips to produce phenotype score for a set of features that is associated with the heart condition and with a user from the first plurality of users; receiving a second input including a plurality of datasets, each dataset from the plurality of datasets associated with a heart of a user from a second plurality of users; extracting, for each dataset, imaging data from a plurality of imaging data; executing the machine-learning model using the plurality of imaging data as an input to identify, for each imaging data from the plurality of imaging data, a plurality of imaging views with respect to a standardized view of the heart for that imaging data; compiling, for each imaging data, the plurality of imaging views to produce a compiled image sequence from a plurality of compiled image sequences; transforming each compiled image sequence from the plurality of compiled image sequences to generate a clip from a second plurality of clips, the clip from the second plurality of clips including an imaging view that resembles an imaging view from the first input; andrefining the phenotype score based on a comparison between the second plurality of clips and the first plurality of clips to generate a digital biomarker that is used to detect the heart condition based on the set of features associated with the heart condition.

23. The computer-implemented method of claim 22, wherein each of the receiving, extracting, identifying, compiling, transforming, and comparing, is performed automatically.

24. The computer-implemented method of claim 22, wherein the compiled image sequence is a cine video or still frame.

25. The computer-implemented method of claim 22, wherein transforming each compiled image sequence includes rotating, cropping to a cardiac outline, and inverting to grayscale, that compiled image sequence to produce the clip from the second plurality of clips.

26. The computer-implemented method of claim 22, wherein the method further includes confirming accuracy of the digital biomarker without Doppler imaging for phenotyping of aortic stenosis.

27. The computer-implemented method of claim 22, wherein the method further includes training the machine-learning model using a training set that includes a plurality of imaging sequences correlated to a plurality of anatomical features associated with the heart condition.

28. The computer-implemented method of claim 22, wherein the digital biomarker is configured to stratify a risk of development of the heart condition based on inputs of data types that are different from those of the first input and the second input.

29. The computer-implemented method of claim 22, wherein the computer-implemented method is executed on a desktop application, a cloud service, or a mobile device.

30. An apparatus comprising: a processor; anda memory operatively coupled to the processor, the memory storing instructions to cause the processor to: receive a first input including a plurality of imaging sequences, each imaging sequence from the plurality of imaging sequences associated with a user from a first plurality of users; extract, for each imaging sequence, a clip from a first plurality of clips; execute a machine-learning model using the first plurality of clips to produce phenotype score for a set of features that is associated with a heart condition and with a user from the first plurality of users; receive a second input including a plurality of datasets, each dataset from the plurality of datasets associated with a heart of a user from a second plurality of users; extract, for each dataset, imaging data from a plurality of imaging data; execute the machine-learning model using the plurality of imaging data as an input to identify, for each imaging data from the plurality of imaging data, a plurality of imaging views with respect to a standardized view of the heart for that imaging data; compile, for each imaging data, the plurality of imaging views to produce a compiled image sequence from a plurality of compiled image sequences; transform each compiled image sequence from the plurality of compiled image sequences to generate a clip from a second plurality of clips, the clip from the second plurality of clips including an imaging view that resembles an imaging from the first input; and refine the phenotype score based on a comparison between the second plurality of clips and the first plurality of clips to generate a digital biomarker that is used to detect a heart condition based on the set of features associated with the heart condition.

Images