Anatomical and functional assessment of coronary artery disease using machine learning

Abstract

To provide anatomical and functional assessment of coronary artery disease (CAD) using machine learning and computational modeling techniques.SOLUTION: Anatomical and functional assessment of coronary artery disease (CAD) using machine learning and computational modeling techniques deploys methodologies for non-invasive Fractional Flow Reserve (FFR) quantification based on angiographically derived anatomy and hemodynamics data, relies on machine learning algorithms for image segmentation and flow assessment, and relies on accurate physics-based computational fluid dynamics (CFD) simulation for computation of the FFR.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] Cross - Reference to Related Applications This application claims the benefit of priority to U.S. Provisional Patent Application No. 62 / 939,370, filed on November 22, 2019, entitled "Anatomical and Functional Assessment of CAD Using Machine Learning", the entire disclosure of which is hereby expressly incorporated by reference herein.

[0002] The present invention generally relates to the fully automated detection of coronary vessels and their branches in angiograms, and more specifically, to the calculation of the diameter of such vessels, the detection of stenosis, and the determination of the stenosis narrowing rate and the functional blood flow limitation from the stenosis.

Background Art

[0003] The description of the background art provided herein is for the purpose of generally presenting the context of the present disclosure. The research of the inventors named as such in the present context of this background art section, and aspects of the description that may not be considered prior art at the time of filing in another way, are not admitted as prior art to the present disclosure, either expressly or implicitly.

[0004] Coronary artery disease (CAD) is one of the leading causes of death in the United States, affecting more than 15 million Americans. CAD is characterized by the accumulation of atherosclerotic plaques in the coronary arteries, which can cause narrowing (also known as stenosis) or occlusion of the coronary arteries, and may cause symptoms such as angina and potentially myocardial infarction.

[0005] As used herein, the "state of vascular occlusion" generally refers to what is understood as CAD, for example, narrowing (localized or diffuse) of the epicardial coronary arteries as seen in imaging studies and characterized by either anatomical or functional indicators. Conversely, the "state of microvascular disease" refers to a disease of the coronary microcirculation characterized by a loss of vasodilatory capacity.

[0006] The main invasive diagnostic method for CAD is coronary angiography, in which a contrast agent is injected into the patient's blood vessels via a catheter and imaged to characterize the severity of the stenosis. This method relies on the visualization of anatomical abnormalities, and the visual inspection is semi-quantitative as it only approximates the lumen area reduction rate. A diameter reduction estimated to be 70% or more often leads to further evaluation or revascularization, such as coronary artery stenting.

[0007] A more quantitative approach based on physiology for evaluating coronary artery stenosis is to calculate the fractional flow reserve (FFR), a measurement criterion defined as the ratio of the hyperemic blood flow in the diseased artery to the expected hyperemic blood flow in the same artery without stenosis. For example, in the coronary artery, the FFR can be expressed as the ratio of the distal coronary artery pressure to the proximal coronary artery pressure. For example, if the FFR is less than 0.80, it indicates the presence of severe stenosis requiring revascularization as the blood flow to the vascular bed distal to the vessel is impaired. The decision of revascularization incorporating FFR has been shown to improve the outcome compared to angiography alone. The determination of FFR is robust against various patient geometries, for example, taking into account the contribution of collateral vessels to blood flow and lesion geometry.

[0008] Despite its advantages, due to the invasive nature of using catheter-based pressure measurements, medical specialists often do not measure the FFR of patients. Some have found that physicians choose not to perform FFR in nearly two-thirds of all cases due to patient risk, lack of resources, and additional costs. Another drawback of FFR is its variability due to various hemodynamic states within the patient.

[0009] There is a need for a more accurate and less invasive technique for diagnosing CAD. More specifically, there is a need for a non-invasive and user-independent approach for measuring FFR that would not pose a risk to the patient.

[0010] Recently, some have proposed non-invasive computational workflows for determining FFR and assessing the severity of CAD. These efforts have adopted two different approaches. The first approach relies on computed tomography angiography (CTA) data to reconstruct the 3D geometry of the coronary artery tree. The vessel lumen is segmented using edge detection or machine learning algorithms, and the output is manually corrected by an expert. Computational fluid dynamics (CFD) simulations are performed on the vessel geometry derived from CTA, and FFR is calculated on the company's server or in the cloud. This non-invasive approach has shown promising results in clinical trials, but the use of CTA is greatly affected by imaging artifacts due to calcification, making it difficult to delineate the boundaries of the vessel lumen. CTA has limited spatial resolution compared to angiography, which also limits its ability to detect small, consecutive lesions or capture fine vessel geometry. Finally, since CTA data does not provide information about blood flow, the boundary conditions (BC) for CFD analysis of hemodynamics typically rely on morphometric data or population data and are not patient-specific.

[0011] A second approach based on a non-invasive computational workflow reconstructs the vascular geometry relying on multi-plane angiography data before performing a physics-based blood flow simulation. The main advantages of using angiography are the excellent ability to detect the vascular lumen boundary in the presence of calcified stenoses and the high spatial resolution compared to CTA, which improves the sensitivity of the geometry reconstruction algorithm. Furthermore, time-resolved angiography has information on how the contrast agent moves within the vessels and can thus be used to estimate blood flow velocity and provide information for the BC for CFD analysis. However, there are fundamental challenges with angiography-based approaches for FFR quantification. Namely, the reconstruction of the 3D geometry of the vessel of interest from a series of 2D images acquired at various positions of the patient table over time. Furthermore, all angiography-based approaches for FFR quantification have created a workflow that requires substantial operator input to identify the vessel of interest and assist with segmentation. Finally, all angiography-based approaches have considered the reconstruction of a single coronary artery or modeled the physics of blood flow using a very simplified approach. These drawbacks effectively offset the advantages of using high-resolution angiography data, which is the most commonly performed procedure in CAD diagnosis.

[0012] Regardless of the approach, all FFR methods derived from calculations have shown low predictive performance around the important diagnostic value of FFR = 0.8 due to the aforementioned limitations in image data quality, lack of information on blood flow, need for operator input, and computational modeling assumptions. Therefore, there is currently no pipeline for accurate FFR calculation that can be effectively deployed in any hospital nationwide.

[0013] There is a significant need for more accurate and less invasive technologies for CAD diagnosis that use non-invasive approaches for fully automated FFR measurement. SUMMARY OF THE INVENTION

[0014] Techniques are provided for the anatomical and functional evaluation of coronary artery disease (CAD) using machine learning and computational modeling techniques. The techniques herein use a new methodology for non-invasive fractional flow reserve (FFR) quantification based on anatomical structures and hemodynamic data derived from angiography, rely on machine learning algorithms for image segmentation and blood flow evaluation, and rely on accurate physics-based computational fluid dynamics (CFD) simulations for the calculation of FFR to address the drawbacks of conventional CAD evaluations.

[0015] In an exemplary embodiment, the technology provides a process for evaluating both the anatomical and functional severity of CAD through the use of both static and dynamic coronary angiography data and through customized machine learning and computational modeling methods, with the ultimate goal of determining the FFR in a less risky and more accurate manner.

[0016] The use of functional markers of disease has gained significant traction in the past few years, supplanting markers based on simpler anatomical structures. In the field of cardiology, fractional flow reserve (FFR) is a hemodynamic metric (i.e., the normalized pressure gradient under conditions of maximal blood flow) that has shown superior diagnostic results compared to anatomy-based markers. In the exemplary embodiments herein, two-dimensional (2D) angiography data, specifically, dynamic angiography data that provides an account of the transport of a contrast agent (dye) down the vessel of interest, is used. The 2D time-resolved angiography data is used to provide information to the computational simulation, thereby obtaining a more accurate prediction of the FFR than would be obtained without the dynamic transport of the contrast agent down the vessel. Further, in the exemplary embodiments, three-dimensional (3D) geometric models are generated from the 2D angiography data, and those 3D geometric models are used in the simulation of hemodynamic states, including the FFR.

[0017] According to one example, a computer-implemented method for evaluating CAD includes receiving, by one or more processors, angiographic image data of a target vascular examination region, where the angiographic image data includes angiographic images captured over a sampling period; applying, by one or more processors, the angiographic image data to a vascular segmentation machine learning model and using the vascular segmentation machine learning model to generate a two-dimensional (2D) segmented vascular image of the vascular examination region; generating, by one or more processors, a three-dimensional (3D) segmented vascular tree geometry model of blood vessels within the vascular examination region from the 2D segmented vascular image; applying, by one or more processors, the 3D segmented vascular tree geometry model to a fluid dynamics machine learning model and using the fluid dynamics machine learning model to assimilate blood flow data over the sampling period of one or more blood vessels within the vascular examination region; applying, by one or more processors, the 3D segmented vascular tree geometry model and the assimilated blood flow data to a 3D high-fidelity computational fluid dynamics model; and determining, by one or more processors, a state of vascular occlusion of one or more blood vessels within the vascular examination region.

[0018] According to one example, a computing device configured to evaluate CAD includes one or more processors that, when executed, cause the one or more processors to receive, by the one or more processors, angiographic image data of a target vascular examination region, wherein the angiographic image data includes angiographic images captured over a sampling period; cause the one or more processors to apply the angiographic image data to a vascular segmentation machine learning model and use the vascular segmentation machine learning model to generate a two-dimensional (2D) segmented vascular image of the vascular examination region; cause the one or more processors to generate a three-dimensional (3D) segmented vascular tree geometry model of blood vessels within the vascular examination region from the 2D segmented vascular image; cause the one or more processors to apply the 3D segmented vascular tree geometry model to a fluid dynamics machine learning model and use the fluid dynamics machine learning model to assimilate blood flow data over a sampling period of one or more blood vessels within the vascular examination region; cause the one or more processors to apply the 3D segmented vascular tree geometry model and the assimilated blood flow data to a 3D high-fidelity computational fluid dynamics model; and cause the one or more processors to determine a state of vascular occlusion of one or more blood vessels within the vascular examination region. The computing device further includes one or more computer-readable memories storing instructions that cause the above-described operations to be performed.

[0019] According to another example, a computer-implemented method for evaluating coronary artery disease includes receiving, by one or more processors, a plurality of angiographic images of a vascular examination region of a subject, where the angiographic images are captured over a sampling period and the vascular examination region includes one or more blood vessels; applying, by one or more processors, the angiographic images to a vascular segmentation machine learning model and using the vascular segmentation machine learning model to generate a two-dimensional (2D) segmented vascular image of the one or more blood vessels; generating, by one or more processors, a one-dimensional (1D) segmented vascular tree geometry model of the one or more blood vessels from the 2D segmented vascular image; applying, by one or more processors, the 1D segmented vascular tree model to a fluid dynamics machine learning model and using the fluid dynamics machine learning model to assimilate blood flow data over the sampling period of the one or more blood vessels; applying, by one or more processors, the 1D segmented vascular tree model and the assimilated blood flow data to a low-dimensional model based on graph theory based on a computational fluid dynamics model; and determining, by one or more processors, the state of vascular occlusion.

[0020] A computing device configured to evaluate CAD includes one or more processors that, when executed, cause the one or more processors to receive, by the one or more processors, plural angiographic image data of a target vascular examination region, where the angiographic images are captured over a sampling period and the vascular examination region includes one or more blood vessels; cause the one or more processors to apply the angiographic images to a vascular segmentation machine learning model and use the vascular segmentation machine learning model to generate a two-dimensional (2D) segmented vascular image of the one or more blood vessels; cause the one or more processors to generate, from the 2D segmented vascular image, a one-dimensional (1D) segmented vascular tree geometry model of the one or more blood vessels; cause the one or more processors to apply the 1D segmented vascular tree model to a hydrodynamic machine learning model and use the hydrodynamic machine learning model to assimilate blood flow data over the sampling period of the one or more blood vessels; cause the one or more processors to apply the 1D segmented vascular tree model and the assimilated blood flow data to a low-dimensional model based on graph theory based on a computational fluid dynamics model; and cause the one or more processors to determine a state of vascular occlusion, and one or more computer-readable memories storing instructions to cause the above.

Brief Description of the Drawings

[0021] The drawings described below illustrate various aspects of the systems and methods disclosed herein. It should be understood that each figure shows an embodiment of a particular aspect of the disclosed systems and methods, and each figure is intended to be consistent with its possible embodiments. Further, to the extent possible, the following description refers to the reference numbers included in the following figures, and features shown in multiple figures are designated by consistent reference numbers.

[0022]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10A

Figure 10B

Figure 11-1

Figure 11-2

Best Mode for Carrying Out the Invention

[0023] Using physics-based machine learning from neural networks and degree reduction of high-fidelity models using graph theory, together with machine learning-based anatomical segmentation and high-fidelity computer simulations of hemodynamics, to analyze dynamic angiography image data to perform an anatomical and functional evaluation of coronary artery disease (CAD) and provide an automated workflow that can provide excellent diagnostic performance for CAD. Technologies are provided for performing non-invasive fractional flow reserve (FFR) quantification based on angiography data, relying on machine learning algorithms for image segmentation, and relying on physics-based machine learning and computational fluid dynamics (CFD) simulations for more accurate functional evaluation of blood vessels. Specifically, in some examples, two-dimensional dynamic angiography data is used to capture the transport of dye down the vessel of interest. This dynamic information can be used to provide information to computational simulations and thus can be used to make a more accurate prediction of FFR, especially in the case of the boundary line. Furthermore, although angiography data does not provide three-dimensional anatomical information, current techniques include a process for developing an image reconstruction algorithm and then obtaining three-dimensional (3D) and one-dimensional (1D) geometric models of the patient's vasculature for use in computer simulations of hemodynamics. Although the techniques are described herein with respect to determining FFR, the same techniques may be used to calculate other fractional flow reserve ratio metrics. Thus, references to determining FFR in the examples herein include determinations such as instantaneous fractional flow reserve (iFR), quantitative coronary flow ratio (QFR), and the like.

[0024] In some examples, systems and methods for evaluating coronary artery disease are provided. The system may receive angiographic image data of a vascular examination region of a subject. The angiographic image data may contain a plurality of angiographic images captured over a sampling period. The system may apply the angiographic image data to a vascular segmentation machine learning model, which is a first machine learning model. The vascular segmentation machine learning model may generate two-dimensional (2D) segmented vascular images of the vascular examination region, from which a three-dimensional (3D) geometric vascular tree model that models the blood vessels having the vascular examination region is generated. In other examples, a one-dimensional (1D) equivalent vascular tree model may be generated from the 3D vascular tree model. The 3D or 1D geometric vascular tree model may be applied to a second machine learning model, a fluid dynamics machine learning model, to assimilate blood flow data over a sampling period of one or more blood vessels within the vascular examination region. From the assimilated blood flow data and from the 3D or 1D geometric vascular tree model, a computational fluid dynamics model is configured to determine the state of the blood vessels in the vascular system, and these states may include the state of vascular occlusion and / or the state of microvascular disease / resistance. Specifically, to determine microvascular disease / resistance, the angiographic images may be acquired and compared under two different hemodynamic states, one being a baseline state and a hyperemic (high flow) state. In yet other examples, the microvascular system may be evaluated only from examining angiographic images captured during the hyperemic state.

[0025] In FIG. 1, a CAD evaluation system 100 includes a computing device 102 (or “signal processor” or “diagnostic device”) configured to collect angiographic image data from a patient 120 via an angiographic imaging device 124 by performing the functions of the disclosed embodiments. As shown, the system 100 may be implemented on one or more processing units 104, which may represent a central processing unit (CPU) of the computing device 102, and / or on one or more graphics processing units (GPUs) including a cluster of CPUs and / or GPUs, all of which may be cloud-based. The features and functions described for the system 100 may be stored in and then implemented from one or more non-transitory computer-readable media 106 of the computing device 102. The computer-readable media 106 may include, for example, an operating system 108 and a CAD machine learning (deep learning) framework 110 having elements corresponding to elements of the deep learning framework described herein. More generally, the computer-readable media 106 may store trained deep learning models including, for example, a vascular segmentation machine learning model, a fluid dynamics machine learning model, a low-dimensional model based on graph theory, executable code, etc., for implementing the techniques herein. The computer-readable media 106 and the processing unit 104 may store image data, segmentation models or rules, hydrodynamic classifiers, and other data herein in one or more databases 112. As discussed in the examples herein, a CAD machine learning framework 110 that applies the techniques and processes herein (e.g., various different neural networks) may generate 3D and / or 1D segmented vascular tree geometry models, FFR and other hydrodynamic evaluations, vascular occlusion state data, and / or microvascular disease data.

[0026] The computing device 102 includes a network interface 114 communicatively coupled to a network 116 for communicating to and / or from a portable personal computer, smartphone, electronic document, tablet, and / or desktop personal computer, or other computing device. The computing device further includes an I / O interface 115 connected to devices such as a digital display 118, user input device 122, etc. As described herein, the computing device 102 may generate, as an electronic document, a display of CAD of a subject that may include the state of blood vessels in the vasculature, such as the state of vascular occlusion (via FFR calculation, iFR calculation, or QFR calculation, anatomical and functional) and the state of prediction of microvascular disease (by contrasting changes in peripheral resistance when two hemodynamic states are recorded), which can be accessed and / or shared on the network 116.

[0027] In the illustrated example, computing device 102 is communicatively coupled to an electronic medical record (EMR) database 126 via network 116. EMR 126 may be a network-accessible database or a dedicated processing system. In some examples, EMR 126 includes data regarding each of one or more patients. The EMR data includes vital sign data (e.g., hemoglobin oxygen saturation derived from pulse oximetry, heart rate, blood pressure, respiratory rate), lab data such as complete blood count (e.g., mean platelet volume, hematocrit, hemoglobin, mean corpuscular hemoglobin, mean corpuscular hemoglobin concentration, mean corpuscular hemoglobin amount, white blood cell count, platelets, red blood cell count, and red blood cell distribution width), lab data such as basic metabolic panel (e.g., blood urea nitrogen, potassium, sodium, glucose, chloride, CO2, calcium, creatinine), demographic data (e.g., age, weight, race and gender, zip code), less common lab data (e.g., bilirubin, partial thromboplastin time, international normalized ratio, lactate, magnesium and phosphite), and other suitable patient metrics that currently exist or may be developed in the future (e.g., O2, use of Glasgow coma score or components thereof, and urine output, antibiotic administration, blood transfusion, fluid administration, etc. over the past 24 hours), and calculated values including shock index and mean arterial pressure may be included. The EMR data may additionally or alternatively include chronic medical and / or surgical conditions. The EMR data may include past data collected from previous patient examinations, including past FFR, iFR, or QFR data. Previous determinations of stenosis, prediction of vascular disease, vascular resistance, CFD simulation data, and other data generated by the techniques herein. EMR 126 may be updated when new data is collected from angiography device 124 and evaluated using computing device 102. In some examples, the techniques may provide continuous training of EMR 126.

[0028] In conventional angiography imaging applications, angiography images are captured by a medical imaging device and then transmitted to an EMR for further processing, including storage and, in some instances, image processing before those images are sent to a medical professional. In the present technique, the state of occlusion and the state of microvascular disease can be determined on a computing device based on angiography images without first offloading those images to an EMR126 for processing. Overall, the technique proposed here can significantly reduce the analysis time for a cardiologist, in part, by bypassing this EMR126 for processing. The EMR126 may simply be polled for data during analysis by the computing device 102 and may be used for storage of the state determinations and other calculations generated by the techniques herein. Indeed, there are many advantages resulting from a faster and more automated analysis than occurs with current techniques. For example, modeling and vascular occlusion / disease state analysis can be performed sequentially and individually for vessels corresponding to either the left or right coronary artery tree, but the cardiologist's results are still generated in minutes, for example, using a 3D or 1D modeler as described herein.

[0029] In the illustrated example, system 100 is implemented on a single server. However, the functionality of system 100 may be implemented across distributed devices connected to each other via communication links. In other examples, the functionality of system 100 may be distributed across any number of devices, including the portable personal computers, smartphones, electronic documents, tablets, and desktop personal computer devices shown. In other examples, the functionality of system 100 may be cloud-based, such as one or more connected cloud CPUs customized to perform machine learning processes and the computing techniques described herein, or a computing system labeled 105. Network 116 can be a public network such as the Internet, a private network such as a research institution or corporate private network, or any combination thereof. The network includes local area networks (LANs), wide area networks (WANs), cellular, satellite, or other network infrastructure, whether wireless or wired. The network can utilize communication protocols including packet-based and / or datagram-based protocols such as Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or other types of protocols. Additionally, network 116 can include multiple devices that facilitate network communication and / or form the hardware infrastructure of the network, such as switches, routers, gateways, access points (such as wireless access points as shown), firewalls, base stations, repeaters, backbone devices, etc.

[0030] The computer-readable medium 106 may include executable computer-readable code stored thereon for programming a computer (e.g., including a processor and a GPU) with the techniques of this specification. Examples of such computer-readable storage media include hard disks, CD-ROMs, digital versatile disks (DVDs), optical storage devices, magnetic storage devices, ROM (read-only memory), PROM (programmable read-only memory), EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), and flash memory. More generally, the processing unit of the computing device 102 may represent a CPU-type processing unit, a GPU-type processing unit, a field programmable gate array (FPGA), another class of digital signal processor (DSP), or other hardware logic components drivable by a CPU.

[0031] The exemplary deep learning frameworks of this specification have been described as being composed of exemplary machine learning architectures, but it should be noted that any number of suitable convolutional neural network architectures can be used. Generally speaking, the deep learning frameworks of this specification can implement any appropriate statistical model (e.g., a neural network or other model implemented through a machine learning process) applied to each of the received images. As discussed in this specification, the statistical model can be implemented in a wide variety of ways. In some examples, the machine learning model has the form of a neural network, a support vector machine (SVM), or other machine learning process, is trained using images or multi-dimensional datasets, and develops models for vessel segmentation or hydrodynamic calculations. Once these models are appropriately trained with a series of training images, the statistical model can be used in real time to analyze subsequent angiography image data provided as input to the statistical model for determining the presence of CAD and for determining the state and disease of vessel occlusion. In some examples, when the statistical model is implemented using a neural network, the neural network can be configured in a wide variety of ways. In some examples, the neural network can be a deep neural network and / or a convolutional neural network. In some examples, the neural network can be a distributed and scalable neural network. The neural network can be customized in a wide variety of ways, such as providing a specific top layer such as a logistic regression top layer. A convolutional neural network can be regarded as a neural network that includes a set of nodes associated with parameters. A deep convolutional neural network can be regarded as having a structure in which multiple layers are stacked. Neural networks or other machine learning processes can include various sizes, numbers of layers, and levels of connectivity. Some layers can correspond to stacked convolutional layers (optionally followed by contrast normalization and max pooling), followed by one or more fully connected layers.This technology may be implemented such that machine learning training can be performed using a small dataset, for example, less than 10,000 images, less than 1,000 images, or less than 500 images. In one example, approximately 400 images were used. To avoid overfitting, a multiple split cross-validation process can be used (e.g., 5-fold cross-validation). In some examples, a regularization process such as L1 or L2 can be used to avoid overfitting. In the case of a neural network trained with a large dataset, for example, more than 10,000 images, the number of layers and the size of the layers can be increased by using dropout to address the potential problem of overfitting. In some cases, the neural network can be designed to refrain from using fully connected upper layers at the top of the network. By forcing dimensionality reduction in the intermediate layers of the network, a very deep neural network model can be designed while dramatically reducing the number of learned parameters.

[0032] Figure 2 shows an exemplary deep learning framework 200 that is an example of the CAD machine learning framework 110 of computer device 102. The framework 200 includes a 3D / 1D segmented vascular tree geometry model generator 204 that includes a preprocessor stage 205, two neural networks, an angiography processing neural network (APN) 202 and a vascular segmentation machine learning model 206 that includes a second stage, and a semantic neural network 207.

[0033] In the illustrated example, the preprocessor 205 receives the clinical angiography image 203A along with data regarding the contrast agent injection used to form it. Optionally, the preprocessor 205 may be coupled to receive a synthetic angiography image 203B, for example, for machine learning training. Further, the preprocessor 205 may be coupled to receive a geometrically adjusted vascular image 203C. In some examples, these inputs may be supplied directly to the vascular segmentation machine learning model 206, more specifically the APN 202. The preprocessor 205 can perform various preprocessing, which may include a noise removal process, a linear filtering process, an image size normalization process, and a pixel intensity normalization process on the received image data.

[0034] The deep learning framework 200 may operate in two different modes, a machine learning training mode and an analysis mode. In the machine learning training mode of the framework, angiography image data 203A, synthetic angiography image data 203B, and / or geometrically adjusted angiography image data 203C (such as horizontal or vertical flips, zoom at any level, rotation, or shear) may be provided to the APN 202. Depending on the data type and data source, different preprocessing functions and values can be applied to the received image data. In the analysis mode in which the machine learning model is being trained, the captured angiography image data 203A of the subject is provided to the APN 202 for analysis and CAD determination. In any mode, the preprocessed image data is received, processed by the APN 202 and the semantic NN 207, and in the analysis model, is provided to a 3D / 1D segmented vascular tree geometry model generator 204 that includes a segmentation machine learning model 206 that generates a 2D segmented vascular image. Thus, the vascular segmentation machine learning model 206 may be a convolutional neural network, such as two different convolutional neural networks with a hierarchical configuration, as shown in the example of FIG. 5. Thus, in some examples, the semantic NN 207 is configured as an unchanged or modified Deeplabv3+ architecture.

[0035] The 3D / 1D segmented vascular tree geometry model generator 204 further includes a 3D modeler 208 configured to generate a 3D vascular tree geometry model of the target region based on the 2D segmented vascular image.

[0036] Once the 3D vascular tree model is generated, the generator 204 may apply further smoothing algorithms and / or surface spline approximation algorithms to further improve the 3D vascular tree model for 3D (e.g., high-fidelity) hemodynamic classification and occlusion analysis.

[0037] To increase the analysis of processing time and the state of vascular occlusion of larger blood vessels and the state of microvascular disease of smaller blood vessels, in some examples, the techniques herein are implemented in a low-dimensional model. In some examples, a 3D segmented vascular tree geometry model generated from captured 2D angiographic images is further reduced to generate a 1D segmented vascular tree geometry model, yet still maintained such that sufficient data is provided for FFR, iFR, or QFR determination, and computational fluid dynamics modeling. To implement model order reduction, in some examples, the vascular tree geometry model generator 204 includes a 1D modeler 209. The 1D modeler 209 creates a skeletonization of the 3D segmented vascular tree model given by the path / centerline of the 3D space of the blood vessels included in the 3D segmented vascular tree model, and a series of 2D cross-sectional contours separated at any distance along each path / centerline of the tree. An exemplary 1D segmented vascular geometry tree model generated from the 3D segmented vascular geometry tree model is shown in FIG. 6.

[0038] The 3D or 1D vascular tree geometry model from generator 208 or 209 is provided to a blood flow data generator 210 that includes a fluid mechanics learning model 212, which may include at least one network of the types of low-dimensional models of blood flow and pressure based on convolutional neural networks (CNNs), autoencoders, or long short-term memory (LSTM), or graph theory.

[0039] As shown, the hydrodynamic machine learning model 212 may include many different types of models, whether trained or not. In some examples, the hydrodynamic machine learning model 212 is a deep learning framework based on Navier-Stokes information configured to determine pressure and velocity data across a 3D vascular space or a 1D vascular space according to a modeler that provides input 208 or 209. In some examples, the deep learning framework based on Navier-Stokes information includes one or more methods of the type of Kalman filtering, physics-informed neural networks, iterative assimilation algorithms based on contrast arrival times at anatomical landmarks, and TIMI frame counting. Dynamic data (see, e.g., FIG. 8) regarding a series of images depicting the transport of a dye down a blood vessel of interest is used to assimilate information regarding blood flow velocity. In some cases, a clinical angiogram is obtained using a contrast agent injection system 203D having accurate information regarding, for example, the pressure applied to the dye bolus, the amount of dye, the timing, etc., and additional information for the hydrodynamic machine learning model may be provided as also shown.

[0040] In other examples, the hydrodynamic machine learning model 212 includes a low-dimensional model based on graph theory obtained through a graph of discrepancies between ground truth data (in silico or clinical) including geometric shapes, pressures, blood flows, and metrics such as FFR, iFR, or QFR. Any of the techniques herein for defining a low-dimensional model based on graph theory can generate results faster compared to occlusion analysis techniques based on finite element modeling (FEM) or other 3D techniques. Further, the techniques herein can model and analyze not only large blood vessels but also the microvasculature, and thus can determine the state of occlusion in large blood vessels and the state of microvascular disease in small blood vessels.

[0041] More generally, the model 212 is configured to determine blood flow data assimilated over a sampling period for one or more blood vessels within a 3D vascular tree geometry model or a 1D vascular tree geometry model. Such determination may include determining the pressure and / or flow velocity of a plurality of connected blood vessels in the 3D vascular tree geometry model or the 1D vascular geometry tree model.

[0042] In some examples, the lumped parameter boundary condition parameters are determined by the hydrodynamic machine learning model 212 of one or more blood vessels in the blood vessel examination region. In some examples, the hydrodynamic machine learning model 212 determines a lumped parameter model of the blood flow in the first blood vessel and a lumped parameter model of the blood flow in each blood vessel branching from the first blood vessel. Any of these may be stored as assimilated blood flow data.

[0043] The blood flow data generator 210 and the assimilated blood flow data from the 3D vascular tree model (or 1D vascular tree model) are provided to a computational fluid dynamics model 214 that may apply physics-based processes to determine the state of blood vessel occlusion and / or the state of microvascular disease of one or more blood vessels within the 3D vascular tree model (or 1D vascular tree model). In some examples, the computational fluid dynamics model includes one or more of a multi-scale 3D Navier-Stokes simulation using a low-dimensional (lumped parameter) model, a low-dimensional Navier-Stokes (1D) simulation using a low-dimensional model derived from a graph theory framework that depends on a 1D non-linear theory model, or a low-dimensional model simulation (lumped parameter model, 0D) of the entire segmented vascular tree model. In the example shown, the computational fluid dynamics model includes, by way of example herein, at least a 3D high-fidelity training model 211 and a graph theory information low-dimensional model 213.

[0044] In some examples, the computational fluid dynamics model 214 is configured to determine the FFR, iFR, and / or QFR of one or more blood vessels within a 3D or 1D blood vessel tree model from blood flow data. In some examples, the computational fluid dynamics model 214 is configured to determine the state of a blood vessel occlusion from the FFR, iFR, and / or QFR of one or more blood vessels. In some examples, the computational fluid dynamics model 214 determines the coronary flow reserve (CFR) of one or more blood vessels from blood flow data from one or more physiological states (baseline and hyperemic), and is configured to determine the state of microvascular disease from the CFR of one or more blood vessels. Determining the state of a blood vessel occlusion includes determining the presence of stenosis in one or more blood vessels. Determining the state of microvascular disease includes determining a lumped parameter model on the boundary of a blood vessel in a blood vessel examination region.

[0045] FIG. 3 shows a process 300 for evaluating coronary artery disease that may be performed by the system 100. 2D angiographic image data is obtained by the medical imaging device 116 and provided to the computer device 102 in a process 302 where optional preprocessing operations may be performed. In a training mode, the 2D angiographic image data may be captured clinical angiographic image data, and such training data may further include synthetic image data, geometrically adjusted image data, etc., as shown in FIG. 2.

[0046] In one example, training of the vascular segmentation machine learning model 206 is performed on 462 clinical angiography images (see, e.g., FIG. 8) augmented via a combination of geometric transformations (zoom, horizontal flip, vertical flip, rotation, and / or shear), and 461 synthetic angiography images (see, e.g., FIG. 9) augmented to an additional set of over 500,000 angiography images and over 500,000 images via these geometric transformations are formed. The clinical angiography images may, as shown, include time series data of multiple frames with segmentation for extraction speed across the entire vascular tree. In some examples, the vascular segmentation machine learning model 206 includes a synthetic image generator configured to generate synthetic images using a combination of transformations such as flipping, shearing, rotating, and / or zooming. In any case, these numbers of images are provided only as empirical examples, and any suitable number of training images captured or synthesized may be used. In the training mode, the image data is applied to the CAD evaluation machine learning framework 110 for the generation of two types of models, the vascular segmentation machine learning model 206 and the hemodynamics machine learning model 212. In the diagnostic mode, the image data is applied to the CAD evaluation machine learning framework 110 for CAD classification and diagnosis.

[0047] In process 304, the CAD evaluation machine learning framework 110 applies the received image data to the vascular segmentation machine learning model, via, for example, the APN 202 and the semantic NN 207 of the vascular segmentation machine learning model 206, to generate a 2D segmented vascular image. The CAD evaluation machine learning framework 110, such as via the 3D modeler 208, receives the 2D segmented vascular image and, in process 306, generates a 3D segmented vascular tree model or a 1D segmented vascular tree model.

[0048] In process 308, the 3D segmented vascular tree model or the 1D segmented vascular tree model is applied to the hydrodynamic machine learning model 212 of the blood flow data generator 210, and the assimilated blood flow data is generated over the sampling period of one or more blood vessels in the 3D vascular tree model or the 1D segmented vascular tree model.

[0049] In process 310, the assimilated blood flow data and the 3D vascular tree model or the 1D segmented vascular tree model are applied to the computational fluid dynamics model 214 that evaluates the data using either the 3D segmented vascular tree model or the 1D vascular tree model, and the health state of the blood vessel is determined by solving either the 3D Navier-Stokes equation or the low-dimensional model based on graph theory to determine the state of vascular occlusion through indicators such as FFR, iFR, QFR, etc. When data regarding two hemodynamic states (e.g., baseline and hyperemic state) is available, the state of microvascular disease, or CFR, is determined from the lumped parameter values at the boundary state of each hemodynamic state.

[0050] The process 400 shown in FIG. 4 is an exemplary implementation of the process 304 for generating a 2D segmented vascular image from angiographic image data that may be executed by the angiographic processing network 202 and the semantic NN 207 of the vascular segmentation machine learning model 206. First, in process 302, 2D angiographic image data of a target vascular region is received. In process 402, image preprocessing (including an image size normalization process and / or a pixel intensity normalization process) is applied before further image processing via an angiographic processing network implemented with a convolutional neural network (e.g., APN 202), and the further preprocessing may include a non-linear filtering process, noise removal, and / or contrast enhancement, which, when combined with process 404, filters out objects such as catheters and bone structures. That is, in process 404, the preprocessed 2D angiographic image is applied to a second convolutional neural network (CNN) trained using clinical angiographic image data, synthetic image data, and / or geometrically modified image data (e.g., semantic NN 207). FIG. 5 shows an exemplary CNN framework 500 configured in combination with an angiographic processing network (APN) 503 and a semantic NN in the form of a Deeplab v3+ network 507 configured to generate a 2D segmented vascular image in process 408. The Deeplab v3+ network 507 is a semantic segmentation deep learning technique including an encoder 502 and a decoder 504. A series of angiographic images 501 are provided as input to an angiographic processing network 503 consisting of several 3×3 and 5×5 convolutional layers, which applies a non-linear filter that performs contrast enhancement, edge sharpening, and other image processing functions. The APN 503, together with the semantic NN 507, forms a vascular segmentation machine learning model such as the model 206. The APN 503 feeding into the encoder 502 applies an atlas convolution to the image data and develops a rate that controls the effective field of view of each convolutional layer. The higher the rate, the larger the area of the image capture for convolution and low-level feature extraction that may be performed on each input image.For example, speeds of 6, 12, and 18 may be used to affect different fields of view and capture different functions, such as functions with different resolutions. Using an atlas or extended convolution, an extended sample of the image, or an image portion, is convolved into a smaller image. The encoder 502 uses several atlas convolution strides to determine low-level features and then applies a 1×1 (depthwise separable) convolution to combine the outputs of many different atlas convolutions. This creates a single matrix of features related to different classes that is input to the decoder 504, along with high-level features determined from the 1×1 convolution applied to the original input image. The convolutions of the encoder 502 may, in some examples, be performed using an atlas or depthwise convolution. The decoder 504 concatenates the low-level and high-level features into a stacked matrix and then applies a transposed convolution to this matrix using a fractional stride size to assign class labels to each pixel based on the spatial and feature information. These transposed convolutions generate a probability map that determines the likelihood that each pixel belongs to the background or blood vessel class. As shown, the softmax function is applied to generate the output segmented 2D image 505.

[0051] In FIG. 6, a process 600 that may be performed by the 3D segmented vascular tree model generator 204 to generate a 3D segmented vascular tree model from the generated 2D segmented vascular image 602 is shown. As shown, any one of four different pipelines may be used to generate the 3D segmented vascular tree model, from which, by way of example, a 1D segmented vascular tree model may also be obtained.

[0052] In the first configuration, in process 603, the 3D modeler 208 receives the 2D segmented vascular image and finds the centerline of each of the 2D segmented vascular images. In process 604, the 3D modeler 208 uses geometric tools that may include epipolar geometry, projective geometry, Euclidean geometry, or any related geometric tools to place points from each of the 2D segmented vascular images in the same position. In process 606, the 3D points having projections (inverse projections) mapped onto the points placed in the same location are triangulated. The local radius of the blood vessel is determined from each 2D segmented blood vessel, and these radius vectors are projected onto the 3D centerline. From there, in process 608, the 3D modeler 208 determines the contour of the blood vessel based on the triangulated 3D points, and in process 610, a 3D blood vessel tree model is generated. From there, a 1D blood vessel tree model is generated in process 620.

[0053] In the second configuration, the 3D modeler 208 generates a plurality of 3D rotation matrices from the 2D segmented vascular images in process 612. Then, the 3D modeler 208 generates a 3D segmented vascular tree model by solving a system of linear least squares equations that map the plurality of 2D segmented vascular images into 3D space in process 614.

[0054] In the third configuration, the 3D modeler 208 orthographically projects voxels in 3D space onto the plurality of 2D segmented vascular images in process 616, and identifies a set of 3D voxels projected inside the plurality of 2D segmented vascular images in process 618. The resulting binary volume is then smoothed to ensure realistic blood vessel boundaries.

[0055] In the fourth configuration, an active contour model is used to reconstruct the 3D geometry of blood vessels. The finishing centerline of each 2D segmented blood vessel image is executed in process 622. The endpoints of each blood vessel are identified in a plurality of 2D segmented blood vessel images and back-projected in process 624 to identify the endpoints in 3D space. A 3D cylinder is drawn between these 3D endpoints, and in process 626, the external and internal forces of this cylinder are defined as contour modeling so as to deform until the projection matches the blood vessel shape of each 2D image, depending on material properties, the imaging parameters of the system, and the reprojection error between the cylinder and the plurality of 2D segmented images. In process 628, a reprojection deformation is executed to force the 2D image into a 3D cylinder, where the deformation may be executed. The cylinder is deformed by the force to minimize the reprojection error. Process 628 may be repeated for all branches of the blood vessel until a complete coronary artery tree is reconstructed.

[0056] In some examples, the hydrodynamic machine learning model 212 of this specification can encode the physical laws underlying the management of a given dataset and is implemented as a neural network that can be described by partial differential equations, specifically, a neural network based on physical information. For example, the hydrodynamic machine learning model may include the partial differential equations of the 3D Navier-Stokes equations. This is a set of four partial differential equations for balancing mass and momentum, and the unknown fields are the 3D velocity vector (vx, vy, vz) and the scalar p.

[0057] In one example, the solution of blood flow within the 3D blood vessel tree model is generated by equations 1 and 2 using the incompressible 3D Navier-Stokes equations of blood flow.

Equation

Equation

[0058] Alternatively, in some examples, the hydrodynamic machine learning model 212 may include a low-dimensional model based on graph theory, as shown in FIG. 7. The low-dimensional model based on graph theory may be executed in a machine learning mode or an analysis mode. In the machine learning mode, using training data 701 regarding measurements such as blood flow, stenosis geometry, FFR, iFR, QFR, etc., a dense graph of the discrepancy 702 between the ground truth data and the low-fidelity model of blood flow given by the 1D non-linear theory model 703 is defined. The ground truth data can be provided either by in-silico simulation of the 3D Navier-Stokes equations 704 or by in-vivo data acquired in a catheterization laboratory 705. The dense graph of the discrepancy 702 may be generated by analyzing the replacement of the diameters of a number of stenoses, lengths, eccentricities, blood flow through the stenoses, or combinations thereof. Once the dense graph of the discrepancy is generated, a low-dimensional model 706 may be derived via non-local calculus, a deep neural network, or direct traversal of the vertices of the graph. The low-dimensional model may be an algebraic equation or an ordinary or partial differential equation. In the analysis mode, the input 707 to the low-dimensional model is the geometry given by the 1D segmented vascular tree extracted by the process 610 and the boundary conditions of blood flow and pressure extracted via the data assimilation process 308. The low-dimensional model 706 and the input 707 create the desired state of the vascular occlusion 708 (anatomical and functional by FFR calculation, iFR calculation, or QFR calculation) and the state of the microvascular disease. The 1D non-linear theory model 703 is a 1D non-linear Navier-Stokes model and includes two partial differential equations of mass balance and momentum balance, and the unknowns are blood flow Q and cross-sectional mean pressure P.

[0059] In one example, the solution of the blood flow obtained by the 1D non-linear theory model is generated using the conservation of mass and momentum of an incompressible Newtonian fluid by the following simultaneous equations, Equations 3 and 4. [Number] In the formula, x is the axial coordinate along the blood vessel, t is the time, A(x, t) is the cross-sectional area of the lumen, U(x, t) is the axial blood flow velocity averaged over the cross-section, P(x, t) is the blood pressure averaged over the cross-section, ρf is the density of blood assumed to be constant, and f(x, t) is the frictional force per unit length. It can be assumed that the momentum correction coefficient of the convective acceleration term in Equation 1 is equal to 1. Equations 3 and 4 can also be derived by integrating the incompressible Navier-Stokes equations over a general cross-section of the cylindrical region.

[0060] In any case, the hydrodynamic machine learning model of this specification may be formed by a data-driven algorithm for inferring solutions of these general nonlinear partial differential equations through a physics-informed surrogate classification model. Regarding the fundamental physical laws governing the time-dependent dynamics of the system, or empirically verified rules or other domain expertise, information about the physical laws may be used as a regularization agent that limits the space of admissible solutions to a manageable size. In return, encrypting such structured information into the machine learning model amplifies the information content of the data recognized by the algorithm, quickly steers it towards appropriate solutions, and enables generalization even when there are few training examples. Furthermore, the low-dimensional models proposed in this specification use the discrepancy between graph theory, and a low-fidelity 1D nonlinear model of blood flow, and either a 3D high-resolution Navier-Stokes model or ground truth data given by in-vivo anatomical and hemodynamic data to accurately and efficiently capture the hemodynamics around stenoses. In various examples, the low-dimensional model is defined from the graph of discrepancies via any of the following three methods: a) CNN, b) nonlocal calculus, c) graph search using a transversal algorithm (see, for example, Banerjee et al., A graph theoretic framework for representation, exploration and analysis on computed states of physical systems, Computer Methods in Applied Mechanics and Engineer, 2019, which is incorporated herein by reference).

[0061] In one example, the hydrodynamic machine learning model is configured to include Hidden Fluid Mechanics (HFM), a physics-informed deep learning framework that can encrypt classes of physical laws governing fluid motion, namely the Navier-Stokes equations, as described in A Navier-Stokes Informed Deep Learning Framework for Assimilating Flow Visualization Data, Raissi et al., Hidden Fluid Mechanics, dated August 13, 2018, incorporated herein by reference. In one example, the hydrodynamic machine learning model applies underlying conservation laws (i.e., mass, momentum, energy) to infer hidden quantities of interest such as velocity and pressure fields that are nothing more than 3D vascular tree models generated at different times from angiographic image data taken at different times. The hydrodynamic machine learning model may apply an algorithm that is independent of geometry or initial and boundary conditions. This makes the HFM configuration very flexible when selecting the type of vascular image data that can be used for training and diagnosis by the model. The hydrodynamic machine learning model is trained to predict pressure and velocity values of blood flow in both two-dimensional and three-dimensional imaged blood vessels. Such information can be used to determine other physics-related characteristics such as pressure or wall shear stress within an artery.

[0062] In some examples, the computational fluid dynamics model is configured to determine lumped parameter models attached to each blood vessel of a 3D blood vessel tree model or a 1D blood vessel tree model. The computational fluid dynamics model may include a series of lumped parameter models (LPMs) for different blood vessels, as shown in FIG. 10A. An exemplary LPM 1000 is provided for a heart model coupled to an inflow surface of a 3D blood vessel tree 1002. The blood vessel tree 1002 is formed from a plurality of different blood vessels labeled with different letters, an A inlet, B-H aortic outlets, and a-k coronary artery outlets. LPM 1006 is used for each outlet B-H representing the microcirculation of blood vessels other than the coronary arteries. LPM 1008 is provided for coronary artery outlets a-k and is coupled to LPM 1000 representing the heart. The parameters of this model are estimated from patient data regarding blood flow and pressure (measured in a catheterization laboratory, or estimated using the data assimilation technique described in 308, or using morphological considerations such as Murray's law). Analysis may be performed in a closed-loop configuration using LPM 1004 that includes both the left and right sides of the heart. Using a model of this nature, hemodynamics under pulsatile conditions may be calculated. A second example of an LPM is shown in FIG. 10B. Here, the inflow boundary condition may be defined by either measured from dynamic angiography data or the mean arterial pressure of the patient by the estimated mean blood flow. The patient-specific outflow boundary condition may be defined by the blood flow through each blood vessel estimated via a hydrodynamic machine learning process 308 that assimilates blood flow data over a sampling period, or by an LPM (resistance) coupled to each of the outlet surfaces of the coronary artery tree. Given the calculated solution of the pressure and flow rate within the blood vessel tree, the LPM of each tree (3D or 1D) of the blood vessel tree may also be estimated assuming that the pressure gradient across the entire LPM drops to a certain level of capillary pressure. Using a model of this nature, hemodynamics under steady-state conditions may be simulated. This process can be repeated for each available hemodynamic state (e.g., baseline and congested). LPMs are used to represent cardiovascular regions where a complete detailed blood flow solution is not required, but it is important for the model to include the relationships of pressure, blood flow, and possibly volume in these regions.They are ideal in areas where detailed spatial information regarding the geometry of blood vessels is not available and which are not fundamentally important. Since their parameters have a fundamental influence on the pressure and velocity fields of the vascular tree model, the parameters need to be adjusted to achieve physiological values that match the data assimilated from the patient's images and additional clinical records available to the individual in question. If insufficient data are available for the parameterization of a particular patient, data from the scientific literature regarding expected values can be used to help determine appropriate parameters.

[0063] In one example, the anatomical and functional evaluation of CAD follows workflow 1100 shown in FIG. 11. A number of angiographic images 1101 taken in different orientations are supplied to a machine learning segmentation module 1102 (corresponding to processes 302, 304, and 402 and 404), and the machine learning segmentation module 1102 automatically creates a 2D segmented image of the angiographic images 1103. These images are then supplied to an algorithm 1104 that generates 3D and 1D segmented vascular compartments 1105 by a combination of the processes given in FIG. 6. This workflow is used to automatically characterize the diameter of the lumen and thus the anatomical severity of CAD. Further, a series of angiographic images 1106 that define the transport of dye down the region of interest in the vascular tree are supplied to a series of hydrodynamic machine learning algorithms 1107 that assimilate information regarding the blood flow velocity and / or pressure of each vessel of interest in the vessel. This system generates velocity and pressure boundary conditions 1108, which, together with the vascular tree 1105, are supplied as input to a low-dimensional model 1109 derived from graph theory and ultimately create the desired functional metrics of CAD 1110, including FFR, iFR, QFR, and microvascular resistance.

[0064] Additional aspects Aspect 1. A computer-implemented method for evaluating coronary artery disease, comprising: (a)Receiving, by one or more processors, angiography image data of a target vascular examination region, wherein the angiography image data includes angiography images captured over a sampling period; (b)Applying, by one or more processors, the angiography image data to a vascular segmentation machine learning model and using the vascular segmentation machine learning model to generate a two-dimensional (2D) segmented vascular image of the vascular examination region; (c)Generating, by one or more processors, a three-dimensional (3D) segmented vascular tree geometry model of blood vessels within the vascular examination region from the 2D segmented vascular image; (d)Applying, by one or more processors, the 3D segmented vascular tree geometry model to a fluid dynamics machine learning model and using the fluid dynamics machine learning model to assimilate blood flow data over a sampling period of one or more blood vessels within the vascular examination region; (e)Applying, by one or more processors, the 3D segmented vascular tree geometry model and the assimilated blood flow data to a 3D high-fidelity computational fluid dynamics model; (f)Determining, by one or more processors, the state of vascular occlusion of one or more blood vessels within the vascular examination region. A computer-implemented method comprising the above steps.

[0065] Aspect 2. The computer-implemented method according to Aspect 1, further comprising determining, by one or more processors, the state of microvascular disease of one or more blood vessels within the vascular examination region by performing (a)-(e) in at least two different hemodynamic states.

[0066] Aspect 3. The computer-implemented method according to Aspect 1, wherein the vascular segmentation machine learning model is a convolutional neural network.

[0067] Aspect 4. The computer-implemented method according to aspect 1, further comprising applying, by one or more processors, at least one of a noise removal process, a linear filtering process, an image size normalization process, and a pixel intensity normalization process to the received angiography image data to create filtered angiography image data.

[0068] Aspect 5. The computer-implemented method according to aspect 4, further comprising supplying the filtered angiography image data to an angiography processing network (APN) trained to remove low-contrast images, catheters, and / or overlapping bony structures because it addresses the main issues of the angiography image data.

[0069] Aspect 6. The computer-implemented method according to aspect 5, further comprising supplying the output of the APN to semantic image segmentation to create an automatic binary 2D segmented vascular image.

[0070] Aspect 7. Generating a 3D segmented vascular tree geometry model comprises finding the centerline of each of the 2D segmented vascular images, using epipolar geometry to place points from each of the 2D segmented vascular images in the same position, triangulating 3D points having projections mapped onto the points placed in the same position, and determining the contour of the blood vessel based on the triangulated 3D points, the computer-implemented method according to aspect 1.

[0071] Aspect 8. Generating a 3D segmented vascular tree geometry model comprises generating a plurality of 3D rotation matrices from a plurality of 2D segmented vascular images, Generating a 3D segmented vascular tree geometry model by solving a system of linear least squares equations that map a plurality of 2D segmented vascular images into 3D space, the computer-implemented method according to aspect 1.

[0072] Aspect 9. Generating a 3D segmented vascular tree geometry model includes Orthogonally projecting voxels in 3D space onto a plurality of 2D segmented vascular images, and Identifying a set of 3D voxels projected inside a plurality of 2D segmented vascular images, the computer-implemented method according to aspect 1.

[0073] Aspect 10. Generating a 3D segmented vascular tree geometry model includes Using an active contour to deform a cylindrical geometry shape through internal and external forces into a plurality of segmented 2D images, the computer-implemented method according to aspect 1.

[0074] Aspect 11. Further including applying at least one of a smoothing algorithm and a surface spline approximation algorithm to the 3D segmented vascular tree geometry model by one or more processors, the computer-implemented method according to aspect 1.

[0075] Aspect 12. The computer-implemented method according to aspect 1, generating a 3D segmented vascular tree geometry model by performing inverse projection of 2D segmented vascular images.

[0076] Aspect 13. The computer-implemented method according to aspect 1, wherein the hydrodynamic machine learning model includes at least one network of the type of convolutional neural network (CNN), autoencoder, or long short-term memory (LSTM).

[0077] Aspect 14. The computer-implemented method according to Aspect 1, wherein the hydrodynamic machine learning model is a deep learning framework based on Navier-Stokes information configured to determine pressure data and velocity data across a 3D vascular space.

[0078] Aspect 15. The computer-implemented method according to Aspect 14, wherein the deep learning framework based on Navier-Stokes information includes one or more methods of the type of Kalman filtering, physics-informed neural networks, iterative assimilation algorithms based on contrast arrival times at anatomical landmarks, and TIMI frame counting.

[0079] Aspect 16. The computer-implemented method according to Aspect 1, wherein determining blood flow data over a sampling period includes determining pressure and flow velocity data of one or more blood vessels over the sampling period.

[0080] Aspect 17. The computer-implemented method according to Aspect 1, wherein determining blood flow data over a sampling period includes determining pressure and flow velocity data of a plurality of connected blood vessels in a blood vessel examination region.

[0081] Aspect 18. The computational fluid dynamics model includes one or more of a multi-scale 3D Navier-Stokes simulation using a low-dimensional (lumped parameter) model of a segmented vascular tree geometry model, a low-dimensional Navier-Stokes (1D) simulation using a low-dimensional model, or a low-dimensional model simulation (lumped parameter model, 0D) model, of the computer-implemented method according to Aspect 1.

[0082] Aspect 19. The computer-implemented method according to Aspect 1, wherein the lumped parameter boundary condition parameters are determined by a hydrodynamic machine learning model of one or more blood vessels in a blood vessel examination region.

[0083] Aspect 20. The computer-implemented method according to aspect 19, further comprising determining a concentration parameter model of the blood flow in the first blood vessel and determining a concentration parameter model of the blood flow in each blood vessel branching from the first blood vessel.

[0084] Aspect 21. Determining a fractional flow reserve (FFR), an instantaneous flow reserve ratio (iFR), or a quantitative flow ratio (QFR) of one or more blood vessels from blood flow data, and Determining a state of vascular occlusion from the FFR, iFR, or QFR of one or more blood vessels, the computer-implemented method according to any one of aspects 1.

[0085] Aspect 22. Determining a coronary flow reserve (CFR) of one or more blood vessels from blood flow data from one or more physiological states, and Determining a state of microvascular disease from the CFR of one or more blood vessels, the computer-implemented method according to aspect 1.

[0086] Aspect 23. The computer-implemented method according to aspect 22, wherein the one or more physiological states include a baseline physiological state and a congestive physiological state.

[0087] Aspect 24. The computer-implemented method according to aspect 1, wherein determining a state of vascular occlusion includes determining the presence of stenosis in one or more blood vessels.

[0088] Aspect 25. The computer-implemented method according to aspect 1, wherein determining a state of microvascular disease includes determining a concentration parameter model on the boundary of a blood vessel in a vascular examination region in a plurality of hemodynamic states.

[0089] Aspect 26. The computer-implemented method according to aspect 1, further comprising supplying a plurality of synthetic angiography images, a plurality of clinical angiography images, and a plurality of enhanced angiography images for training a vascular segmentation machine learning model.

[0090] Aspect 27. The computer-implemented method according to Aspect 1, wherein the angiographic image data includes angiographic images captured over a sampling period, and includes angiographic images captured during a baseline state and angiographic images captured during a pharmacologically induced hyperemic state.

[0091] Aspect 28. A computer-implemented method for evaluating coronary artery disease, comprising: (a) receiving, by one or more processors, a plurality of angiographic images of a vascular examination region of a subject, wherein the angiographic images are captured over a sampling period and the vascular examination region includes one or more blood vessels; (b) applying, by one or more processors, the angiographic images to a vascular segmentation machine learning model and using the vascular segmentation machine learning model to generate a two-dimensional (2D) segmented vascular image of the one or more blood vessels; (c) generating, by one or more processors, a one-dimensional (1D) segmented vascular tree geometry model of the one or more blood vessels from the 2D segmented vascular image; (d) applying, by one or more processors, the 1D segmented vascular tree geometry model to a fluid dynamics machine learning model and using the fluid dynamics machine learning model to assimilate blood flow data over the sampling period of the one or more blood vessels; (e) applying, by one or more processors, the 1D segmented vascular tree geometry model and the assimilated blood flow data to a low-dimensional model based on graph theory based on a computational fluid dynamics model; (f) determining, by one or more processors, the state of vascular occlusion.

[0092] Aspect 29. The computer-implemented method according to Aspect 28, further comprising determining, by one or more processors, the state of microvascular disease of the one or more blood vessels within the vascular examination region by performing (a) to (e) in at least two different hemodynamic states.

[0093] Aspect 30. The computer-implemented method according to aspect 28, wherein the angiographic images captured over the sampling period include angiographic images captured during the baseline state and angiographic images captured during the congested state.

[0094] Aspect 31. The computer-implemented method according to aspect 30, wherein generating a 1D segmented vascular tree geometry model of the blood vessels within the vascular examination region from the 2D segmented vascular images includes creating a skeletonization of the 3D segmented vascular tree geometry model given by the 3D space of the blood vessels included in the 3D segmented vascular tree geometry model and the path / centerline in a series of 2D cross-sectional contours separated at any distance along each path / centerline of the tree.

[0095] Aspect 32. The computer-implemented method according to aspect 28, wherein the vascular segmentation machine learning model is a convolutional neural network.

[0096] Aspect 33. The computer-implemented method according to aspect 28, further comprising applying, by one or more processors, a noise removal process, a linear filtering process, an image size normalization process, and a pixel intensity normalization process to the received angiographic images.

[0097] Aspect 34. The computer-implemented method according to aspect 28, wherein the fluid dynamics machine learning model includes at least one network type of a convolutional neural network (CNN), an autoencoder, or a long short-term memory (LSTM), or a low-dimensional model of blood flow and pressure based on graph theory.

[0098] Aspect 35. The computer-implemented method according to aspect 28, wherein the fluid dynamics machine learning model is a deep learning framework based on Navier-Stokes information configured to determine pressure data and velocity data over a 1D vascular space.

[0099] Aspect 36. The computer-implemented method according to aspect 35, wherein the deep learning framework based on Navier-Stokes information includes one or more methods of Kalman filtering, neural networks based on physical information, iterative assimilation algorithms based on contrast arrival times at anatomical landmarks, and types of TIMI frame counting.

[0100] Aspect 37. The computer-implemented method according to aspect 28, wherein the fluid mechanics learning model is a low-dimensional model based on graph theory obtained by defining a dense graph of the discrepancy between ground truth data and a low-fidelity model of blood flow given by a 1D non-linear theory model.

[0101] Aspect 38. The computer-implemented model according to aspect 37, wherein the ground truth data used to define the dense graph can be either in vivo data acquired in a catheterization laboratory or an in silico simulation of the 3D Navier-Stokes equations.

[0102] Aspect 39. The computer-implemented model according to aspect 37, wherein the vertices of the dense graph are defined by investigating the discrepancy between the ground truth data and the low-fidelity 1D non-linear model of blood flow, and include the replacement of the diameter of the stenosis, the length, the eccentricity, the blood flow through the stenosis, and combinations thereof.

[0103] Aspect 40. The computer-implemented model according to aspect 37, wherein the low-dimensional model based on graph theory is obtained by non-local calculus, deep neural networks, or direct traversal of the vertices of the generated graph.

[0104] Aspect 41. The computer-implemented model according to aspect 37, wherein the low-dimensional model based on graph theory may be an algebraic equation or an ordinary or partial differential equation.

[0105] Aspect 41. The computer-implemented method according to aspect 28, wherein determining blood flow data over a sampling period includes determining pressure and flow velocity data of one or more blood vessels over the sampling period.

[0106] Aspect 43. The computer-implemented method according to aspect 42, wherein the one or more blood vessels include a plurality of connected blood vessels.

[0107] Aspect 44. The computer-implemented method according to aspect 28, wherein the computational fluid dynamics model includes low-dimensional model simulations based on graph theory, the input is provided by a 1D segmented vascular tree geometry model, and the boundary conditions of blood flow and pressure are assimilated by a computational fluid dynamics machine learning model.

[0108] Aspect 45. The computer-implemented method according to aspect 28, further comprising determining a lumped parameter model of the blood flow of each of the one or more blood vessels.

[0109] Aspect 46. Determining a fractional flow reserve (FFR), an instantaneous flow reserve ratio (iFR), or a quantitative flow ratio (QFR) of one or more blood vessels from the blood flow data; and Determining the state of vascular occlusion from the FFR, iFR, or QFR of the one or more blood vessels, the computer-implemented method according to any of aspects 1.

[0110] Aspect 47. Determining a coronary flow reserve (CFR) of one or more blood vessels from blood flow data from one or more physiological states; and Determining the state of microvascular disease from the CFR of the one or more blood vessels, the computer-implemented method according to aspect 1.

[0111] Aspect 48. The computer-implemented method according to aspect 1, wherein determining the state of vascular occlusion includes determining the presence of stenosis in 28 or more blood vessels.

[0112] Aspect 49. The computer-implemented method according to aspect 28, further comprising supplying a plurality of synthetic angiography images, a plurality of clinical angiography images, and a plurality of enhanced angiography images to train a vascular segmentation machine learning model.

[0113] Aspect 50. One or more processors, which when executed, cause the one or more processors to: receive angiography image data of a vascular examination region of a subject by the one or more processors, the angiography image data including an angiography image captured over a sampling period; apply the angiography image data to a vascular segmentation machine learning model by the one or more processors and generate a two-dimensional (2D) segmented vascular image of the vascular examination region using the vascular segmentation machine learning model; generate a three-dimensional (3D) segmented vascular tree geometry model of the blood vessels within the vascular examination region from the 2D segmented vascular image by the one or more processors; apply the 3D segmented vascular tree geometry model to a fluid dynamics machine learning model by the one or more processors and assimilate blood flow data over the sampling period of one or more blood vessels within the vascular examination region using the fluid dynamics machine learning model; apply the 3D segmented vascular tree geometry model and the assimilated blood flow data to a 3D high-fidelity computational fluid dynamics model by the one or more processors; and determine the state of vascular occlusion of one or more blood vessels within the vascular examination region by the one or more processors. One or more computer-readable memories storing instructions that cause the above actions. A computing device configured to evaluate CAD.

[0114] Aspect 50. One or more processors, which when executed, cause the one or more processors to receive, by the one or more processors, plural angiography image data of a target vascular examination region, where the angiography images are captured over a sampling period and the vascular examination region includes one or more blood vessels; cause the one or more processors to apply the angiography images to a vascular segmentation machine learning model and use the vascular segmentation machine learning model to generate a two-dimensional (2D) segmented vascular image of the one or more blood vessels; cause the one or more processors to generate a one-dimensional (1D) segmented vascular tree geometry model of the one or more blood vessels from the 2D segmented vascular image; cause the one or more processors to apply the 1D segmented vascular tree model to a fluid dynamics machine learning model and use the fluid dynamics machine learning model to assimilate blood flow data over the sampling period of the one or more blood vessels; cause the one or more processors to apply the 1D segmented vascular tree model and the assimilated blood flow data to a low-dimensional model based on graph theory based on a computational fluid dynamics model; and cause the one or more processors to determine a state of vascular occlusion; and one or more computer-readable memories storing instructions to cause the above, a computing device configured to evaluate CAD.

[0115] Throughout this specification, multiple instances may implement components, operations, or structures described as a single instance. Individual operations of one or more methods are illustrated and described as separate operations, but one or more of the individual operations may be performed simultaneously and need not be performed in the order illustrated. Structures and functions presented as separate components within an exemplary configuration may be implemented as a combined structure or component. Similarly, structures and functions presented as a single component may be implemented as separate components. These and other variations, modifications, additions, and improvements are within the scope of the subject matter of this specification.

[0116] Furthermore, certain embodiments are described herein as including logic or a number of routines, subroutines, applications, or instructions. These can constitute either software (e.g., code embodied on a non-transitory machine-readable medium) or hardware. In hardware, routines and the like are tangible units capable of performing particular operations and can be configured or arranged in a particular manner. In an exemplary embodiment, one or more computer systems (e.g., a stand-alone, client, or server computer system), or one or more hardware modules of a computer system (e.g., a processor or group of processors), can be configured as hardware modules that operate to perform the particular operations described herein by software (e.g., an application or a portion of an application such as the contrast agent injection system shown in FIG. 2).

[0117] In various embodiments, the hardware modules can be implemented mechanically or electronically. For example, a hardware module can include dedicated circuitry or logic that is permanently configured (e.g., special-purpose processors such as a field programmable gate array (FPGA) or an application specific integrated circuit (ASIC), etc.) to perform particular operations. A hardware module can also include programmable logic or circuitry that is temporarily configured by software (e.g., realized in a general-purpose processor or other programmable processor) to perform particular operations. It will be appreciated that whether to implement a hardware module mechanically, with dedicated and permanently configured circuitry, or with temporarily configured circuitry (e.g., configured by software) can be determined considering cost and time.

[0118] Accordingly, the term "hardware module" should be understood to encompass a tangible entity that is physically constructed or permanently configured (e.g., embedded in hardware) or temporarily configured (e.g., programmed) to operate in a particular manner or to perform a particular operation described herein. Considering embodiments in which a hardware module is temporarily configured (e.g., programmed), each of the hardware modules need not be configured or instantiated at any given point in time. For example, if a hardware module includes a general-purpose processor configured using software, the general-purpose processor can be configured as different hardware modules at different points in time. Thus, software may configure the processor, for example, to constitute a particular hardware module at one point in time and a different hardware module at another point in time.

[0119] A hardware module can provide information to other hardware modules and receive information from other hardware modules. Thus, the hardware modules described herein can be understood as being communicatively coupled. When such hardware modules are present simultaneously, communication can be achieved via signal transmission that connects the hardware modules (e.g., through appropriate circuitry and buses). In embodiments where multiple hardware modules are configured or instantiated at different times, communication between such hardware modules can be achieved, for example, through storage and retrieval of information in a memory structure accessible to the multiple hardware modules. For example, a hardware module can execute an operation and store the output of that operation in a memory device to which the hardware module is communicatively coupled. Subsequently, a further hardware module can later access the memory device to retrieve and process the stored output. A hardware module can also initiate communication with an input or output device and operate on a resource (e.g., collection of information).

[0120] The various operations of the exemplary methods described herein can be performed, at least in part, by one or more processors temporarily configured (e.g., by software) to perform the relevant operations or permanently configured to perform the relevant operations. Whether temporarily configured or permanently configured, such processors can configure processor-implemented modules to operate to perform one or more operations or functions. The modules referred to herein can, in some exemplary embodiments, include processor-implemented modules.

[0121] Similarly, the methods or routines described herein can be at least partially processor-implemented. For example, at least some of the operations of a method can be performed by one or more processors or processor-implemented hardware modules. Certain performance of the operations can exist not only within a single machine, but also can be distributed among one or more processors deployed across several machines. In examples of some embodiments, one or more processors can be present in a single location (e.g., within a home environment, within a workplace environment, or as a server farm), but in other embodiments, the processors can be distributed across a number of locations.

[0122] Reliable performance of the operations can exist not only within a single machine, but also can be distributed among one or more processors deployed across several machines. In some exemplary embodiments, one or more processors or processor-implemented modules can be present in a single location (e.g., within a home environment, within a workplace environment, or within a server farm). In other exemplary embodiments, one or more processors or processor-implemented modules can be distributed across a number of locations.

[0123] Unless otherwise indicated, the descriptions herein using words such as "processing," "computing," "calculating," "determining," "presenting," "displaying," etc., can mean the operations or processes of a machine (e.g., a computer) that manipulates or transforms data represented as physical (e.g., electronic, magnetic, or optical) quantities within one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other mechanical components that receive, store, transmit, or display information.

[0124] As used herein, any reference to "one embodiment" or "an embodiment" means that a particular element, feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment. The appearances of the phrase "in one embodiment" in various places in this specification are not necessarily all referring to the same embodiment.

[0125] Some embodiments can be described using the expressions "coupled" and "connected" along with their derivatives. For example, some embodiments can be described using the term "coupled" to indicate that two or more elements are in direct physical or electrical contact. However, the term "coupled" can also mean that two or more elements are not in direct contact with each other but still cooperate or interact with each other. The embodiments are not limited to this context.

[0126] Those skilled in the art will recognize that various modifications, changes, and combinations can be made to the above-described embodiments without departing from the scope of the invention, and such modifications, changes, and combinations should be considered to be within the scope of the concepts of the invention.

[0127] Accordingly, although the invention has been described with respect to specific examples, these examples are illustrative only and are not intended to be limiting of the invention. It will be apparent to those skilled in the art that changes, additions, or deletions can be made to the disclosed embodiments without departing from the spirit and scope of the invention.

[0128] The above description is for clarity of understanding only, and since modifications within the scope of the invention may be apparent to those skilled in the art, no unnecessary limitations should be understood therefrom.

Claims

**Claim 1** A computer-implemented method for evaluating coronary artery disease, comprising: (a) receiving, by one or more processors, a plurality of angiographic images of a target vascular examination region, wherein the angiographic images are captured over a sampling period, and the vascular examination region includes one or more blood vessels; (b) applying, by the one or more processors, the angiographic images to a vascular segmentation machine learning model and using the vascular segmentation machine learning model to generate two-dimensional (2D) segmented vascular images of the one or more blood vessels; (c) generating, by the one or more processors, a one-dimensional (1D) segmented vascular tree geometry model or a three-dimensional (3D) segmented vascular tree geometry model of the one or more blood vessels from the 2D segmented vascular images; (d) applying, by the one or more processors, the 1D segmented vascular tree geometry model or the 3D segmented vascular tree geometry model to a fluid dynamics machine learning model and using the fluid dynamics machine learning model to assimilate blood flow data over the sampling period of the one or more blood vessels; and (e) applying, by the one or more processors, the 1D segmented vascular tree geometry model or the 3D segmented vascular tree geometry model and the assimilated blood flow data to a low-dimensional model based on graph theory based on a computational fluid dynamics model to determine a state of vascular occlusion. A computer-implemented method as claimed in claim 1. **Claim 2** The computer-implemented method according to claim 1, further comprising determining, by the one or more processors, a state of microvascular disease of the one or more blood vessels in the vascular examination region by performing (a) to (e) in at least two different hemodynamic states. **Claim 3** The computer-implemented method according to claim 1, wherein the angiographic images captured over the sampling period include angiographic images captured during a baseline state and angiographic images captured during a hyperemic state. **Claim 4** Generating the 1D segmented vascular tree geometric model of blood vessels within the vascular examination region from the 2D segmented vascular image includes creating a skeletonization of the 3D segmented vascular tree geometric model given by the 3D space of blood vessels included in the 3D segmented vascular tree geometric model and the path / centerline in a series of 2D cross-sectional contours separated at an arbitrary distance along each path / centerline of the 3D segmented vascular tree geometric model. The computer-implemented method according to claim 3.

5. The computer-implemented method according to claim 1, wherein the vascular segmentation machine learning model is a convolutional neural network.

6. The computer-implemented method according to claim 1, further comprising applying, by the one or more processors, a noise removal process, a linear filtering process, an image size normalization process, and a pixel intensity normalization process to the received angiographic image.

7. Does the hydrodynamic machine learning model include at least one type of network of the type of convolutional neural network (CNN), autoencoder, or long short-term memory (LSTM), or a low-dimensional model of blood flow and pressure based on graph theory? Is the hydrodynamic machine learning model a deep learning framework based on Navier-Stokes information configured to determine pressure data and velocity data across a 1D vascular space? Does the deep learning framework based on Navier-Stokes information include one or more methods of the type of Kalman filtering, neural network based on physical information, iterative assimilation algorithm based on contrast arrival time to anatomical landmarks, and TIMI frame counting; or The computer-implemented method according to claim 1, wherein the hydrodynamic machine learning model is a low-dimensional model based on graph theory obtained by defining a dense graph of the mismatch between a 1D non-linear theory model and ground truth data.

8. The computer-implemented method according to claim 7, wherein the ground truth data used to define the dense graph is either in-vivo data acquired in a catheterization laboratory or obtained by in-silico simulation of the 3D Navier-Stokes equations.

9. The vertices of the secret graph are defined by investigating the discrepancy between the ground truth data and the 1D non-linear model of blood flow, including replacement of the stenosis diameter, length, eccentricity, blood flow through the stenosis, and combinations thereof, the computer-implemented method of claim 7.

10. The low-dimensional model based on graph theory is a deep neural network, or The low-dimensional model based on graph theory includes ordinary or partial differential equations, the computer-implemented method of claim 7.

11. Determining blood flow data over the sampling period includes determining pressure and flow velocity data of the one or more blood vessels over the sampling period, the computer-implemented method of claim 1.

12. The computational fluid dynamics model includes a low-dimensional model simulation based on graph theory, with the input including the 1D segmented vascular tree geometry model blood flow and pressure boundary conditions, the computer-implemented method of claim 1.

13. Determining from the blood flow data a fractional flow reserve (FFR), instantaneous flow reserve ratio (iFR), or quantitative flow ratio (QFR) of the one or more blood vessels; and Determining the state of the vascular occlusion from the FFR, iFR, or QFR of the one or more blood vessels; or further comprising; or Determining a coronary flow reserve (CFR) of the one or more blood vessels from blood flow data from one or more physiological states; and Determining the state of microvascular disease from the CFR of the one or more blood vessels; The computer-implemented method of claim 1, further comprising.

14. Determining the state of the vascular occlusion includes determining the presence of stenosis in the one or more blood vessels, the computer-implemented method of claim 1.

15. The computer-implemented method of claim 1, further comprising supplying a plurality of synthetic angiographic images, a plurality of clinical angiographic images, and a plurality of enhanced angiographic images to train the vascular segmentation machine learning model.

Images