Method and system for evaluating functionally significant vascular occlusion based on machine learning

Abstract

Methods and systems are provided for assessing occlusion of a blood vessel of interest from a volumetric image dataset. The volumetric image dataset is analyzed to extract data representing an axial trajectory of the blood vessel of interest. Generating a multi-plane reformatted (MPR) image and data representing an axial trajectory of the blood vessel of interest; the MPR image is supplied as input to a first machine learning network that outputs feature data characterizing a plurality of features of the blood vessel of interest along an axial trajectory of the blood vessel of interest. Additional data characterizing at least one additional feature of the blood vessel of interest along an axial trajectory of the blood vessel of interest is generated. And inputting the data output by the first machine learning network and the additional data into a second machine learning network, wherein the second machine learning network outputs data representing the severity of the anatomical lesion of the blood vessel of interest.

Description

Technical Field

[0001] The present application relates to the technical field of medical imaging, in particular computed tomography angiographic imaging, but may find application in any field where quantification of flow in an obstructed or partially obstructed conduit is required, such as in non-destructive testing applications. Background Art

[0002] Coronary artery disease (CAD) is one of the leading causes of death worldwide. CAD generally refers to conditions involving narrowed or blocked blood vessels, which can result in reduced or absent blood supply to the portion distal to the narrowing, which can lead to reduced oxygen supply to the heart muscle, resulting in, for example, ischemia and chest pain (angina). Narrowing of blood vessels is called stenosis and is caused by atherosclerosis, which is the buildup of fat, cholesterol, and other substances (plaque) in and on the blood vessel walls (see Figure 1 ). Atherosclerotic plaques can be classified according to their components, such as calcified plaques, soft plaques and mixed plaques (i.e., plaques containing calcified and non-calcified components). Such non-calcified components include extracellular matrix, smooth muscle cells, macrophages, foam cells, lipids and fibrous tissue. Calcified plaques are considered to be stable, and their amount in the coronary arteries is a powerful predictor of cardiovascular events. Unlike calcified plaques, non-calcified plaques and mixed plaques are considered to be unstable and more prone to rupture. In the event of a rupture in a coronary artery, a plaque rupture may lead to an acute major event (such as a stroke or a heart attack). A heart attack may lead to myocardial infarction, resulting in irreversible damage to the myocardium. Because different types of plaques and different grades of stenosis lead to different patient management strategies, it is very important to detect and characterize coronary artery plaques and stenosis grades.

[0003] Besides the grade of stenosis (anatomical stenoses), another important aspect in the prevention and treatment of CAD is the functional assessment of such narrowed anatomical stenoses or occluded vessels.

[0004] Currently, X-ray angiography is an imaging modality used during the treatment of stenotic (narrowed) coronary arteries in a catheterization laboratory with the aid of a minimally invasive procedure, also known as percutaneous coronary intervention (PCI). During PCI, a (interventional) cardiologist feeds a deflated balloon or other device on a catheter from the femoral or radial artery in the groin up through the blood vessels until they reach the site of the blockage in the artery. X-ray imaging is used to guide the catheter through. PCI typically involves inflating a balloon to open the artery, thereby restoring unimpeded blood flow. A stent or scaffold may be placed at the site of the blockage to keep the artery open. For example, for moderate coronary artery anatomical lesions (defined as 30%-70% lumen narrowing), it is not always obvious whether the stenosis is a risk to the patient and whether action is desired. An overestimation of the severity of the stenosis may result in treatment that is not necessary in hindsight, and therefore exposes the patient to unnecessary risks. However, an underestimation of the severity of the stenosis may induce risk because the patient is not treated when the stenosis is actually severe and actually obstructs flow to the myocardium. Especially for these situations, it is desirable to have additional functional assessment to help make good decisions.

[0005] In the past 10 to 15 years, fractional flow reserve (FFR) has been increasingly used as a method for identifying and effectively targeting coronary artery lesions that are most likely to benefit from PCI. FFR characterizes the pressure difference across coronary artery stenosis to determine the possibility that stenosis hinders oxygen delivery to the heart muscle. The characterization of FFR usually involves percutaneous insertion of a pressure conversion line inside the coronary artery and measuring the pressure behind the lesion (distal) and in front of the lesion (proximal), and is performed in the catheterization laboratory. This is best performed under hyperemia, because in the case of maximum hyperemia, the blood flow to the myocardium is proportional to the myocardial perfusion pressure. Therefore, FFR provides a quantitative assessment of the functional severity of coronary artery lesions, as described in "Measurement of Fractional Flow Reserve to Assess the Functional Severity of Coronary-Artery Stenoses" (N Engl J Med 1996, 334: 1703-1708) by Pijls et al. Although the European Society of Cardiology (ESC) and American College of Cardiology / American Heart Association (ACC / AHA) guidelines recommend the use of FFR in patients with moderate coronary artery stenosis (30%-70%), visual assessment of X-ray coronary angiography alone is still used in more than 90% of procedures to select patients for percutaneous coronary intervention (regardless of whether QCA is supported) (Kleiman et al., "Bringing it all together: integration of physiology with anatomy during cardiac catheterization", J Am Coll Cardiol. 2011;58:1219–1221). However, FFR has some disadvantages. For example, characterizing FFR may be associated with the additional cost of a pressure line that can only be used once. In addition, characterizing FFR may require invasive catheterization with associated costs and procedure time. In addition, in order to induce (maximal) hyperemia, additional drug infusions (adenosine or papaverine) may be required, which is an additional burden for the patient.

[0006] Coronary computed tomography (CT) angiography (CCTA) is a noninvasive imaging modality used for anatomical assessment of the coronary arteries, but does not assess the functional significance of coronary artery lesions. The main advantage of CCTA is its excellent ability to exclude CAD due to its significantly high negative predictive value and its noninvasive nature. Although CCTA can reliably exclude the presence of severe coronary artery disease, many high-grade stenoses seen on CCTA are not flow-restricting. The possibility of this false-positive result has raised concerns that widespread use of CCTA may lead to clinically unnecessary coronary revascularization procedures. This lack of specificity of CCTA is one of the major limitations of CCTA in determining the hemodynamically important aspects of CAD (Meijboom et al., "Comprehensive assessment of coronary artery stenoses: computed tomography coronary angiography versus conventional coronary angiography and correlation with fractional low reserve in patients with stable angina", Journal of the American College of Cardiology 52(8) (2008) 636-643). As a result, CCTA may lead to unnecessary interventions for patients, which may pose additional risks to the patient and may result in unnecessary health care costs.

[0007] In "Computational Fluid Dynamics Applied to CardiacComputed Tomography for Noninvasive Quantification of Fractional Flow Reserve" by Taylor et al. (Journal of the American College of Cardiology, Vol. 61, No. 22, 2013) and US8315812, a noninvasive method (FFRCT) for quantifying FFR based on CCTA is described. The technique uses computational fluid dynamics (CFD) applied to CCTA after semi-automatic segmentation of the coronary artery tree, which includes a portion of the ascending aorta covering the area where both the left and right coronary arteries diverge. The three-dimensional (3D) blood flow and pressure of the coronary arteries are simulated, where the blood is modeled as an incompressible Newtonian fluid using the Navier-Stokes equations and solved using the finite element method on a parallel supercomputer under appropriate initial and boundary conditions. FFRCT models the condition of adenosine-induced hyperemia without adenosine infusion. This process is computationally complex and time-consuming and may take several hours and relies heavily on 3D anatomical coronary artery models due to the segmentation, which is particularly affected by the same limitations as those mentioned above.

[0008] In addition to the development of CFD-based FFR prediction methods, methods have emerged to correlate quantitative indices derived from CCTA with measured FFR values. These clinical indices characterize coronary arteries by, for example, intraluminal attenuation gradients (Wong et al., 2013; Ko et al., 2016) or plaque volumes (Diaz-Zamudio et al., 2015; Otaki et al., 2021), or describe specific lesions by quantifying the degree of stenosis (Gould et al., 1975; Otaki et al., 2021) or contrast density differences (Dey et al., 2014; Hell et al., 2015). While the mathematical simplicity and intuitive design of the calculated indices enable their interpretation, they limit their ability to model the complex relationship between FFR and coronary artery properties on CCTA. Therefore, in order to improve FFR prediction with clinical indices, machine learning classifiers combining multiple indices have been employed (Ko et al., 2015; Itu et al., 2016; Dey et al., 2018; Otaki et al., 2021; Yang et al., 2021). This has resulted in significant increases in performance compared to the performance of a single index. However, these index-based works share the following disadvantages with CFD-based methods: Calculating the index requires accurate segmentation of the coronary artery lumen, which can be very challenging, especially in the presence of pathology Ghanem et al. (2019). While these methods typically use automated segmentation methods as a starting point, errors in automated segmentation often require extensive manual interaction.

[0009] Therefore, there is a need to obtain coronary lesion parameters (such as plaque type, anatomical lesion severity, and functional coronary lesion severity) independent of the specific morphology of the coronary artery system. Summary of the invention

[0010] According to aspects herein, a method for assessing vessel occlusion through deep learning-based analysis of volumetric image data is provided.

[0011] In earlier deep learning works (U.S. Application No. 15933854 and U.S. Application No. 16379248), several methods were disclosed to assess the functional severity of vascular obstruction by focusing on extracting the MPR of the region of interest of the myocardium and / or the artery of interest.

[0012] In the present application, new deep learning methods and systems are provided that use convolutional neural networks (CNNs) or variational autoencoders to extract additional features or characteristics along the vessels of interest. In addition, features or characteristics can be extracted directly from the coronary centerline tree, whether in the aorta or side branches and regardless of whether there is a bifurcation at that location, where such features indicate each coronary centerline point. These features can be used in combination with other extracted features to assess vessel obstruction. To this end, a second network is trained to perform regression of FFR values, FFR drop, pullback FFR, and classification of the functional significance of arterial obstruction.

[0013] In embodiments herein, a method for assessing occlusion of a vessel of interest in a patient includes:

[0014] Obtaining a volume image dataset (e.g., CCTA image data) for the vessel of interest (such as, for example, a coronary artery or a coronary artery tree);

[0015] analyzing the volumetric image dataset to extract data representing an axial trajectory of the vessel of interest;

[0016] generating a multi-planar reformatted (MPR) image based on the volumetric image dataset and the data representing the axial trajectory of the vessel of interest;

[0017] supplying the MPR image as input to a first machine learning network, the first machine learning network outputting feature data given the MPR image, the feature data characterizing a plurality of features of the vessel of interest along an axial trajectory of the vessel of interest;

[0018] generating additional data by separate and distinct analysis from the first machine learning network, the additional data characterizing at least one additional feature of the vessel of interest along an axial trajectory of the vessel of interest; and

[0019] The data output by the first machine learning network and the additional data are supplied as input data to a second machine learning network, which outputs data characterizing the severity of the anatomical lesion of the blood vessel of interest given the input data.

[0020] The method may further include displaying or outputting data indicative of the severity of the anatomical lesion of the vessel of interest.

[0021] The additional data may be generated by an analysis of the MPR images and / or by an analysis of the volume image dataset and / or by a coronary artery centerline tree derived from the volume image dataset.

[0022] The additional data may characterize at least one of side branches and bifurcations along the axial trajectory of the vessel of interest, and / or at least one of soft plaque area, mixed plaque area, or other characteristic features along the axial trajectory of the vessel of interest.

[0023] The additional data may also characterize a localized portion of the myocardium associated with the vessel of interest.

[0024] In an improvement, the data output by the second machine learning network includes a fractional flow reserve (FFR) value for the entire vessel of interest, and advantageously, the second machine learning network can be trained by supervised learning using training data, the training data comprising reference annotations based on measurements of FFR values ​​for multiple patients.

[0025] In a further refinement, the data output by the second machine learning network includes fractional flow reserve (FFR) values ​​for centerline points along the vessel of interest; and the second machine learning network is trained by supervised learning using training data, the training data including reference annotations based on measurements of FFR values ​​associated with vessel centerline points for multiple patients.

[0026] In a further refinement, the data output by the second machine learning network represents a prediction of the presence of functionally significant stenosis; and the second machine learning network is trained by supervised learning using training data, the training data comprising reference annotations of multiple patients indicating the presence of functionally significant stenosis.

[0027] In an embodiment, the plurality of features represented by the feature data output by the first machine learning network include at least one feature related to luminal properties of the vessel of interest (such as luminal area and / or luminal attenuation) along the axial trajectory of the vessel of interest.

[0028] In an improvement, the multiple features represented by the feature data output by the first machine learning network include at least one feature related to the plaque characteristics of the vessel of interest (such as calcium plaque area, soft plaque area, mixed plaque area) along the axial trajectory of the vessel of interest.

[0029] Advantageously, the first machine learning network may comprise a convolutional neural network trained using training data comprising reference annotations for the plurality of features represented by feature data output by the first machine learning network.

[0030] The reference annotation may be derived by manually segmenting the corresponding volumetric image data and / or automatically segmenting the corresponding volumetric image data.

[0031] Advantageously, the second machine learning network may comprise a convolutional neural network trained using training data comprising volumetric image data and corresponding reference annotations for output data characterizing the severity of anatomical lesions of the vessel of interest.

[0032] The reference annotation may be derived by manually segmenting the corresponding volumetric image data and / or automatically segmenting the corresponding volumetric image data.

[0033] The convolutional neural network of the second machine learning system may include a regression head that outputs a fractional flow reserve (FFR) value.

[0034] In an improvement, the convolutional neural network of the second machine learning system also includes an accumulator that outputs fractional flow reserve (FFR) values ​​for centerline points along the vessel of interest.

[0035] The convolutional neural network of the second machine learning system may include a classification head that outputs data representing a prediction of the presence of functionally significant stenosis.

[0036] According to one aspect, embodiments herein also relate to a system for assessing blockage of a vessel of interest in a patient, the system comprising at least one processor configured to perform some or all of the operations of the method according to embodiments herein when executing program instructions stored in a memory.

[0037] Advantageously, the system may include an imaging acquisition subsystem and / or a display subsystem, wherein the imaging acquisition subsystem is configured to acquire the volume image dataset and the display subsystem is configured to display data characterizing the severity of the anatomical lesion of the vessel of interest.

[0038] Further variations are possible. For example, an embodiment involves generating first feature data to supply to the first machine learning network, the first feature data characterizing the presence of zero or more bifurcations or side branches along the axial trajectory of the vessel of interest.

[0039] In another embodiment, some or all of the additional feature data and / or the MRP image are adjusted based on a simulated or planned treatment of the vessel of interest.

[0040] In another embodiment, advantageously, the first machine learning network may be configured to output, given the MPR image, a plurality of latent space encodings characterizing features of the vessel of interest along the axial trajectory of the vessel of interest and / or additional feature data characterizing additional features of the vessel of interest along the axial trajectory of the vessel of interest. The plurality of latent space encodings and / or the additional feature data output by the first machine learning network may be supplied to the second machine learning network, which, given input data, outputs data characterizing the FFR pullback of the vessel of interest taking into account a treatment simulated or planned for the vessel of interest.

[0041] Embodiments may also provide a method and system for extracting a coronary artery tree from volumetric image data of a vessel of interest of a patient, the method comprising one, some, or all of the following operations:

[0042] obtaining a volume image dataset for the blood vessel of interest;

[0043] Tracking a plurality of seed points in the image dataset;

[0044] extracting an initial representation of a coronary artery tree in the image dataset using the plurality of seed points;

[0045] Inputting the initial representation of the coronary artery tree into a first set of graph convolutional neural networks to generate a refined representation of the coronary artery tree;

[0046] A second set of graph convolutional neural networks is used to generate labels for segments of the refined representation of the coronary artery tree.

[0047]

[0011] Further aspects are described and claimed hereinafter. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] The characteristics of the present disclosure and the advantages derived therefrom will become more apparent from the following description of non-limiting embodiments shown in the accompanying drawings described below.

[0049] Figure 1 An example of coronary atherosclerosis is shown.

[0050] Figure 2 A flow chart of a machine learning-based method for determining the severity of functionally significant lesions in one or more coronary arteries according to an embodiment of the present application is shown.

[0051] Figure 3 A functional block diagram of an exemplary CT system is shown.

[0052] Figure 4Shown is a coronary artery centerline tree extracted from a CCTA dataset.

[0053] Figure 5a to Figure 5e The creation of a volumetric MPR image is shown.

[0054] Figure 6 An example of the architecture of a network for extracting arterial properties is shown.

[0055] Figures 7a to 7c Schematic diagram showing the co-registration method with the pullback FFR reference value of the CCTA image dataset.

[0056] Figure 8 An example of the architecture of a network for extracting five arterial characteristics is shown.

[0057] Fig. 9 is a schematic diagram of the narrow evaluation network.

[0058] Fig.10 is a schematic diagram of expanding / adapting the stenosis assessment network (FIG. 7) to include the FFR value for each centerline point along the vessel of interest as an additional output.

[0059] Fig.11 An exemplary graphical user interface (display screen) showing FFR values ​​for centerline points of a vessel of interest is shown.

[0060] Fig.12 An example of calculating FFR pullback by obtaining coronary circumflex using a CAAS workstation is shown.

[0061] Fig.13 is to expand / adjust the narrow evaluation network ( Fig. 9 or Fig.10 ) to include a schematic diagram of the extraction of soft plaque area and mixed plaque area for use in a stenosis assessment network as described by Isgum et al. in US10699407B2.

[0062] Fig.14 Is to expand / tune the deep learning network ( Fig. 9 , Fig.10 , Fig.13 ) to include a schematic diagram of the extraction of myocardial data characterizing the heart for a "machine learning based stenosis assessment" network.

[0063] Fig.15 is a schematic diagram of the heart, with the myocardium of the heart subdivided into regions covered by the coronary arteries.

[0064] Fig.16 Is to expand / tune the deep learning network ( Fig. 9 , Fig.10 , Fig.13) to include another schematic diagram of the extraction of myocardial data characterizing the heart for the "machine learning based stenosis assessment" network.

[0065] Fig.17 A flow chart of a machine learning-based method for determining functionally significant lesion severity along an axial trajectory of a vessel of interest (eg, one or more coronary arteries) according to an embodiment of the present application is shown.

[0066] Fig.18 As shown by Fig.17 The flowchart describes a high-level schematic of the machine learning-based approach.

[0067] Fig.19a is a schematic diagram of the architecture of the deployed variational autoencoder.

[0068] Fig.19b is a schematic diagram of the architecture of a deployed variational autoencoder, showing the unsupervised and supervised parts of the variational autoencoder.

[0069] Fig. 20 An example of a network architecture for an FFR pullback network is shown.

[0070] Fig.21 is a narrow evaluation network based on machine learning (206, Figure 2 ) and the machine learning-based FFR retracement network (1706, Fig.17 ) is a schematic diagram of a combined network.

[0071] Fig. 22 Provides a reference for Fig.21 Illustration of the machine learning-based FFR pullback network architecture integrating myocardial properties.

[0072] Fig.23 A flow chart of a machine learning based method for simulating virtual stent placement according to an embodiment of the present application is shown.

[0073] Fig.24 Provides illustration of changing arterial properties to simulate successful PCI treatment.

[0074] Fig.25 A method for automatically defining a lesion segment and calculating a healthy reference area within the lesion segment is shown.

[0075] Fig.26 An automatic approach to model virtual stent placement using the dense latent space of a variational autoencoder is shown.

[0076] Fig. 27 Shows the use of reference Fig. 27 Incorrect virtual stent placement of the described method.

[0077] Fig.28 A flow chart of a method based on machine learning according to an embodiment of the present application is shown, which is used for automatic extraction and anatomical labeling of a coronary artery tree.

[0078] Fig.29 Shown by Fig.28 A high-level approach to describing the flow chart.

[0079] Fig.30 The merging of sub-images in the case of overlay is shown.

[0080] Fig.31 The extraction of the coronary artery tree at multiple steps of graph tracing is shown.

[0081] Fig.32 The tracking of the coronary artery centerline leaks into the coronary veins and by dividing the segment into smaller sub-segments, well-defined segments are generated for tree refinement. This allows the centerline in the coronary arteries to be retained and the centerline in the coronary veins to be removed.

[0082] Fig.33 Shown for use with Fig.28 Flowchart of the related GCN network architecture.

[0083] Fig.34 Provided Fig.29 , which shows the extracted coronary artery tree in which each tubular artery segment is anatomically labeled.

[0084] Figures 35a to 35d Illustrations of coronary tree extraction when starting from seeds within coronary arteries and from seeds within coronary veins are provided.

[0085] Fig.36 A high-level block diagram of an example of a CT system is shown.

[0086] Fig.37 An example computer system is shown. DETAILED DESCRIPTION

[0087] The term "unseen" as used throughout the text refers to items that are not used during the training phase. Items in this context refer to volumetric images, reference values, features, and / or other things that are used to train the machine learning model during the training phase. Instead, unseen features, images, geometries, and other unseen items refer to aspects of the patient or object of interest that are being analyzed during the prediction phase of the operation.

[0088] The term "FFR value" refers to the FFR value at a specific location within a blood vessel. In the case where the FFR value is used without reference to a location (centerline location), the FFR value may refer to the FFR value at the most distal location within the blood vessel of interest.

[0089] The term "FFR pullback" or "FFR pullback curve" refers to the FFR value along the axial trajectory of the vessel of interest, e.g., Fig.12 1203 or Fig.26 The 2610 is shown.

[0090] The term "FFR drop" refers to the attenuation of the FFR value along the axial trajectory of the vessel of interest from the proximal end to the distal end of the vessel. The steepness of this attenuation allows the separation of focal coronary artery disease from diffuse coronary artery disease. Focal coronary artery disease can be defined as an abrupt pressure drop (FFR drop) with FFR pullback within a relatively short vessel segment. On the other hand, diffuse coronary artery disease is defined as a gradual pressure loss (FFR drop) along the axial trajectory of the vessel without a noticeable abrupt pressure drop at any location along the vessel.

[0091] Focal lesions are localized blockages that can be treated by dilating the stenosis with an inflatable balloon and possibly followed by placement of a stent or scaffold. Diffuse lesions require different treatment approaches and need to be distinguished from focal lesions to prevent unnecessary costs, patient risks, and patient comfort through suboptimal treatment decisions. Therefore, FFR pullbacks can be used to determine whether a vessel has focal or diffuse lesions based on the shape of the virtual pullback.

[0092] Throughout this specification, common terms in the field of machine learning / deep learning are used. For a detailed explanation of these terms, refer to "A survey on deep learning in medical image analysis" by Litjens et al. (Med Image Anal. December 2017; 42: 60-88), "A review on deep learning in medical image analysis" by Suganyadevi et al. (International Journal of Multimedia Information Retrieval, Vol. 11, 1 (2022)), and "Machine learning for medical imaging: methodological failures and recommendations for the future" by Varoquaux et al. (NPJ Genomic Medicine, Vol. 5, 1 48, April 12, 2022); the entire contents of which are incorporated herein by reference.

[0093] The present application relates to methods and systems for machine learning to assess functionally significant vascular occlusion of one or more blood vessels of a target organ based on a contrast-enhanced volumetric image dataset. Machine learning is a subfield of computer science that "gives computers the ability to learn without being explicitly programmed." Machine learning has evolved from the study of pattern recognition and computational learning theory in artificial intelligence. Machine learning explores the study and construction of algorithms that can learn from data and make predictions about data - such algorithms overcome the need to follow strict static program instructions by making data-driven predictions or decisions and by building models from sample inputs. Machine learning is employed in a range of computing tasks when designing and programming explicit algorithms is not feasible. Given a dataset of images with known class labels, a machine learning system can predict class labels for new images. In addition, machine learning can also be unsupervised, using machine learning algorithms to analyze and cluster unlabeled datasets. These algorithms discover hidden patterns or data groupings without human intervention. At a high level, machine learning can be divided into two phases; 1) a training phase, in which the model is trained to learn the specific features of the task (e.g., FFR prediction); and 2) a testing / validation phase, in which the trained model is deployed on unseen data to perform the task (e.g., prediction of FFR).

[0094] In an embodiment, the target organ may be a coronary artery or blood vessel, and may be the heart or a portion of the heart. A functionally significant vascular obstruction (also referred to as a stenosis or lesion) is a hemodynamically significant obstruction of a blood vessel, and with respect to the coronary arteries, defines the likelihood that the obstruction of (one or more) coronary arteries will impede oxygen delivery to the myocardium and cause angina symptoms. Fractional flow reserve is a hemodynamic index used to assess functionally significant coronary artery obstruction (one or more). In addition to fractional flow reserve, other hemodynamic indices may be used to assess functionally significant coronary artery obstruction (one or more), such as coronary flow reserve, transient wave-free ratio, hyperemic myocardial perfusion, microcirculatory resistance index, and pressure drop along the coronary arteries.

[0095] Embodiments of the present application utilize machine learning to determine coronary artery parameters associated with CAD, such as the functional severity of one or more vessel occlusions from a CCTA dataset. Machine learning is a subfield of computer science that "gives computers the ability to learn without being explicitly programmed." Developed from the study of pattern recognition and computational learning theory in artificial intelligence, machine learning explores the study and construction of algorithms that can learn from and make predictions about data - such algorithms overcome the need to follow strict static program instructions by making data-driven predictions or decisions, by building models from sample inputs. Machine learning is employed in a range of computing tasks where designing and programming explicit algorithms is not feasible.

[0096] Figure 2 , Fig.17 , Fig.23 and Fig.28 Each depicts a flow chart illustrating operation according to an embodiment of the present application. The operation employs an imaging system capable of acquiring and processing CCTA datasets of an organ (or portion thereof) or other object of interest. Figure 2 , Fig.17 , Fig.23 and Fig.28 The operations of (and the operations of any other methods, algorithms, and processes described herein) are performed by one or more processors when executing program instructions. The one or more processors may be implemented on a variety of computing devices, such as smart phones, tablet devices, laptop computers, desktop computers, workstations, remote servers, cloud servers, medical networks, etc. Alternatively, the one or more processors may be distributed among one or more separate computing devices, such that a portion of the operations are performed by one computing device, while the remainder of the operations are performed by one or more other computing devices.

[0097] Figure 3 30 is a functional block diagram of an exemplary CT system, which includes a CT imaging device 302, which operates under commands from a user interface module 301 and provides data to a data analysis module 303. A clinician or other user acquires at least one CT image of a patient to obtain CCTA image data of a volume of interest (e.g., the coronary arteries around the patient's heart). The CCTA image data can be stored in a DICOM format on a hard disk, a PACS server, a network server, or a cloud server. The data analysis module 303 can be implemented by a personal computer, a workstation, or other computer processing system. The data analysis module 303 processes the acquired CCTA image data of the CT system 302 to generate, for example, coronary artery analysis quantification.

[0098] The user interface module 301 interacts with the user and communicates with the data analysis module 303. The user interface module 301 may include different kinds of input and output devices, such as a display screen for visual output, a touch screen for touch input, a mouse pointer or other pointing device for input, a microphone for voice input, a speaker for audio output, a keyboard and / or keypad for input, etc. Module 304 provides an assessment of the patient's significant coronary artery stenosis. In order to assess the functional significance of the stenosis in the patient's vessel of interest, module 304 is configured to apply deep learning as described in the present application directly to the raw CCTA image data acquired by the system. In order to achieve robust training with limited data, two subsequent networks (artery characterization network ( Figure 2 , 204, or Fig.17 , 1704) and narrow evaluation networks ( Figure 2 , 206)) or FFR retracement network ( Fig.17 , 1706) to divide the task. Preprocessing for artery characterization network ( Figure 2 202 or Fig.17 1702) extracts data representing the axial trajectory of the blood vessel of interest from the CCTA image data of the blood vessel of interest, and extracts data representing the axial trajectory and the image data ( Fig.17 of 1703 Figure 2 203) generates a multi-planar reconstruction (MPR) of the vessel of interest. The MPR may be represented by a volumetric (three-dimensional or "3D") MPR image of the vessel of interest and / or a two-dimensional (or "2D") MPR image of the vessel of interest.

[0099] Figure 2 , Fig.17 , Fig.23 or Fig.28 The operations of may be performed by software code implemented in a computer product (e.g., a disc or other form of permanent storage (such as a USB drive, network server, or cloud server)). The software code may be loaded directly into the memory of a data processing system for execution. Figure 2 , Fig.17 , Fig.23 or Fig.28 operation.

[0100] In this example, it is assumed that the imaging system has acquired and stored at least one CCTA dataset of a vessel of interest.For this purpose, any imaging device capable of providing a CT scan may be used.

[0101] The present application is particularly advantageous in the analysis of coronary lesion parameters based on CCTA datasets and it will be disclosed primarily with reference to this field, in particular with regard to patient classification.

[0102] Now refer to Figure 2 The embodiments of the present application are disclosed. Obviously, the steps described herein can be performed in any logical order and can be partially omitted.

[0103] exist Figure 2 In step 201, a CCTA image dataset is obtained. Such a CCTA image dataset may represent a volumetric CCTA image dataset, such as a single contrast-enhanced CCTA dataset. The CCTA dataset may be obtained from a digital storage database (such as a picture archiving and communication system (PACS) or a VNA (vendor neutral archive), a local digital storage database, a cloud database), or may be directly obtained from a CT imaging modality. The CCTA dataset may be obtained by a CT imaging operation in which a contrast agent is injected to enhance a vessel of interest (e.g., a coronary artery or blood vessel). At least one purpose of CCTA is to identify cardiac and coronary anatomical structures by injecting exogenous contrast agents (usually by intravenous injection in the antecubital vein to enhance cardiac and / or coronary anatomical structures during imaging). According to the Society of Cardiovascular Computed Tomography guidelines for the performance and acquisition of CCTA described by Abbara et al. in "SCCT guidelines for the performance and acquisition of coronary computed tomographicangiography: A report of the Society of Cardiovascular Computed Tomography Guidelines Committee Endorsed by the North American Society for Cardiovascular Imaging (NASCI)" (J Cardiovasc Comput Tomogr. 2016 Nov-Dec; 10(6):435-449), contrast medium is injected at a time in such a way that the coronary system contains enough contrast medium to clearly distinguish the coronary artery lumen from the surrounding soft tissue. This enables the physician to assess luminal narrowing and coronary artery stenosis with optimal image quality and therefore with optimal accuracy. To ensure adequate coronary artery opacity, the guidelines describe starting CCTA image acquisition typically when a predetermined threshold attenuation value is reached in a predetermined anatomical structure (most commonly this involves the descending aorta), or by waiting a certain delay time after the first visible enhancement in the ascending aorta. Additionally, CT imaging operations may be triggered by analyzing the patient's ECG signals.

[0104] In step 202, the processor extracts data representing an axial trajectory extending along the vessel of interest. For example, the axial trajectory may correspond to a centerline extending along the vessel of interest. When the vessel of interest represents a coronary artery, the axial trajectory may correspond to a coronary artery centerline, wherein the processor extracts the coronary artery centerline. The coronary artery centerline represents the center of the coronary arterial lumen along the coronary artery portion of interest. This may be a single coronary artery, a coronary artery bifurcation, or a complete coronary artery tree. In the event that the coronary artery portion of interest includes one or more bifurcations, the coronary artery centerline will also include the bifurcations, but not their side branches.

[0105] In the case of analyzing bifurcations and / or coronary artery trees, data representing multiple centerlines may be extracted in step 202. For the purposes of the current application, it is not necessary for the extracted coronary centerline to accurately represent the center of the coronary artery lumen. Although the coronary centerline should not exceed the coronary artery lumen, a rough estimate of the coronary centerline is sufficient. The extraction of the coronary centerline can be performed manually or (semi) automatically. An example of a semi-automatic method is described by Metz et al. in "Semi-automatic coronary artery centerline extraction in computed tomography angiography data" (Conference Proceedings / IEEE International Symposium on Biomedical Imaging: From Nano to Macro, May 2007). An example of an automatic coronary centerline extraction method is described by Wolterink et al. in "Coronary artery centerline extraction incardiac CT angiography using a CNN-based orientation classifier" (Med Image Anal. 2019 January; 51: 46-60), in which machine learning is used to automatically extract the coronary centerline. The method extracts the coronary artery centerline between the ostium and the most distal point as visible in the CCTA image data set after placing a single seed point in the artery of interest. In a preferred embodiment, the complete coronary artery centerline tree is automatically extracted, and each coronary artery segment is automatically labeled, for example, according to the model introduced by the American Heart Association (Austen et al. "A reporting system on patients evaluated for coronary artery disease. Report of the Ad Hoc Committee for Grading of Coronary Artery Disease, Council on Cardiovascular Surgery, American Heart Association", 51st edition, 5-40, 1975). Reference Fig.28 Flow chart and its Fig.29 The method is further described in the high-level method shown. The vessel of interest (e.g., left anterior descending artery (LAD), left circumflex artery (LCx), or right coronary artery (RCA)) may be identified or predefined by the user, or by reference to Fig.28 The flowchart discusses the method to automatically identify vessels of interest. Figure 4An example of a centerline tree (403) extracted from a CCTA dataset (401) is shown. The CCTA dataset consists of several 2D images (401), resulting in an acquisition volume represented by 402, which is represented by volume rendering of the acquisition volume. In the case of multiple arteries, the centerline tree (403) can be automatically performed on multiple vessels of interest (e.g., on the LAD, LCx and RCA) as shown by Figure 2 or Fig.17 or Fig.23 The plurality of vessels of interest may be predefined and may be determined by Fig.28 The described method automatically extracts.

[0106] In step 203, a three-dimensional (3D) multi-planar reformatted (MPR) image of the coronary artery of interest is created using the data representing the axial trajectory (or centerline) extending along the vessel of interest as extracted in step 202. Figure 5a to Figure 5b An illustration of creating a volumetric 3D MPR image is provided. Figure 5a Image 501 shows the CCTA dataset ( Figure 2 , 201), wherein the right coronary artery 502 is selected as an example to create a 3D MPR image. Regarding the 3D MPR images, a distinction is made between straight MPR and curved MPR. For straight MPR as well as for curved MPR, an isotropic 3D MPR image is created from the image dataset 201 using the extracted axial trajectory (e.g., centerline). The resolution of the 3D MPR image is predefined, e.g., 0.2 mm, using, e.g., trilinear interpolation. The 3D MPR image can also be created in a non-isotropic manner, e.g., with an in-plane pixel size of 0.1 mm and a distance of 0.25 mm between consecutive centerline points using, e.g., trilinear interpolation.

[0107] Creation of a 3D MPR image may involve sampling image data along an extracted axial trajectory 502 (eg, a coronary artery centerline) to define a cuboid image 503 with the coronary artery centerline located at the center 504 of the cuboid image, thereby producing a rectilinear MPR. Figure 5b Image 503 shows a cuboid resampled image (direct MPR), and for ease of explanation, one "slice" is visualized within the cuboid resampled image. Figure 5c Image 505 shows one "slice" of the same resampled image, but with the visualization plane rotated about centerline 504 to show visualization of the coronary bifurcation (506) within the extracted right coronary artery.

[0108] Alternatively, creation of a 3D MPR image may involve sampling the image along a curved path along the axial trajectory 502 (eg, the coronary artery centerline), thereby producing a curved MPR. Figure 5d The image 508a and Figure 5e Image 508b shows two examples of curved 3D MPR images and is visualized as a single "slice" where the slice orientation refers to the curved plane that can be rotated along the curved coronary artery. Again, this is for visualization purposes only, and the application will use either full 3D rectilinear MPR images or curved MPR images. The advantage of the curved MPR images is that the curvature or tortuosity of the extracted centerline can be taken into account within the described machine learning network architecture within this application.

[0109] In an alternative embodiment, the MPR image created in step 203 may be represented by a two-dimensional (or "2D") MPR image of the vessel of interest.

[0110] In step 204, the MPR image of step 203 is supplied to a machine learning based artery characterization network which, given the MPR image as input, extracts data (signals) characterizing features of the vessel of interest along the centerline of the vessel of interest.

[0111] In an embodiment, the machine learning-based artery characterization network of step 204 employs a convolutional neural network (CNN) architecture. A CNN architecture typically includes an input layer, a hidden layer, and an output layer. The hidden layer includes one or more layers that perform convolution. Typically, this includes a layer that performs a dot product of a convolution kernel with the input matrix of the layer. The product is typically a Frobenius inner product, and its activation function is typically a rectified linear unit (ReLU). As the convolution kernel slides along the input matrix for the layer, the convolution operation generates a feature map, which in turn contributes to the input of the next layer. This is followed by other layers, such as pooling layers, fully connected layers, and normalization layers.

[0112] Figure 6 An example CNN architecture for extracting arterial properties is shown. For each point along the centerline of the vessel of interest, the arterial characterization network predicts multiple arterial properties related to the severity of arterial lesions based on the MPR image (601). Figure 6 In the artery characterization network shown, three features are extracted through a CNN architecture. These features or characteristics include lumen cross-sectional area, lumen attenuation (optionally), and calcification area, all of which are specified along the axial trajectory of the vessel of interest. An example of a CNN architecture is shown in Figure 1. Figure 6602 in which a stack of a predetermined number of consecutive cross-sectional slices (e.g., 5 cross-sections) within the MPR image is analyzed and consists of, for example, four alternating convolution blocks and pooling operations. The convolution block includes two convolution layers (e.g., with a kernel size of 3 and 16 filters), each of which is followed by batch normalization and a rectified linear unit (ReLU) activation function. Finally, three separate output head regression values ​​for the lumen area, the average attenuation in the lumen, and the calcification area of ​​the central slice of the input stack are provided, thereby generating output arterial characteristics (in the current example, lumen area, lumen attenuation, and calcification area) along the axial trajectory (e.g., centerline) of the vessel of interest ( Figure 6 , 603).

[0113] To train the artery representation network ( Figure 2 204 or Figure 6 CNN), using the arterial reference value ( Figure 2, 208). The arterial reference values ​​(208) are provided by a database containing data of a plurality of patients. Each set within the database contains, for each patient, a) a contrast-enhanced CT image dataset (201 represents a reference image set during the training phase) and corresponding b) arterial characteristic reference values. For the three characteristics described above, reference annotations of the coronary arterial lumen and coronary artery calcification are required, from which the lumen cross-sectional area, calcification area and lumen attenuation along the axial trajectory of the vessel of interest are derived. These reference annotations can be created by manually segmenting the CCTA dataset used during training, or by automatically segmenting the lumen and calcification of the CCTA dataset used during training and then manually correcting it when necessary. Automatic segmentation of lumen and calcification can be performed, for example, by segmenting the original CT image volume using methods such as those described by Wolterink et al., "Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks" (Medical Image Analysis 34, 123-136 (2016)) and Wolterink et al., "Graph Convolutional Networks for Coronary Artery Segmentation in Cardiac CT Angiography" (in Zhang D, Zhou L, Jie B, Liu M, ed. Graph Learning in Medical Imaging. Lecture Notes in Computer Science. Cham: Springer International Publishing (2019), pp. 62-9). Thereafter, the automatic segmentation is transformed into the MPR of the artery, visually inspected and corrected when necessary. Since X-ray angiography is superior in image resolution compared to CCTA, in a preferred embodiment, lumen segmentation on the MPR image is guided by QCA3D analysis results extracted from X-ray angiography image data from the same patient, which was acquired within three months of the CCTA data of the patient in question.QCA3D can be performed, for example, using the method as described by Girasis C et al. in "Advanced three-dimensional quantitative coronary angiographic assessment of bifurcation lesions: methodology and phantom validation" (EuroIntervention 2013; 8: 1451-1460) using a CAAS workstation (Pie Medical Imaging BV, The Netherlands).

[0114] To ensure that the arterial reference values ​​(208) are aligned with the spatial coordinates of the MPR image (203), the arterial reference values ​​may be transformed to the MPR image domain. This step may be skipped in case the arterial reference values ​​are obtained by using the MPR image as a result from step 203 (e.g., manual annotations of plaque type, functional lesion severity such as FFR for example). When the arterial reference values ​​are obtained, for example, by using annotations of a contrast-enhanced CT dataset (step 201), this step transforms the annotations into an MPR view. Such a transformation is performed by using the extracted centerline (as a result of step 202). The transformation of the coronary artery tree characteristics (205) may be performed according to the above description.

[0115] To ensure that the fractional flow reserve value along the coronary artery as measured in the catheterization laboratory (e.g., the pullback FFR reference value) is aligned with the spatial coordinates of the MPR image, a co-registration is performed between the image dataset 201 and the invasively measured pullback FFR. To allow co-registration of the pullback FFR measurement results with the CT dataset, pullback motion information is obtained, which indicates the pullback rate or speed during the pullback of the FFR line from the FFR line starting position (e.g., the distal position in the coronary artery) to the FFR line ending position (e.g., the proximal position in the coronary artery or the opening of the coronary artery). The pullback motion information can be obtained by measuring the longitudinal motion of the FFR line during the pullback. The measurement can be obtained in various ways, such as with the help of a motion measurement system, or for example by using a motorized pullback device that maintains a constant pullback speed. One or more processors of the system use the time required to pull back the FFR line and the pullback speed to calculate the length of the pullback distance. In order to align the pullback FFR artery reference value to the MPR image, the one or more processors transform the length of the pullback distance to the image dataset 703 used.

[0116] Figures 7a to 7c A schematic diagram of a co-registration method using a pullback FFR reference value (eg, pullback distance) of a CCTA image dataset is provided. Figure 7aImage 701 shows an x-ray coronary angiography image acquired in a catheterization laboratory. Figure 7b Image 702 shows a volume rendered CCTA image belonging to the same patient. Figure 7c Image 706 shows an x-ray fluoroscopic image without the presence of contrast fluid.

[0117] Figure 7a An x-ray coronary angiogram image 701 of FIG. 701 shows the right coronary artery with the FFR pressure wire inserted into the right coronary artery to a desired distal location where the pressure sensor can obtain a first / distal pressure measurement of interest. Point 703 indicates the position of the pressure sensor on the FFR pressure wire when located at a distal location within the coronary artery prior to being pulled back on the x-ray angiogram image. The location represented by point 703 may also be referred to as the distal pressure sensor location. Due to the radiopaque markers on the FFR line 707 (e.g., Figure 7c 706 ) enables positioning of the pressure sensor along the FFR line, the distal location of the pressure sensor (and the entire FFR pressure line) being easily identifiable on the x-ray fluoroscopic image (without contrast fluid present, 706 ).

[0118] Figure 7b Image 702 shows a volume rendered CCTA image (of the same patient for which the pullback FFR reference value was obtained). In image 702, the right coronary artery 704 is identified, for example, as Figure 2The result of step 202. Co-registration of the pullback FFR reference value is performed by identifying the position of the FFR pressure line before the pullback in the CCTA dataset (705), for example, manually identifying the FFR reference value supported by anatomical landmarks (such as, bifurcation locations), and aligning the FFR values ​​by matching the length (length of the 3D extracted centerline and the length of the FFR pullback). Identification of the position of the FFR pressure line before the pullback in the CCTA dataset can also be performed by registering the x-ray angiography image with the CCTA dataset, such as, for example, by using the method of Baka et al. "Oriented Gaussian Mixture Models for Nonrigid 2D / 3D Coronary Artery Registration" (IEEE Trans Med Imaging. 2014 May; 33(5): 1023-34). Baka et al. describe a method for registering 2D x-ray angiography images to a 3D volumetric image dataset (CCTA) by using a point set registration technique based on a Gaussian mixture model (GMM). Since positioning of the pressure sensor (707) on the FFR line can be easily performed using the x-ray fluoroscopic image (706) with the aid of image processing techniques, the position can be directly transformed into CCTA image data (705) using the deformation field generated by 2D / 3D registration as described by Baka et al.

[0119] In other embodiments, the machine learning-based artery characterization network of step 204 may employ a variational autoencoder (VAE) architecture configured to extract features or characteristics of a vessel of interest given an MPR image of the vessel of interest as input. The details of the variational autoencoder (VAE) architecture are described below with respect to Fig.17 Step 1704 and Fig.19a and Fig.19b to describe.

[0120] In yet other embodiments, the machine learning-based artery characterization network of step 204 may be configured to extract other data (signals) characterizing features of the vessel of interest given an MPR image as input. This may be achieved by including multiple artery characteristics in the artery characterization. For example, Figure 8 An example CNN network architecture is shown for extracting five arterial characteristics along the centerline of a vessel of interest based on an MPR image (801). The five arterial characteristics include lumen cross-sectional area, lumen attenuation, calcification area, soft plaque area, and mixed plaque area. Figure 6 Compared with the network shown and described, Figure 8During the training of the network shown, two additional reference features (soft plaque and mixed plaque) need to be extracted from the annotation. The annotation can be based on manual segmentation or (semi-)automatic segmentation in the CCTA image dataset. Automatic plaque segmentation can be based on the method described, for example, by Jávorszky et al. in "Deep learning-based atherosclerotic coronaryplaque segmentation on coronary CT angiography" (European Radiology Vol. 32, 10 (2022)).

[0121] Depend on Figure 8 802 in provides an example of the CNN architecture, which analyzes a stack of a predetermined number of consecutive cross-sectional slices (e.g., 5 cross-sections) and consists of, for example, four alternating convolution blocks and pooling operations. The convolution block includes two convolution layers (e.g., kernel size 3, with 16 filters), each of which is followed by batch normalization and a ReLU activation function. Finally, five separate output head regression values ​​for the lumen area, average attenuation in the lumen, calcification area, soft plaque area, and mixed plaque area of ​​the central slice of the input stack are provided, thereby generating output arterial characteristics along the centerline of the vessel of interest (in the current example, lumen area, lumen attenuation, calcification area, soft plaque area, and mixed plaque area) ( Figure 8 , 803).

[0122] The geometry of the blood vessels has an impact on the characteristics of blood flow and the local appearance of the vessels. Figure 2 In step 205, Figure 2 The axial trajectory of the blood vessel of interest is extracted from the CCTA image data of step 201 (e.g., from Figure 2 Other data (signals) characterizing the characteristics of the vessel of interest (obtained in step 202) may be extracted from the CCTA image data. For example, for each point along the axial trajectory (i.e., centerline) of the vessel of interest, two additional features may be extracted from the CCTA image data. The first indicates the presence of a bifurcation at the arterial centerline point ( Fig. 9 901). Figure 2 As described in step 202 of FIG. 1 , the coronary centerline of the vessel of interest may be performed manually or (semi-) automatically. In a preferred embodiment, the coronary centerline of the vessel of interest may be determined using a method such as Fig.29 The method shown automatically extracts a complete coronary artery centerline tree. When using this method, the presence and location of bifurcations along the centerline of the artery of interest can be automatically determined by identifying centerline points along the centerline of the artery of interest where branching arteries exist. Alternatively, the user can manually identify, for example, CCTA slices ( Figure 4 401) or volume-rendered CCTA images ( Figure 4 In yet another alternative method, a deep learning network is trained on the MPR image to automatically identify branch arteries (bifurcation). The second additional feature indicates whether the centerline point belongs to a main branch (i.e., left main (LM), LAD, LCX, RCA) or a side branch ( Fig. 9 902). This can be done using Fig.29 The automatic anatomical labeling method shown is for example Figure 2 The characteristics may be automatically extracted as described in step 202 of . In addition, a vessel of interest may be identified or selected by a user. If the vessel of interest includes a side branch or a bifurcation, the user may identify the location of the side branch or bifurcation. These characteristics may be normalized to zero mean and unit variance for the training data set. An example of additional characteristics characterizing a vessel of interest may be the presence of a stent (from a previous coronary intervention) at a point in the vessel centerline and / or the type of stent.

[0123] In step 206, the machine learning-based narrow assessment network is configured to, given the first network ( Figure 2 204 or Figure 6 or Figure 8 The feature data output by CNN and the feature data from CCTA image data ( Figure 2 The functional significance of stenosis is evaluated by using other feature data extracted from 205) as input.

[0124] In an embodiment, a machine learning based narrow assessment network may utilize a CNN network architecture as described herein. Fig. 9 An example CNN network architecture for stenosis assessment is shown. At a high level, Fig. 9 The CNN architecture consists of three stages. In the first stage (903), the representation network ( Figure 2 204) The predicted lumen area and its attenuation are first pre-encoded and then concatenated with the calcification area and with additional features indicative of bifurcation and as Figure 2 Based on the results of step 205, it is determined whether to perform the analysis in the main branch or side branch of the artery. In the second stage (904), the combined encoding is fed to the encoder. In the encoder, the features are first pooled, and then convolution and transformer layers are applied. For the final classification, in the third stage (905), two separate output heads (regression head and classification head) are applied. In the regression head, the output of the second stage (904) is processed by two convolutional layers and ReLU activation functions. The resulting sequence is pooled along the artery dimension and subtracted from 1 to produce a single FFR value. In the classification head, the output of the second stage (904) is aggregated to a fixed length of, for example, 2.5 mm. Thereafter, two dense layers are used in combination with a sigmoid activation function to produce an output probability for the presence of a functionally significant stenosis in the artery.

[0125] The following describes in detail Fig. 9 Three stages of a machine learning-based narrow evaluation network.

[0126] In the first stage (903), the network receives as input five arterial features (luminal area, mean luminal attenuation (optional), calcification area, bifurcations, and side branches). In order to focus on changes in the luminal area and its attenuation rather than their absolute values, the percentage difference is calculated for each location in the artery relative to the previous location. Because relevant features in the luminal area and its attenuation may be subtle and may be present at different locations along the artery (i.e., stenosis is expected to cause changes in the attenuation of the luminal area distal to the appearance), these two features are first encoded separately. This is done using two non-shared convolutional layers with a leaky rectified linear unit (Leaky ReLU) activation function applied between the layers. Thereafter, the remaining features are concatenated with the encoded features from the luminal area and its attenuation.

[0127] In the second stage (904), the information of all five arterial characteristics extracted is merged by a common encoder, which consists of a convolutional layer and a transformer layer as follows: In order to increase the receptive field and reduce the dimensionality, an average pooling with a kernel size of, for example, 4 is applied, followed by two convolutional layers with dilations of, for example, 1 and 2, respectively. Each convolutional layer is followed by a Leaky ReLU activation function, sample normalization, and dropout. Subsequently, the arterial encoding is cascaded with the original lumen area and its attenuation and fed to the transformer layer (Vaswani et al., "Attention is All you Need", Advances in Neural Information Processing Systems. Vol. 30. Curran Associates Inc. 2017). Due to the global receptive field, the transformer layer connects all arterial points to each other. This potentially enables modeling of interactions between multiple lesions and proximal and distal segments of the artery.

[0128] In the third stage (905), two separate output heads (a regression head and a classification head) are configured to perform separate tasks: the regression head performs regression of the FFR value, and the classification head performs classification of the presence of functionally significant stenosis in the artery. Inspired by the additive nature of sequential flow resistance, the regression head is designed to predict the pressure drop along the artery. First, two convolutional layers are applied, each followed by a Leaky ReLU activation function, example normalization, and dropout. Thereafter, a third convolutional layer with a single output filter map is followed by a ReLU activation function to enforce the positivity of the pressure drop. Finally, a sum pooling layer is used to sum the predicted pressure drops along the artery, and the resulting total FFR drop is transformed into a final FFR value by subtracting the resulting total FFR drop from 1 (907). The classification head output (906) predicts the presence of functionally significant stenosis (FFR≤0.8). To explicitly associate the proximal and distal segments, first, adaptive sum pooling with, for example, 5 output features is applied, followed by, for example, two dense layers each with Leaky ReLU activation and dropout. Finally, a dense layer with a single output filter map and sigmoid activation produces functionally distinct narrow output probabilities.

[0129] In an embodiment, Fig. 9 All convolutions in the entire narrow evaluation network can use a kernel size of, for example, 3 combined with zero padding to prevent feature shrinkage. In addition, for all convolutions and for the transformer, a relatively small number of filter maps (e.g., 16) are used to balance the required expressiveness and prevent overfitting. For the same purpose, all dropout probabilities can be set to, for example, 0.5.

[0130] To train the narrow evaluation network ( Fig. 9 CNN Figure 2 206), the reference standard ( Figure 2 , 209). The reference standard is a database containing data of multiple patients. Each set within the database contains, for each patient, a) a contrast-enhanced CT image dataset (201 represents a reference image set during the training phase) and corresponding b) CAD-related reference values, which represent hemodynamic indices for assessing functionally significant coronary artery obstruction. For example, the CAD-related reference values ​​may represent at least one invasively measured fractional flow reserve, coronary blood flow reserve, instantaneous wave-free ratio, resting full cycle ratio, diastolic-free hyperemia ratio, diastolic pressure ratio, resting Pd / Pa ratio, hyperemic myocardial perfusion, microcirculatory resistance index, pressure drop along the coronary arteries, and fractional flow reserve along the coronary arteries. Note that the reference standard (209) and the arterial reference values ​​(208) require the same contrast-enhanced CT image dataset.

[0131] During training, the regression head is supervised using the mean squared error with, for example, a CAD reference FFR. Since the invasive reference FFR is usually not measured at the most distal location, the predicted pressure drop contribution from anatomical locations far from the measurement location is masked during training and testing. Based on the measurement protocol from clinical practice, the measurement location is assumed to be, for example, 10 mm away from the annotated lesion location. The classification task is supervised using a binary cross entropy loss function. The loss terms of the regression head and the classification head are weighted equally.

[0132] Finally, in step 207, one or more outputs are provided. In an embodiment, the output represents the probability of the presence of functionally significant stenosis in the vessel of interest. In yet another embodiment, the output represents FFR as a value between 0.0 and 1.0. In order to combine the strength of the results from the classification head (906) and the results of the regression head (907), their outputs are merged into a single probability for the presence of functionally significant stenosis in the vessel of interest. Although the classification head directly predicts the probabilities for the positive and negative classes, the regressed FFR values ​​are distributed around a threshold value (≤0.8) of the positive FFR and in the range [0.0, 1.0]. To allow their merging, the predicted FFR values ​​are first transformed into pseudo-probabilities by linearly scaling a symmetric window around the positive FFR threshold of 0.8 using the formula of Equation 1:

[0133]

[0134] To obtain the final predictions for the output, the pseudo probabilities are averaged with the probabilities from the classification head.

[0135] Optionally, in order to increase the robustness of the prediction results and determine the uncertainty of the prediction results, the output of step 207 can be calculated as the average of multiple training networks (both 204 and 205). For example, by performing a random ten-fold cross validation, in which ten networks are trained on a random 90% subset and tested on the remaining 10%. During the test, the network is integrated by averaging the predicted probabilities and FFR values, as for example, by Müller et al. in "An analysis on ensemble learning optimized medical image classification with deep convolutional neural networks" (IEEE Access, Vol. 10, pp. 66467-66480, 2022). For the prediction of uncertainty, the standard deviation is calculated based on the probability and FFR value, as for example, by Lakshminarayanan et al. in "Simple and scalable predictive uncertainty estimation using deep ensembles" (Advances in Neural Information Processing Systems. Vol. 30. Curran Associates Inc (2017)). The uncertainty measurement may be valuable in clinical practice, where the method may be used in a semi-automatic setting by introducing, for example, a hybrid approach. In particular, patients with arteries where the method indicates a high prediction uncertainty may be recommended for invasive measurements.

[0136] Experimental setup

[0137] In the embodiment, the artery representation network is trained for 800 epochs using the mean absolute error as the loss function and the ADAMW (Loshchilov and HutterF, "Decoupled weight decay regularization", International Conference on Learning Representations-ICLR 2019) optimizer ( Figure 2 204 or Figure 6 CNN), and the learning rate is 10 -5 , with a batch size of 512. The loss term for lumen attenuation was scaled by a factor of 0.1. After training, the network was applied to each cross section of the MPR to obtain the lumen area, its mean attenuation, and the calcification area along the length of the artery.

[0138] In an embodiment, the narrow evaluation network is trained for 150 epochs using an ADAMW optimizer with a cyclic learning rate of a linear schedule ( Figure 2 206 or Fig. 9 The cyclic learning rate was varied between 5e-4 and 1e-5 over a period of 40 epochs. Because the different artery lengths constrained the network for stenosis assessment to process only a single artery at a time, the loss was accumulated over 8 training iterations before backpropagation, corresponding to an effective batch size of 8.

[0139] Figure 2 An extension of the flowchart.

[0140] This section describes the Figure 2 Several extensions of the flowchart of the described machine learning-based method for determining the severity of functionally significant lesions in one or more coronary arteries.

[0141] Extension 1: FFR value for each centerline point.

[0142] like Figure 2 206 described and relative to Fig. 9 The machine learning-based stenosis assessment network further illustrates that it can be configured to provide two outputs: a regressed FFR value for the entire vessel of interest; and a binary classification of the presence of functionally significant blood flow obstruction in the vessel of interest. Figure 3 In particular, the architecture of the deep learning network can be adapted to provide a regressed FFR value for a centerline point along a vessel of interest, thereby generating an FFR value at the centerline point of the vessel of interest (FFR pullback curve). Fig.10 Shown based on Fig. 9 This extended architecture of the narrow evaluation network architecture presented.

[0143] About narrow evaluation network ( Figure 2 206, and relative to Fig. 9 The regression head is extended to include FFR output for each centerline point. All other methods are similar to those from Figure 2 The extension focuses on the regression head as part of the narrow evaluation network ( Fig. 9 ). The regression head is designed to predict the pressure drop along the artery. First, two convolutional layers are applied, each followed by a leaky ReLU activation function, example normalization, and dropout. Thereafter, a third convolutional layer with a single output filter map is followed by a ReLU activation function to enforce the positivity of the pressure drop, which corresponds to predicting the FFR drop at each location or point along the centerline of the vessel of interest (given as Figure 6 601 and Figure 8 801) and by Fig.10 This is performed by the "Accumulate" block shown in 1001 of FIG. This thereby produces an FFR drop for each centerline point along the vessel of interest, and the FFR drop can be converted to an FFR value for each centerline point by subtracting the FFR drop from 1.0 ( Fig.10 , 1002), thereby generating an FFR pullback curve. The FFR value of each centerline point of the vessel of interest can be visually presented to the user. For example, the FFR value of each centerline point can be drawn (visually depicted) along the vessel of interest in an image of the vessel of interest. In another example, the FFR value of each centerline point can be drawn (visually depicted) in a CCTA volume rendering of the vessel of interest. In yet another example, the FFR value of each centerline point can be drawn (visually depicted) in a 3D model of the vessel of interest. In yet another example, the FFR value of each centerline can be drawn (visually depicted) by color coding a 3D representation of the vessel of interest (in a CCTA volume rendering view or in a 3D segmentation) representing the vessel of interest.

[0144] Fig.11 An exemplary graphical user interface (display screen) that visually conveys such information to a user is shown. Fig.11 1102 shows a volume rendered image of CCTA data in which the three main coronary arteries (right coronary artery (RCA), left anterior descending coronary artery (LAD) and left circumflex coronary artery (LCX)) are enhanced. The RCA is the vessel of interest and is generated by Fig.11 The multi-planar reconstruction (MPR) shown in section B of 1101 is Fig.11In 1101-section B, the left side (1103) corresponds to the opening (proximal) position of the RCA, and the right side (1104) corresponds to the distal position of the RCA. 1101-section C shows a luminal area or luminal diameter curve along the RCA, where the x-axis corresponds to the x-axis of the MPR view of 1101-section B and the centerline of the RCA. Finally, 1101-section D shows the FFR values ​​along the centerline points in the RCA, where the x-axis is the same as the x-axis of the MPR view and the diameter / area curve. In addition, in 1102, the FFR values ​​along the centerline of the volume-rendered RCA are used to color-code the corresponding segments of the volume-rendered RCA. Note that the vertical markers in 1101-part C, D correspond to the minimum area / diameter (bold markers) and degree of obstruction (dashed markers) derived from common quantitative coronary analysis techniques (e.g., as depicted by Girasis C et al. in “Advanced three-dimensional quantitative coronary angiographic assessment of bifurcation lesions: methodology and phantom validation” (Euro Intervention 2013; 8: 1451-1460)).

[0145] For training Fig.10 The stenosis assessment network shown outputs an FFR value for each centerline point, and a reference FFR value (209) can be measured along the coronary vessel of interest, thereby generating an invasively measured fractional flow reserve value at each location along the coronary centerline. This can be obtained by performing a manual or motorized pullback during the measurement of the fractional flow reserve. In the catheterization laboratory, an interventional cardiologist or physician places the FFR line at a distal location within the coronary artery of interest. During the automatic or manual pullback, the FFR value is continuously measured until the FFR line reaches the coronary artery opening (Sonck et al., "Motorized fractional flow reserve pullback: Accuracy and reproducibility", Catheter Cardiovasc Interv. 2020 Sep 1; 96(3): E230-E237).

[0146] Optionally, to ensure that the fractional flow reserve values ​​along the coronary arteries as measured in the catheterization laboratory (eg, the pullback FFR reference values) are aligned with the spatial coordinates of the MPR images, co-registration may be performed. This may be done, for example, by Figure 2 The method described in step 208 is performed.

[0147] Optionally, where x-ray angiography image data is available for training, x-ray angiography can be used to calculate a reference FFR value for each centerline point based on 3D coronary artery reconstruction, for example, as taught by Bouwman et al. in US11083377B2 (Method and apparatus for quantitative hemodynamic flow analysis). Bouwman et al. described a method for calculating vFFR pullback along the coronary artery of interest based on three-dimensional coronary artery reconstruction. As described by Masdjedi et al. in "Validation of a three-dimensional quantitative coronary angiography-based software to calculate fractional flow reserve: the FAST study", Euro Intervention, 2020 Sep 18; 16(7): 591-599, due to the high spatial resolution of x-ray angiography, the accuracy of the calculated vFFR is quite high, wherein the vFFR values ​​and pullback vFFR values ​​are obtained using CAAS workstation 8.0 (Pie Medical Imaging, The Netherlands). In an optional embodiment, a reference standard ( Figure 2 209 or Fig.16 1609) is obtained by calculation of FFR pullback or calculation of distal FFR value, thereby eliminating the need to invasively measure hemodynamic parameters (such as, for example, FFR). This can be performed in the following manner: using the patient's x-ray angiography image data and calculating the (pullback) FFR value by using, for example, the vFFR (vascular-FFR) workflow within the CAAS workstation.

[0148] The CAAS workstation's vFFR method uses two x-ray angiograms taken at least 30 degrees apart to generate a 3D coronary reconstruction. vFFR is calculated on the fly by utilizing a proprietary algorithm that combines the morphology of the 3D coronary reconstruction with conventionally measured patient-specific aortic pressure. Fig.12 An example of obtaining a calculated FFR pullback of the coronary circumflex artery by using a CAAS workstation is shown. 1201 shows the segmentation of the coronary circumflex artery in each 2D X-ray angiography image, thereby obtaining a 3D reconstruction of the coronary artery (1202). Curve 1203 shows the vFFR value calculated along the length of the 3D reconstructed coronary artery. The same method can also be performed on CCTA data sets, thereby eliminating the need for corresponding x-ray angiography image data for each patient.

[0149] Extension 2: FFR values ​​and additional arterial characteristics for each centerline point.

[0150] This part uses Figure 2 Another extension of the described workflow method extends Extension 1, which can be Figure 3 Specifically, the narrow evaluation network ( Fig. 9 The architecture of the stenosis assessment network (SEN) takes five different arterial properties as input. Three of the five arterial properties (luminal area, luminal attenuation, and calcification area) are derived from the MPR image of the vessel of interest (in step 203), and two of the five arterial properties (side branches, bifurcations) are derived from the extracted coronary artery centerline tree (step 205). Figure 2 The described method is not limited to the described vessel characteristics and can be easily extended to additional vessel characteristics derived from analysis of MPR images, analysis of image data of vessels of interest, or analysis of coronary artery centerline trees, and can also be extended to other characteristics describing the vessels of interest. For example, data representing the soft plaque area and mixed plaque area of ​​the vessel of interest (which can be derived using the method described by Isgum et al. in US10699407B2 (Method and system for assessing vessel obstruction based on machine learning)) can be used as data (signals) characterizing the characteristics of the vessel of interest for input to the stenosis assessment network. In summary, Isgum et al. (US10699407B2) first extracted the centerlines of the coronary arteries within the CCTA images. These were used to reconstruct stretched multiplanar reformatted images of the coronary arteries. In order to perform the automatic analysis, a multi-task recursive convolutional neural network was applied to the coronary multiplanar reformatted images to perform two multi-class classification tasks simultaneously. In the first task, the network detected and characterized the type of coronary plaque (plaque-free, non-calcified, mixed, calcified). In the second task, the network detected and determined the anatomical significance of coronary artery stenosis (no stenosis, insignificant (ie, lumen narrowing <50%), significant (ie, lumen narrowing ≥50%)) and / or functional coronary artery lesion severity.

[0151] Fig.13 Provides the ability to integrate data describing the soft plaque area and mixed plaque area of ​​a vessel of interest into Fig.10 An example of the architecture of a deep learning network. Fig.13In the embodiment of the present invention, the soft plaque area (1301) and the mixed plaque area (1302) are added as the characteristics of the vessel of interest. Optionally, the data describing the soft plaque area and the mixed plaque area of ​​the vessel of interest may be pre-encoded before passing through the encoder. In addition, the data describing one or more characteristics of the vessel of interest may be a binary signal of the MPR image along the vessel of interest, which means one when soft plaque or mixed plaque exists, and means zero when soft plaque and mixed plaque do not exist.

[0152] Additionally or alternatively, other data (signals) describing the characteristics of the vessel of interest may be integrated into Figure 2 into the architecture of the deep learning network. For example, data (signals) related to the image quality of the MPR view of the vessel of interest may be integrated into the architecture of the deep learning network. For example, such data (signals) may indicate the presence of staircase artifacts, and / or represent the noise level within the MPR view. In another example, data (signals) related to the relative luminal attenuation of the vessel of interest (compared to another main coronary vessel) may be integrated into the architecture of the deep learning network. For example, in the case where the vessel of interest represents the RCA, the data (signals) may represent the relative attenuation of the RCA compared to the LAD and / or compared to the LCX. This signal will be related to the flow difference between the main coronary arteries, which is expected to exist in the case of functionally significant flow reduction due to epicardial obstruction and / or (local) microvascular disease.

[0153] Extension 3: Including myocardial properties

[0154] Isgum et al. in US10176575B2 (Method and system for assessing vessel obstruction based on machine learning) recognize that CCTA image acquisition is typically initiated once a predetermined threshold attenuation value is reached in a predetermined anatomical structure (most commonly this involves the descending aorta), or by waiting a specific delay time after enhancement is first visible in the ascending aorta. This has the following effect: once the injected contrast medium is present in the coronary arteries, the injected contrast medium will also be transported to successively smaller generations of coronary arterioles (from which the injected contrast medium passes into the coronary microvascular system), which will enhance the myocardium. When functionally significant coronary artery stenosis causes ventricular myocardial ischemia, due to the acquisition characteristics of CCTA described above, there are differences in myocardial texture characteristics between normal and ischemic portions of the myocardium at the time of CCTA image acquisition. Isgum et al. describe a method for detecting the presence of functionally significant stenosis in one or more coronary arteries based on machine learning using only myocardial features. In summary, Isgum et al. (US10176575B2) first segmented the myocardium of CCTA images. Then, from the segmented myocardium, the encoding was extracted in an unsupervised manner using a convolutional autoencoder and used to calculate features ([f1, f2, f3, …, fn]). The convolutional autoencoder contains two parts, namely the encoder and the decoder. The encoder compresses the data into a lower dimensional representation through convolutional layers and maximum pooling layers. The decoder expands the compressed form through deconvolution layers and upsampling layers to reconstruct the input data. In order to represent the entire myocardium, the statistics of the encodings of all voxels within the myocardium are used as features. Finally, based on the extracted features, a support vector machine is used to classify the patients as patients with functionally significant coronary artery stenosis or patients without functionally significant coronary artery stenosis.

[0155] This section describes how Figure 2 or Fig.17 Another extension of the method described by the flowchart of Figure 3 Specifically, the module 304 is implemented by Figure 2 Box 206 or Fig.17 The architecture of the stenosis assessment network described in block 1706 may be adapted to employ data characterizing the myocardium of the heart.

[0156] exist Fig.14In the present invention, an example of the integration of properties of the myocardium of the heart is provided, which are extracted from CCTA image data by, for example, using the method described in US10176575B2. Data representing such properties of the myocardium are used as input to an encoder of a stenosis assessment network (206 or 1706). For example, a feature vector (1401) obtained from the myocardial analysis described in US10176575B2 can be provided as input to the encoder ( Fig. 9 904 or Fig. 20 2003). This can be performed by treating the feature vector as an additional input to the encoder (1402), or by concatenating the feature vector with the result of the encoder just before one of the dense layers of the classification head (1403), or by concatenating the myocardial feature vector produced by the convolutional encoder with the other input data (1403).

[0157] Note that arterial properties as described by Extension 2 above can also be used as "extra information" within the method as described by US10176575B2 (from US10176575B2 Fig.15 155).

[0158] Alternatively, instead of integrating the feature vectors from the myocardial analysis, the FFR classification results as described by US10176575B2 may be integrated into Fig.14 In this case, the FFR myocardial classification (e.g., from US10176575B2 Fig.15 The result of block 158) can be directly cascaded to Fig.14 The myocardial feature vector as described by US10176575B2 is based on the complete myocardium (e.g., Fig.15 In order to focus the myocardial characteristics on the area covered by the vessel of interest, the method described in US10176575B2 can be adapted. Two possible methods are described below.

[0159] Method 1:

[0160] This can be done by, for example, using Fig.28 The method of the flowchart applies the Voronoi algorithm (Guibas L et al., "Primitives for the manipulation of general subdivisions and the computations of Voronoi diagrams", ACM Trans Graph 4: 74-123 1985) on the extracted axial trajectory of the vessel of interest to define the myocardial area covered by the vessel of interest. Fig.15An example of subdividing the myocardium of the heart into regions covered by the coronary arteries is provided. In this example, the vessel of interest is identified by the black artery (1501, the second LAD diagonal identified by the black arrow), and the myocardial region covering the artery is identified by the pink region (1502, identified by the gray arrow). In the event that the vessel of interest covers more than one myocardial region identified by the Voronoi algorithm, those multiple regions may be merged into one region.

[0161] In this method, the calculation of the feature vector as described by US 10176575 B2 is limited to a defined area and as described, for example, by Fig.16 As shown, the obtained myocardial feature vector is integrated into the stenosis assessment network (206). Fig.16 In the example, the myocardial feature vector is considered as an additional input signal and after being pre-encoded using the convolutional network (1606), the additional code is fed to the encoder. To calculate the myocardial features, for each centerline point along the vessel of interest (1605 or Fig. 22 2201), from the corresponding myocardial area (1602 or Fig. 22 2202) aggregate features. For example, by interpreting the corresponding myocardial region as a single cluster and using the standard deviation of all features from all voxels. Alternatively, by interpreting the corresponding myocardial region as multiple clusters and using the standard deviation to generate a feature vector for each cluster, and then using the maximum value of all clusters to create one feature vector for the corresponding myocardial region. Another alternative is to define the corresponding myocardial region by the portion of the myocardium perfused to a particular centerline point by all downstream segments, and repeat this operation for all centerline points within the vessel of interest.

[0162] Method 2:

[0163] In this second approach, feature vector computation as described by US10176575B2 may be performed in small regions at or around each centerline location along the vessel of interest, and the resulting feature vectors may be used as additional arterial characteristics input to the encoder of the stenosis assessment network. Alternatively, a feature vector computation as described with reference to US10699407B2 may be performed. Fig.15 The eigenvectors described are calculated and the resulting eigenvectors can be used as input to Fig.16 Additional arterial characteristics of the encoder of the stenosis assessment network shown (1605 or Fig. 22 Optionally, a precoder (1606 or 2201) may be applied to such (one or more) new arterial characteristics. Fig. 22 of 2202).

[0164] Myocardial ischemia occurs when blood flow to the heart muscle (myocardium) is blocked by a coronary artery that is partially or completely blocked due to plaque buildup (atherosclerosis). This often results in chest pain (angina) that the patient experiences. Up to half of patients who undergo selective coronary angiography to investigate chest pain do not show signs of obstructive coronary artery disease. These patients are often discharged with a diagnosis of non-cardiac chest pain, but many may have an ischemic basis for their symptoms. This type of ischemic chest pain in the absence of obstructive coronary artery disease is called INOCA (ischemia in non-obstructive coronary arteries). INOCA involves a mismatch between the supply and demand of myocardial oxygen caused by microvascular dysfunction. Microvascular dysfunction involves dysfunction of the small blood vessels that supply the myocardium and is more common in women, especially during middle age. INOCA can also be caused by vasospastic disorders (caused by spasm of the coronary arteries).

[0165] An advantage of the method described by Isgum et al. in US10176575B2 (Method and system for assessing vessel obstruction based on machine learning) is that obstructive coronary ischemia as well as non-obstructive coronary ischemia can be identified. However, since treatment strategies differ between obstructive coronary ischemia and non-obstructive coronary ischemia, there is a need to identify the difference from a patient treatment perspective.

[0166] By integrating methods such as described by US10176575B2, identification of microvascular dysfunction can be integrated into the stenosis assessment network.

[0167] Since INOCA is a microvascular dysfunction without epicardial coronary artery obstruction, it is not possible to predict the severity of INOCA in deep learning networks (e.g., Fig. 9 , Fig.10 , Fig.13 , Fig.14 , Fig.16 , Fig. 20 , Fig.21 or Fig. 22 In cases where a functionally significant blood flow obstruction is identified by the output of the provided network, INOCA can be identified by examining the vessel of interest. This can be done by:

[0168] · Examine the arterial properties generated by the “arterial characterization” network ( Figure 6 or Figure 8 or Fig.19a and Fig.19b ). For example, QCA analysis is performed on the lumen area curve. If, for example, the obstruction severity is > 50%, it is classified as obstructive coronary artery disease.

[0169] · Performing a QCA analysis on the segmentation of the coronary artery tree, for example, by the method described by Wolterink et al. in "Graph convolutional networks for coronary artery segmentation in cardiac CTangiography" (in International Workshop on Graph Learning in Medical Imaging. Springer, Cham, 2019). Wherein, for example, the QCA analysis is performed by the method described by Hof et al. in WO2012028190A1 (Method and apparatus for quantitative analysis of a tree of recursively splitting tubular organs). If, for example, a blockage severity of >50% is identified in the coronary artery tree, this is classified as obstructive coronary artery disease.

[0170] · Combination of the QCA analysis as described above with the amount of coronary plaque obtained by extension 2, or by the following methods: as described by Isgum et al. in US10699407B2, or as taught by Wolterink et al. in "Automatic Coronary Artery Calcium Scoring in CardiacCT Angiography Using Paired Convolutional Neural Networks" (Medical Image Analysis, 2016), or for example by Dey et al. in "Automated 3-dimensional quantification of noncalcified and calcified coronary plaque from coronary CTangiography" (Cardiovasc Comput Tomogr. 2009, 3(6):372-382).

[0171] ·exist Fig. 9 , Fig.10 , Fig.13 , Fig.14 , Fig.16 , Fig. 20 , Fig.21 or Fig. 22This may involve adding another “classification INOCA head” similar to the “classification head” that is trained to output a value characterizing microvascular dysfunction, e.g., IMR

[0172] (microcirculatory resistance index) and / or coronary flow reserve.

[0173] Optionally, the method as described by US10176575B2 (and integrated into a deep learning network as described herein) can be improved by including a CT calcium scan. In contrast to a CCTA scan, a CT calcium scan is acquired without the injection of any contrast medium. Combining a CT calcium scan provides information of the myocardium without the presence of any contrast fluid, thereby producing a "baseline myocardium". By combining a CT calcium scan, the machine learning network is able to integrate myocardial intensity without any contrast enhancement, thereby improving the detection of subtle contrast changes between healthy myocardial areas and ischemic myocardial areas. After registration of the CT calcium scan to the CCTA scan, both image data sets can be used for the method as described by US10176575B2.

[0174] Fig.17 Another embodiment of the present application is described. Obviously, the steps described here can be performed in any logical order and can be partially omitted. Fig.17 A flow chart of a machine learning-based approach for determining functionally significant lesion severity along the axial trajectory of a vessel of interest is shown. The machine learning-based approach is similar to the reference method, although there are some basic differences. Figure 2 Similar to the method described in Extension 2 “FFR value and additional arterial characteristics at each centerline point”. In order to be able to quickly and quantitatively assess the distribution of CAD, a deep learning-based approach is used to predict FFR pullback given the MPR image of the vessel of interest as input. The method consists of two stages and is Fig.18Shown at a high level. First, an axial trajectory (e.g., centerline) of a vessel of interest is extracted from an input CCTA image (1801), and an MPR image is generated from the axial trajectory of the vessel of interest and the CCTA image data of the vessel of interest. In the first stage of the deep learning-based method, the vessel of interest is characterized along the length of the MPR image (1802). In an embodiment, this first stage may combine unsupervised learning and supervised learning. Unsupervised learning provides for the extraction of non-handcrafted features from the MPR image. Supervised learning explicitly incorporates clinical knowledge into the machine learning model. Subsequently, features characterizing the vessel of interest derived from both the supervised learning method and the unsupervised learning method are fed to a deep learning network (1803) to predict the FFR drop along the artery in a second stage (1804). In order to enable the distinction between focal CAD and diffuse CAD, it is important that the prediction of the FFR drop along the axial trajectory of the vessel of interest is accurate. This is documented by supervising the FFR pullback prediction with an FFR pullback reference (1709, Fig.17 ). To this end, a novel loss function inspired by the Earth Mover's Distance (EMD) is used. Since the registration of the reference FFR signal with the MPR image is challenging, mismatches cannot be excluded. Therefore, if the overlap between the reference FFR drop and the true FFR drop in the input artery is low, directly using a loss function such as the mean absolute error (MAE) evaluating all vessel points separately may lead to an ambiguous loss signal. In contrast, the proposed loss function increases continuously with the distance between the predicted FFR drop and the reference FFR drop. While the EMD loss concentrates the FFR drop around the assumed lesion location, the uncertainty in the correct lesion location may lead to a tendency to underestimate the steepness of the FFR drop. Since distinguishing between focal CAD and diffuse CAD requires a correct prediction of the steepness of the FFR pullback, an additional loss function is designed to penalize the histogram of the FFR drop. The reference FFR drop is converted into a vector graph using the EMD loss. Fig.17 A deep learning-based approach for predicting FFR drawdowns is described in detail.

[0175] In a first step 1701, a CCTA image dataset of a vessel of interest is obtained. Such a CCTA image dataset represents a volumetric CCTA image dataset (e.g., a single contrast-enhanced CCTA dataset) and is related to Figure 2 The description of step 201 is the same as that of step 201.

[0176] In step 1702, an axial trajectory extending along the vessel of interest is extracted, and this step is similar to Figure 2 The description of step 202 is the same as that of step 202.

[0177] In step 1703, a three-dimensional (3D) multi-planar reformatted (MPR) image of the vessel of interest is created and this step is similar to Figure 2 The description of step 203 is the same as that of step 203.

[0178] In steps 1704 and 1705, the first stage of the deep learning based method is adopted. In this first stage, the features of the blood vessels of interest can be extracted through a combination of unsupervised learning and supervised learning.

[0179] In an embodiment, this first stage employs an artery characterization network (e.g., Fig.17 1704 and Fig.18 1802), which uses unsupervised machine learning to characterize the characteristics of the blood vessel of interest given the MPR image of step 1703 as input.

[0180] In an embodiment, an artery characterization network (e.g., Fig.17 1704 and Fig.18 1802) can employ a variational autoencoder (VAE), which can be configured to extract non-handcrafted features from MPR images. A variational autoencoder (VAE) is a generative model that approximates the data generating distribution as described by Kingma et al. in "Auto-encoding variationalbayes" (arXiv arXiv:1312.6114, 2013). Through approximation and compression, the resulting model captures the underlying data manifold; a constrained, smooth, continuous, lower-dimensional latent (feature) space in which the data is distributed (Kingma et al., "Semi-supervised learning with deep generative models", Advances in neural information processing systems, 2014, pp. 3581–3589). VAE implements latent features using independent normal distributions, thereby increasing the interpretability and compactness of the latent space relative to conventional convolutional autoencoders. These beneficial properties of the latent space are used within a machine learning-based artery characterization network as described by this step.

[0181] A typical VAE consists of two main parts, the encoder and the decoder. The encoder compresses (encodes) the data into a lower dimensional latent space through convolution operations and downsampling (max pooling), and then expands (decodes) the compressed form to reconstruct the input data through deconvolution operations and upsampling (up pooling). Training the VAE while minimizing the distance loss between the encoder input and the decoder output ensures that the abstract encoding generated from the input contains enough information to reconstruct it with low error. Once the VAE is trained, the decoder is removed and the encoder is used to generate encodings for unseen data.

[0182] Fig.19a and Fig.19b An example of the architecture of a VAE network for characterizing features of a vessel of interest given an MPR image (from step 1703) as input is shown. The VAE network includes a convolutional encoder (1902) configured to compute a plurality of features (z) from a stack of a predetermined number of consecutive cross-sectional slices (e.g., thirteen cross-sectional slices of the input MPR image (1901)). The convolutional encoder (1902) may include convolutional blocks, which may include, for example, two convolutional layers, batch normalization, and ReLU activations. The second convolutional layer in each block may have a stride of, for example, 2, which enables downsampling of the resolution. To increase the amount of contextual information, the output of the network after the second and fourth convolutional blocks may be cascaded ("fused") with the output of the same network applied to, for example, 6 and 3 adjacent MPR slices, respectively, which increases the receptive field in the z direction. In addition, bifurcation information (1903) of the vessel of interest may be injected into the convolutional encoder, for example, before the first, third, and after the last convolutional block. In an embodiment, the bifurcation information may be a bifurcation feature map indicating whether the MPR slice is located at a bifurcation, and the bifurcation feature map may be injected into the corresponding convolution block through cascading. Figure 2 The method described in step 204 is performed by Fig.17 Box 1705 generates or exports and provides fork information.

[0183] In order to configure the encoder (1902) to extract relevant information from an arbitrary input MPR image (1901) into a code representing the distribution of latent features in the input MPR image, the encoder (1902) and the primary decoder (1904) are trained to reconstruct the central input slice of the input stack according to the code (z) output by the encoder (1902). Like the encoder (1902), the primary decoder (1904) consists of alternating convolutional blocks and pooling operations. However, for the primary decoder (1904), the last convolutional layer in each block is transposed to enable upsampling. In addition, only a single slice is reconstructed instead of the entire input stack.

[0184] To enhance the machine learning network to extract features characterizing the geometry of arteries and plaques, the auxiliary decoders (1905a, 1905b) are configured to: process the encoding (or a portion thereof) output by the encoder (1902) to predict segmentation masks for lumen characteristics and calcified plaque characteristics and non-calcified plaque characteristics. The auxiliary decoder 1905a can be implemented by a linear layer that processes the encoding (z) output by the encoder (1902) by regression to output feature data (1906) characterizing the lumen area of ​​the vessel of interest on the axial trajectory of the vessel of interest. The auxiliary decoder 1905b can have an architecture similar to the main decoder (1904) and has a difference in the last layer, the last layer has four channels, and uses softmax activation to output feature data characterizing the lumen attenuation of the vessel of interest on the axial trajectory of the vessel of interest and calcified plaque and non-calcified plaque. Optionally, the auxiliary decoder (1905a, 1905b) may implement a plurality of linear layers configured to extract feature data characterizing arterial and plaque geometry from the codes (z) output by the encoder (1902).

[0185] In addition, the encoder (1902) may be configured to generate a predetermined number (e.g., 32) of predicted means and standard deviations (1907) of the codes (z) output by the encoder (1902). The codes (z) or a specific subset thereof output by the predicted means and standard deviations (1907) may be supplied as input to the auxiliary decoders (1905a, 1905b) to enable the auxiliary decoders (1905a, 1905b) to extract relevant feature data from the codes (z). The codes (z) output by the predicted means and standard deviations (1907) and the regressed lumen area (1906) from the feature data output by the auxiliary decoder (1905a) may be supplied to a machine learning based FFR pullback network ( Fig.17 1706 Fig. 20 2008). Optionally, the feature data output by the auxiliary decoder (1905b) may also be supplied to a machine learning based FFR pullback network ( Fig.17 1706 Fig. 20 of 2008).

[0186] Fig.19a and Fig.19b The VAE provides a machine learning-based artery representation network ( Fig.17 1704), a machine learning-based arterial characterization network combines supervised and unsupervised features extracted from MPR images and from the coronary artery tree. Fig.19b The unsupervised and supervised parts of the VAE are also shown. Fig.19a and Fig.19bBoth show the VAE architecture of the artery characterization network based on machine learning. A stack of a predetermined number of slices around a central MPR slice at a specific position along the centerline of the vessel of interest is encoded using a convolutional encoder (1902). The encoder (1902) consists of, for example, 5 convolution blocks, each of which includes 2 convolutions. The bifurcation information (1903) can be injected into the convolution block of the encoder (1902) as described above. At the output of the encoder (1902), the latent vector encoding (z) is sampled from the predicted feature vectors μ (mean) and σ (standard deviation). The auxiliary decoder 1905a receives the latent vector encoding (z) as input and directly regresses the lumen area of ​​the vessel of interest on the axial trajectory of the vessel of interest. The auxiliary decoder 1905b receives only a subset of the latent vector encoding (z) as input and reconstructs the segmentation of the lumen attenuation of the vessel of interest on the axial trajectory of the vessel of interest and the segmentation of the calcified plaque characteristics and the non-calcified plaque characteristics. The primary decoder (1904) and the auxiliary decoder (1905b) are disconnected during processing of unseen data.

[0187] To train the supervised part of the VAE ( Fig.19b 1951), the reference value ( Fig.171708). The reference values ​​may be obtained from a database containing data of a plurality of patients. Each set within the database contains, for each patient, a) a contrast-enhanced CT image dataset (1701 represents a reference image set during the training phase) and corresponding b) arterial property reference values. For arterial properties, reference annotations of the coronary artery lumen and plaque (calcified, non-calcified). These reference annotations may be created by manual segmentation of the CCTA dataset used during training, or by automatic segmentation of the lumen and plaque components of the CCTA dataset used during training followed by manual correction if necessary. Automatic segmentation of lumen and plaque components can be performed, for example, by segmenting in the original CT image volume using methods such as those described by Wolterink et al., "Automatic coronary artery calcium scoring in cardiac CT angiography using paired convolutional neural networks" (Medical Image Analysis 34, 123-136 (2016)) and Wolterink et al., "Graph Convolutional Networks for Coronary Artery Segmentation in Cardiac CT Angiography" (In: Zhang D, Zhou L, Jie B, Liu M, ed. Graphical Learning in Medical Imaging. Lecture Notes in Computer Science. Springer International Publishing (2019), pp. 62-9). Thereafter, the automatic segmentation is transformed into an MPR image of the artery, visually inspected and corrected when necessary. Since X-ray angiography is superior in image resolution compared to CCTA, in a preferred embodiment, lumen segmentation on the MPR image is guided by QCA3D analysis results extracted from X-ray angiography image data from the same patient, which was acquired within three months of the CCTA data of the patient in question. QCA3D can be performed, for example, using a method described by Girasis C et al. in "Advanced three-dimensional quantitative coronary angiographic assessment of bifurcation lesions: methodology and phantom validation" (EuroIntervention 2013; 8: 1451-1460) on a CAAS workstation (Pie Medical Imaging BV, the Netherlands).

[0188] To ensure that the reference values ​​1708 are aligned with the spatial coordinates of the MPR image (1703), the reference values ​​may be transformed into the MPR image domain. This step may be skipped in the case where the reference values ​​(e.g., annotations of lumen and plaque type) are obtained by using the MPR image as a result from step 1703. When the reference values ​​are obtained, for example, by using annotations of a contrast-enhanced CT dataset (step 1701), this step transforms the annotations into an MPR view. Such a transformation is performed by using the extracted vessel trajectory (as a result of step 1702).

[0189] In an embodiment, when training a machine learning-based artery characterization network ( Fig.17 1704 and Fig.18 During the process of 1802), three separate loss functions can be used. The mean absolute error (MAE) loss function can be used to minimize the loss in the MPR reconstruction loss (1904). Similarly, the MAE loss function can be used to supervise the regression of lumen features (1906). For the segmentation loss (1905), a binary cross entropy loss function can be used.

[0190] In an embodiment, the latent space encoding (z) of the VAE may be regularized using the Kullback-Leibler divergence to promote a dense latent space and disentangled latent features. The Kullback-Leibler divergence metric is a statistical measure from information theory that is often used to quantify the difference between one probability distribution and a reference probability distribution. It is an asymmetric measure that measures the relative entropy or difference in the information represented by two distributions. It can be thought of as measuring the distance between two data distributions, thereby showing how different the two distributions are from each other. The Kullback-Leibler divergence penalizes the difference between a normal distribution parameterized by a predicted mean and standard deviation and a standard normal distribution.

[0191] In a VAE, the output of the encoder (1907) can generate a latent space encoding (z) by predicting the mean and standard deviation for each feature, and then drawing random samples from a normal distribution parameterized by the mean and standard deviation. In the absence of any additional losses, a network with such resampling will always predict a standard deviation of 0, because this minimizes the randomness of the prediction and thus maximizes the information transmitted. However, similar to a conventional convolutional autoencoder, this will result in a latent space with gaps (regions with outputs that do not look like real coronary arteries) and entangled (correlated and meaningless) latent features. In contrast, to promote a dense latent space and disentangled latent features, the latent space of the VAE can be regularized using the Kullback-Leibler divergence (KLD, a measure of the difference between distributions) between a normal distribution parameterized by the predicted mean and standard deviation and a standard normal distribution. Therefore, the network must decide how much accuracy is required for each latent, thereby balancing the reconstruction loss and regularization. Compared to a conventional autoencoder, this has the following advantages:

[0192] Intuitively, the sampling process combined with KLD promotes the densification of the latent space by enforcing that all points close to the predicted mean also produce approximately correct outputs.

[0193] By sampling the latent space independently, features are encouraged to be uncorrelated, which tends to produce features that are more meaningful and thus more suitable for our narrow evaluation.

[0194] Optionally, the auxiliary output decoder (1905b) of the VAE can remain connected during processing of unseen data, and the predicted lumen, calcified plaque and non-calcified locations can be used to calculate geometric parameters from the vessel of interest by using quantitative coronary artery analysis (QCA). First, a 3D model is created in the spatial coordinate system of the MPR image (1703) or in the spatial coordinate system of the CT image (1701) from the segmented regions (the results of the auxiliary output decoder (1905) of the VAE) (such as the vessel lumen, (calcified and non-calcified) vessel plaques). Next, based on the 3D model, the anatomical results are calculated, for example using a method as described by Girasis C et al. in "Advanced three-dimensional quantitative coronary angiographic assessment of bifurcation lesions: methodology and phantom validation" (EuroIntervention 2013; 8: 1451-1460). Examples of such quantitative anatomical results are length, equivalent diameter along the axial trajectory of the vessel of interest, cross-sectional area along the axial trajectory of the vessel of interest, occlusion length, minimum equivalent diameter, minimum luminal area, percent diameter stenosis, percent area stenosis, reference diameter / area, vessel volume, (calcified, non-calcified) plaque volume, plaque burden (plaque volume / vessel volume). As described by Gronenschild E et al. in "CAAS II: A Second Generation system for Off-Line and On-Line Quantitative Coronary Angiography", (Cardiovascular Diagnosis 1994; 33: 61-75), a healthy reference diameter or area curve (representing the diameter / area in the case of healthy vessels) is calculated, for example, by fitting a line through the diameter values ​​or area values ​​along the axial trajectory of the vessel of interest, wherein the diameter values ​​or area values ​​within the range of the lesion are excluded during the fitting.

[0195] Optionally, as by Figure 2 The method of step 204 is described and referred to Figure 8 To explain further, vessel properties (such as lumen attenuation, lumen area, calcification area, soft plaque area, and mixed plaque area) may be extracted along the axial trajectory of the vessel of interest.

[0196] In an embodiment, given the MPR image of step 1703 as input, this first stage (1704) of the deep learning based method may also employ supervised machine learning to characterize the features of the vessel of interest. In order to take into account the effects of the geometry and local appearance of the artery on blood flow, additional features may be defined. Specifically, Fig.17 In step 1705, the axial trajectory (eg, centerline) of the blood vessel of interest may be obtained from the CCTA image data set of step 1701 (eg, from the Fig.17 Additional data (signals) characterizing features of the vessel of interest may be extracted from the axial trajectory of the vessel of interest (obtained in step 1702). For example, for each point along the axial trajectory of the vessel of interest, two or more additional features may be extracted. One of the additional features may indicate the presence of a bifurcation at a centerline point of the artery. Another additional feature may indicate whether the centerline point belongs to a main branch (i.e., left main (LM), LAD, LCX, RCA) or a side branch. Other examples of additional features characterizing the vessel of interest may be the presence of a stent (from a previous coronary intervention) at the centerline point of the vessel and / or the type of stent.

[0197] The second phase of deep learning-based methods ( Fig.17 Step 1706 and Fig.18 1803) uses a machine learning based FFR pullback network, which is configured to characterize the FFR pullback along the axial trajectory of the vessel of interest given the characteristics of the vessel of interest (1704) and the coronary artery tree characteristics (1705) output from the artery characterization network.

[0198] In an embodiment, a machine learning based FFR pullback network may utilize a CNN network architecture as described herein. Fig. 20An example CNN network architecture for characterizing FFR pullback along an axial trajectory of a vessel of interest is shown. The input signal is represented by 2001. In order to detrend the lumen area signal, the percentage difference relative to the value at the previous position is calculated at each position along the axial trajectory of the vessel of interest. The calculated lumen area percentage difference signal and the VAE encoding are pre-encoded (2002) respectively to extract independent features from the local environment. For the lumen area signal, multiple convolutional layers (e.g., four) with increasing levels of dilation for expanding the receptive field are applied to include the entire vessel of interest. The VAE encoding is preferably pre-encoded with a smaller number of convolutional layers (e.g., two) to prevent overfitting. Thereafter, we concatenate the resulting features with the remaining features (2003) and feed the resulting encoding into a common convolutional path for regression of FFR pullback (2004), which includes convolutional layers, average pooling, and ReLU activation functions. The FFR reduction regression network (2004) is inspired by the additive nature of sequential flow resistance. The FFR drop regression network predicts FFR pullback by first predicting the FFR drop for each point in the artery of interest (2005). In this way, the prediction target at each point is independent of the previous output. Subsequently, the FFR pullback is calculated by summing the predicted FFR drops (2006). In order to obtain an arterial level FFR prediction from the predicted FFR drops (2007), the minimum value of the FFR pullback is taken, which is the same as the most distal value. The FFR pullback prediction network uses a kernel size of, for example, three for all convolutions with zero padding to maintain feature size. In addition, we use, for example, 16 filter maps to balance expressiveness and overfitting. In order to account for potential misalignment with the reference FFR drop at each point, we use average pooling. To further prevent overfitting, we set the dropout probability to, for example, 0.5 and use sample normalization throughout the FFR drop regression network.

[0199] In order to train the FFR pullback network based on machine learning ( Fig.17 1706 and Fig.18 1803), using the reference standard ( Fig.171709). The reference standard may be provided from a database containing data of a plurality of patients. Each set within the database contains, for each patient, a) a contrast-enhanced CT image dataset (1701 representing a reference image set during a training phase) and corresponding b) CAD-related reference values, the CAD-related reference values ​​representing hemodynamic indices for assessing functionally significant coronary artery obstruction. For example, the CAD-related reference values ​​may represent at least one invasively measured fractional flow reserve, coronary blood flow reserve, instantaneous wave-free ratio, resting full cycle ratio, diastolic-free hyperemia ratio, diastolic pressure ratio, resting Pd / Pa ratio, hyperemic myocardial perfusion, microcirculatory resistance index, pressure drop along the coronary arteries, and fractional flow reserve along the coronary arteries. Note that the reference standard (1709) and the arterial reference values ​​(1708) require the same (reference) contrast-enhanced CT image dataset.

[0200] In cases where the prediction of FFR along the axial trajectory of the vessel of interest is of interest, the reference standard needs to represent the FFR along the axial trajectory of the vessel of interest. This can be obtained from a manual or motorized invasive FFR pullback. In the catheterization laboratory, an interventional cardiologist or physician places the FFR wire at a distal position within the coronary artery of interest. During the automatic or manual pullback, the FFR value is measured continuously until the FFR wire reaches the coronary artery opening (Sonck et al. "Motorized fractional flow reserve pullback: Accuracy and reproducibility", Catheter Cardiovasc Interv. 2020 Sep 1; 96(3): E230-E237). Alternatively, where x-ray angiography image data is available for training, reference FFR values ​​for each centerline point value may be calculated based on 3D coronary reconstruction using x-ray angiography, for example, as taught by Bouwman et al. in US11083377B2 (Method and apparatus for quantitative hemodynamic flow analysis), and as further described previously in the description of Extension 1 of the current patent application. Bouwman et al. describe a method for calculating vFFR pullback along the coronary artery of interest based on three-dimensional coronary reconstruction. Where the CAD-related reference value represents a non-congestive index (e.g., instantaneous wave-free ratio, resting full cycle ratio, diastolic non-congestive ratio, diastolic pressure ratio, resting Pd / Pa ratio), the output of the FFR regression also represents such a non-congestive index.

[0201] In an embodiment, in a machine learning based FFR pullback network ( Fig.17 1706 and Fig.181803), the prediction of FFR drop can be supervised by the reference FFR drop along the axial trajectory of the vessel of interest. To this end, a novel loss function inspired by the bulldozer distance (EMD) and introducing the so-called EMD loss can be used. Instead of comparing two probability distributions locally, EMD is a way of measuring the global similarity between distributions. Intuitively, EMD is the minimum amount of "work" required to transform one distribution into another. Thus, "work" is defined as the amount of probability mass that needs to be moved multiplied by the distance it needs to move. Therefore, unlike point-by-point comparisons like the mean absolute error (MAE), this loss function increases with the distance between the predicted FFR drop and the reference FFR drop. In the case of misalignment between the predicted pullback curve and the reference pullback curve, this may produce improved gradient updates. In a single dimension, for i∈[0,l] via x i The EMD between the discretized two probability distributions p1 and p2 is calculated as:

[0202]

[0203] where l represents the length of the vessel of interest and i is the running index.

[0204] The formula in Equation 2 is used to calculate the loss between the predicted FFR drop and the reference FFR drop. Intuitively, this calculation corresponds to the cumulative difference between the FFR curves:

[0205]

[0206] However, since the calculation of FFR based on FFR reduction is asymmetric, L FFR The penalty for proximal FFR drop is greater than L FFR Penalizes differences in far-end FFR drops. To treat differences in FFR drops equally (regardless of their location), we design a symmetric version of L FFR To this end, we add a second term calculated from the assumed FFR curve The second item Calculated by adding the FFR drop from distal to proximal:

[0207]

[0208] Alternatively, in order to be able to distinguish between focal FFR reduction and diffuse FFR reduction, it is crucial that the predicted retracement curve and the reference retracement curve decline with a similar steepness. This is achieved by penalizing the difference between the histogram of the predicted FFR reduction and the reference histogram, and introduces a so-called histogram loss. However, directly binning the output values ​​is not differentiable, so the neural network cannot be trained. To solve this problem, the Parzen-Rosenblatt window method is used to approximate the bin by using a normal distribution with the mean at the center of the corresponding bin. As described by Parzen et al. in "On Estimation of a Probability Density Function and Mode" (The Annals of Mathematical Statistics 33 (3), 1962, pp. 1065-1076), the Parzen-Rosenblatt window method is a widely used nonparametric method for estimating the probability density function p(x) of a specific point p(x) from a sample p(xn), which does not require any knowledge or assumptions about the underlying distribution. Specifically, we employ 32 normal distributions, for example, equally spaced between, for example, -0.1 and 0.5 (i.e., the expected range of FFR reduction), where σ is, for example, 0.1. This produces a series of values ​​(i.e., a histogram) describing the number of occurrences of FFR reductions of a particular magnitude in both the predicted FFR curve and the reference FFR curve. Thereafter, the differences between these histograms are penalized using a weighted absolute error. Each bin is weighted according to the magnitude of the corresponding FFR reduction, thereby giving more weight to larger FFR reductions that are less common but more clinically important.

[0209] In an embodiment, there may be a negative FFR drop within the reference standard (1709). For example, such a negative FFR drop may be caused by an invasive withdrawal pressure measurement. As described by T et al. in “Influence of hydrostatic pressure on intracoronary indices of stenosis severity in vivo” (Clin Res Cardiol 2018107: 222-232), this negative FFR drop (meaning an increase in FFR from proximal to distal) is mainly related to the hydrostatic pressure effect. Optionally, another loss function (the so-called monotonic loss) can be included to exclude any hydrostatic pressure. By disabling the ReLU activation function as the last layer (see Fig.17 ), allowing negative FFR drops. These are discouraged by penalizing the sum of negative FFR drops. Finally, Fig.17 In step 1707, output ( Fig. 20 2006 and Fig. 20 2007). The output is data representing the FFR pullback along the axial trajectory of the vessel of interest.

[0210] Fig.11 An exemplary graphical user interface (display screen) is shown that visually communicates the results from step 1507 to the user. Fig.11 1102 shows a volume rendered image of CCTA data in which the three main coronary arteries (right coronary artery (RCA), left anterior descending coronary artery (LAD) and left circumflex coronary artery (LCX)) are enhanced. The RCA is the vessel of interest and is generated by Fig.11 The multi-planar reconstruction (MPR) shown in section B of 1101 is Fig.11 In 1101-section B, the left side (1103) corresponds to the opening (proximal) position of the RCA, and the right side (1104) corresponds to the distal position of the RCA. 1101-section C shows the lumen area or lumen diameter curve along the RCA, wherein the x-axis corresponds to the x-axis of the MPR view of section B of 1101 and the centerline of the RCA. Finally, 1101-section D shows the FFR values ​​along the centerline points in the RCA, wherein the x-axis is the same as the x-axis of the MPR view and the diameter / area curve. In addition, in 1102, the FFR values ​​along the centerline of the volume-rendered RCA are used to color-code the corresponding segments of the volume-rendered RCA. Note that the vertical marks in 1101-sections C and D correspond to the minimum area / diameter (bold line marks) and degree of obstruction (dashed line marks) derived from common quantitative coronary artery analysis techniques.

[0211] Optionally, in Fig.17At step 1710 of the method, different methods are used to train the machine learning-based arterial characterization network (1704) and the machine learning-based FFR pullback network (1706). This "end-to-end training" is particularly valuable when the reference standard (1709) includes a reference pullback value and a single point (distal) reference value (e.g., distal FFR and pullback FFR). First, both networks are trained simultaneously using the pullback reference value. Thereafter, the networks are trained separately (using a single point reference value) or one of the networks is frozen. For example, the machine learning-based FFR pullback network is frozen so that only the weights of the machine learning-based arterial characterization network are updated. In this example, in order to calculate the loss, only the loss of the machine learning-based FFR pullback network is used. Therefore, the machine learning-based arterial characterization network is optimized relative to the final machine learning-based FFR pullback network. This method has the following advantages: the machine learning-based arterial characterization network is optimized for the actual purpose of assessing stenosis. This automatically focuses on the relevant segments (i.e., those segments where lesions are present). Here "attention" specifically means: the contribution to the loss function, and hence to the weight update, is larger for lesions. This is not the case when using lumen area for supervision, where the largest loss contribution may come from segments with large lumen areas (which are less relevant for stenosis assessment than segments with small lumen areas (i.e., lesions).

[0212] Optionally, the machine learning based FFR pullback network is defined as a machine learning based narrow evaluation network (206, Figure 2 ) and the machine learning-based FFR retracement network (1706, Fig.17 ) combination. Fig.21 An example of such a combined network is shown. Figure 2 Public machine learning-based narrow evaluation network, Fig. 9 , Fig.10 , Fig.13 or Fig.14 The network consists of three stages. In the first stage (2101), the input (whether pre-encoded or not) as the result of the machine learning based artery characterization network (1704) and the coronary tree characteristics (1705) are fed to the second stage encoder (2102). The main difference is that the VAE encoding (from 1704) is used as an additional input (2104). In the third stage (2103), there are two output heads. Such as reference Fig. 20 The FFR regression head (2105) described above and the ... Fig. 9 , Fig.10 , Fig.13 or Fig.14 The classification header (2106) described. Fig.21The output of the network shown is threefold: 1) predicted FFR pullback values ​​(2106); 2) individual FFR values ​​for each artery (2107); and 3) a prediction of the presence of functionally significant stenosis (2108), based on the output of the classification head (2109) combined with the output of the individual FFR predictions (2107).

[0213] In addition, if Figure 2 An extension of the flowchart described in 3, Fig.17 The flowchart of allows the inclusion of myocardial properties, and the description from extension 3 "Including myocardial properties" also applies as referenced Fig.17 Flowchart of the method described. Fig. 22 Provides a reference for Fig.21 Illustration of the machine learning-based FFR pullback network architecture integrating myocardial properties.

[0214] Experimental setup

[0215] The artery representation network (1704) was trained for 1200 epochs using MAE as reconstruction loss, binary cross entropy as segmentation loss, and Kullback-Leibler divergence to regularize the latent space with a weighting factor of 1 at each epoch. To improve the representation of arteries specifically, the reconstruction loss within the reference segmentation (lumen and plaque) was weighted by a factor of 5. To supervise the lumen area regression during training, MAE with a weight of 10 was used. A learning rate of 10 was used. -5 The network was optimized with the ADAMW (Loshchilov and Hutter, 2019) optimizer with a batch size of 512. Once trained, we applied the network to each section of the MPR to extract the lumen area and VAE encoding along the centerline.

[0216] The FFR pullback network (1706) was trained for 150 epochs using an ADAMW optimizer with a cyclic learning rate with a linear schedule. The cyclic learning rate varied between 5e-4 and 1e-5 over a period of 40 epochs. Due to the constraints imposed by the different artery lengths, the network can only process a single artery at a time. Therefore, the loss is accumulated over eight training iterations before backpropagation, corresponding to an effective batch size of 8. To supervise the FFR pullback, we balance fidelity and sensitivity to noise in the pullback reference by pooling the output of the network to a step size of 2 mm using average pooling with a kernel size of 4. To minimize misalignment, the invasive pullback measurement was manually registered to the input by shifting the start of the pullback signal so that the FFR drop optimally overlaps the luminal narrowing. Since the pullback reference only covers a portion of the artery, the pullback supervision is only applied to that portion by masking the distal overlap. The EMD loss and histogram loss are weighted by, for example, factors 0.1 and 5, respectively. Optionally, the monotonic loss is weighted by, for example, 20. The coefficients were chosen based on preliminary experiments to achieve similar magnitudes of the loss terms.

[0217] Now refer to Fig.23 Another embodiment of the present application is disclosed. Obviously, the steps described herein can be performed in any logical order and some of them can be omitted. Figure 2 and Fig.17 The method described in the flowchart of predicts FFR drawdown by first characterizing the artery in terms of lumen area and unsupervised features, and then using these features to predict FFR drawdown. This two-step method enables manual changes to intermediate outputs (i.e., features) that can be useful for correcting potential errors, and also predicts FFR drawdown after successful percutaneous coronary intervention (PCI). PCI refers to a series of minimally invasive surgeries used to open blocked coronary arteries (those coronary arteries that deliver blood to the heart). By restoring blood flow, the treatment can improve symptoms of blocked arteries, such as chest pain or shortness of breath. PCI involves combining coronary angioplasty with stenting, which is the insertion of a permanent wire mesh tube consisting of a drug eluting or bare metal stent. The stent delivery balloon from the angioplasty catheter is inflated with a medium to force contact (stent apposition) between the struts of the stent and the vessel wall, thereby widening the vessel diameter. After entering the blood flow through the femoral or radial artery, the procedure uses coronary catheterization to make the vessel visible on X-ray imaging. Thereafter, the interventional cardiologist may perform a coronary angioplasty using a balloon catheter, in which a deflated balloon is advanced into the blocked artery and inflated to relieve the narrowing; specific devices such as stents may be deployed to hold the vessel open. A variety of other procedures may also be performed. Fig.23A flowchart for calculating FFR pullback after successful PCI treatment using CCTA images acquired before treatment is shown. As described above, this workflow can also be used in conjunction with an artery characterization network based on machine learning ( Figure 2 204 or Fig.17 1704) produces manual adjustment of vascular characteristics.

[0218] exist Fig.23 At step 2301, the execution is as follows Figure 2 A method described by a flowchart or as represented by Fig.17 A flow chart describing the method.

[0219] exist Fig.23 Step 2302, adjust the derived arterial characteristics ( Figure 2 204 or Fig.17 1704). As described above, arterial properties can be altered manually, semi-automatically, or automatically. To simulate successful PCI, the lumen area can be enlarged in the environment surrounding the lesion to mimic a placed stent, and plaque components (calcified, non-calcified, mixed) can be removed from the treatment range. Fig.24 An illustration of changing arterial properties to simulate successful PCI treatment is provided. An MRI image of the vessel of interest is shown by 2401, with virtual stent placement indicated by 2402, which also refers to the extent of the lesion. A cross-sectional image at the proximal end of the lesion extent is shown by 2403, and a cross-sectional image at the distal end of the lesion extent is shown by 2404. The two cross-sectional slices will represent the healthy cross-sectional area just outside the occlusion extent. Fig.24 For illustration, two derived arterial properties are shown ( Figure 2 204 or Fig.17 1704); the cross-sectional area (2405) along the axial trajectory of the vessel of interest and the calcified plaque area (2406) along the axial trajectory of the vessel of interest. Successful PCI is simulated by changing the cross-sectional area curve (2405) within the lesion range (2402) to obtain a healthy cross-sectional area as shown in 2407. All plaque components within the virtual stent placement indicated by 2402 are removed because these plaque components no longer block blood flow after stent placement. For example, the portion of the calcified plaque area along the axial trajectory of the vessel of interest (2406) that is located within the virtual stent (2402) can be set to zero.

[0220] A method for automatically defining the lesion segment and calculating the healthy reference area within the lesion segment is now described. This will allow simulation of a successful PCI procedure and calculation of the FFR pullback after such a simulated PCI intervention. Fig.25 Inside, as Figure 2 Step 204 or Fig.17As a result of step 1704, the derived cross-sectional area (2501) along the axial trajectory of the vessel of interest is visualized. The lesion range (2502) (also called the occlusion range or virtual stent range) is calculated, for example, based on the location (2503) where the curvature of the area curve (2501) covering the minimum area changes significantly, for example, as taught by Kooijman et al. in "Computer-aided quantitation of the severity of coronary obstructions from single view cineangiograms" (International Symposium on Medical Imaging and Image Interpretation. 1982: 59-64 (IEEE, catalog number 82CH1804-4)). Next, a straight line (2404) is fitted through the area values ​​along the axial trajectory of the vessel of interest, wherein the area values ​​within the lesion range (2402) are excluded during fitting, as described in "CAAS II: A Second Generation system for Off-Line and On-Line Quantitative Coronary Angiography" (Cardiovascular Diagnosis 1994; 33: 61-75) by Gronenschild E et al. Line (2504) represents the cross-sectional area along the axial trajectory of the vessel of interest when the vessel is healthy, and is also referred to as the reference area along the vessel of interest. There are branch arteries (bifurcation) within the lesion range (such as Figure 2 Step 205 or Fig.17 In the case of step 1705 of ), different methods can be used to simulate the effect of the pressure drop in the healthy coronary artery after the branch to calculate the reference area along the vessel of interest. Fig.252505 shows an example of a cross-sectional area curve, where the presence of a bifurcation is indicated by 2506. The proximal reference area (2508) is calculated by extrapolating a fitted line (through the area values ​​along the axial trajectory of the vessel of interest except all values ​​from the proximal end of the lesion range) from the proximal end of the lesion range (2502) to the bifurcation location (2506). Similarly, the distal reference area (2507) is calculated by extrapolating a fitted line (through the area values ​​along the axial trajectory of the vessel of interest except all values ​​from the distal end of the lesion range) from the distal end of the lesion range (2502) to the bifurcation location (2506). In addition, Murray's law can be incorporated into the calculation of the proximal reference area (2508) and the distal reference area (2507). Murray's law describes the relationship between the radii of the junctions in a network of tubular conduits that transport fluids. Whenever a branch of radius r splits into two branches of radii r1 and r2, then all three radii should follow the formula r 3 =r1 3 +r2 3 .

[0221] Now refer to Fig.26 An alternative automated method for simulating virtual stent placement is described that exploits the dense latent space of a variational autoencoder. Fig.26 In FIG. 1 , an MRI image of a vessel of interest is shown by 2601, wherein a virtual stent placement is indicated by 2602. A cross-sectional image at the proximal end of the lesion extent is shown by 2603, and a cross-sectional image at the distal end of the lesion extent is shown by 2604. The two cross-sectional slices will represent a healthy cross-sectional area just outside the occlusion extent. Fig.26 The curve 2605 of FIG. 2605 shows an arterial representation network ( Fig.17 The cross-sectional area (2609) of the result of 1704) is shown. Fig.17 The result of 1706 is the FFR retracement (2610).

[0222] As mentioned above Fig.19a and Fig.19b As described, a variational autoencoder can be trained to extract relevant information from the feature vector (z). From this VAE feature vector, the feature (z) is used to directly regress the lumen area (see reference Fig.19a To obtain an unsupervised encoding of the treated lesion, the encoding of the healthy segment in front of the lesion (proximal) and the encoding of the healthy segment behind the lesion (distal) are used as input to the linear layer. Fig.19aLinear interpolation is performed between 1907). Since the encoding space of the VAE is dense, the interpolated encoding corresponds to the real coronary artery segment. In addition, the disentangled nature of the encoding ensures smooth transitions. The interpolated encoding is fed to the lumen area regression network (1906) to obtain a cross-sectional area curve (2613) containing the virtual treatment. This is determined by Fig.26 Further shown. Image 2606b shows a cross-sectional slice from the MPR image (2601) at a proximal position of the lesion extent (2602), and similarly, image 2608b shows a cross-sectional slice from the MPR image (2601) at a distal position of the lesion extent (2602). Image 2607b shows a cross-sectional slice from the MPR image (2601) within the lesion segment (2601). Within this slice, the lumen is represented by 2611 and the plaque is represented by 2612. To illustrate the interpolation behavior in the latent space, images 2606a, 2607a and 2608a show the encoded interpolation results. The images themselves are the results of a decoder (1904) trained to reconstruct the center input slice of the input stack from the feature vector (z), and the overlays within these images are the results of a segmentation decoder (1905). In the case where the virtual stent does not cover the entire disease extent, the effect is that the described interpolation can also be used in the feature space to simulate such "incorrect" treatment. This refers to Fig. 27 . The MPR of the vessel of interest is shown by 2701, where the virtual stent placement is indicated by 2702. In this example, the proximal (2703) slice does not show the condition, but the distal side (2704) of the virtual stent still shows plaque condition. At the distal cross-sectional slice, there is a calcified plaque (2707). The cross-sectional slices represented by column (2705) show cross-sectional slices with superposition along the length of the virtual stent placement. The cross-sectional slices represented by column (2706) show the interpolated results of the encoding. The images themselves are the result of a decoder (1904) trained to reconstruct the center input slice of the input stack from the feature vector (z), and the superposition within these pictures is the result of a segmentation decoder (1905). As in the example, the virtual stent does not cover the entire range of conditions, and there is still a distal calcified plaque component in the simulated healthy cross-sectional slice at the distal portion of the virtual stent. Optionally, instead of encoding the healthy segment in front of the lesion (proximal) and encoding the healthy segment behind the lesion (distal) ( Fig.19a 1907), a linear interpolation is performed between the feature vector (z) of the healthy segment in front of the lesion (proximal) and the feature vector (z) of the healthy segment behind the lesion (distal).

[0223] Finally, in Fig.23At step 2303, the FFR pullback after virtual stent placement is calculated. In step 2302, the vessel characteristics simulating the virtual stent placement have been calculated. The adjusted vessel characteristics are fed to the Figure 2 The machine learning based stenosis assessment network of step 206, or the adjusted vessel characteristics are fed to Fig.17 The machine learning based FFR pullback network of step 1706 is used to calculate the FFR and / or FFR pullback of the simulated virtual stent placement. Fig.26 The image 2606 provides the adjusted vessel characteristics (2302) fed to Fig.17 An example of a machine learning based FFR pullback network of step 1706 of FIG. 1 is shown by 2611 as an FFR pullback curve after virtual stent placement. The example shows that the FFR drop within the diseased segment ( 2602 ) is eliminated when compared to the FFR pullback curve ( 2610 ) before treatment.

[0224] Optionally, virtual stent placement may be simulated without adjusting the vessel properties (2302), and Figure 2 or Fig.17 The MPR of the vessel of interest can be adjusted directly within the workflow. First, the lesion extent is determined from the lumen area curve (e.g., as previously described in reference Fig.25 Next, the MPR slices are sliced ​​to cover the lesion range within the MPR of the vessel of interest. Fig.26 , picture 2601 shows the MPR of the vessel of interest, by interpolating between healthy cross-sectional MRI slices 2603 and 2604 to create a healthy MPR image along the axial trajectory of the vessel of interest. This interpolation can also be performed with the help of a deep learning network that is trained to generate / interpolate new cross-sectional slices from the MPR image between the healthy proximal cross-sectional slice and the healthy distal cross-sectional slice from such an MPR image. Finally, this new MPR of the vessel of interest is fed to the machine learning based artery characterization network ( Figure 2 204 or Fig.17 1704) and execute Figure 2 or Fig.17 The remaining steps.

[0225] Now refer to Fig.28 Another embodiment of the present application is disclosed. Obviously, the steps described herein can be performed in any logical order and can be partially omitted. Fig.28 A flow chart of a machine learning based method for automatically extracting and anatomically labeling coronary artery trees is shown.

[0226] Fig.29A high-level overview of a machine learning-based approach for automatically extracting coronary artery trees in CCTA images by iteratively tracking automatically placed seed points is provided. Subsequently, an ensemble of graph convolutional neural networks (GCNs) is used to refine the extracted tree and label its segments. At a high level, reference Fig.29 , after initialization (2901) with a seed (2905) and an opening (2906), a coronary artery tree is extracted by 1) iterative tracking (2902), and then 2) applying a collection of GCNs to refine the initially extracted tree (2903). Finally, another GCN is configured to label anatomical segments (2904).

[0227] exist Fig.28 In step 2801, a CCTA image dataset is obtained. This image dataset represents a volumetric CCTA image dataset, for example, a single contrast-enhanced CCTA dataset. The CCTA dataset can be obtained from a digital storage database (such as a picture archiving and communication system (PACS) or a VNA (vendor neutral archive), a local digital storage database, a cloud database), or can be directly obtained from a CT imaging modality. During CCTA imaging, a contrast agent is introduced into the patient. In addition, CCTA imaging can be ECG-triggered.

[0228] The coronary artery tree is represented as an undirected dendrogram. Each point in the centerline corresponds to a node in the undirected dendrogram, and the connections between the centerline points are represented by undirected edges. Fig.28 In step 2802 of the invention, the processor initializes a dendrogram. The dendrogram (2901) is initialized by automatically placing seed points in the coronary arteries (2905) and the left and right coronary artery ostia (2906). Subsequently, the dendrogram is constructed directly during the coronary artery extraction by simultaneously tracking the coronary artery centerlines (2907) from the identified seed points. In this process, new points in the coronary arteries are iteratively appended to the dendrogram. In this way, multiple redundant tracking of segments is prevented and computational redundancy is reduced.

[0229] For initialization, the locations of seed points and coronary artery openings are predicted by two fully convolutional neural networks (seed-CNN and opening-CNN). The same architecture for both networks includes seven 3D convolutional layers with a kernel width of 3. In layers 1-4, the number of channels is set to 32, and in layers 5 and 6, the number of channels is set to 64. The last layer produces a single output channel. To increase the receptive field, in layers 3 and 4, dilation factors of 2 and 4 are used, respectively, while in the remaining layers, the dilation is set to 1. Seed CNN and opening CNN are trained to predict the negative exponent of the distance to the nearest coronary centerline or opening, respectively, for each voxel. This presents a heatmap-like prediction map indicating where the coronary arteries and openings are located. Thereafter, the seed points are identified as local maxima from the predicted heatmap.

[0230] exist Fig.28 In step 2803, the processor performs coronary artery tree tracking. In order to add new nodes to the tree, a CNN (tracking-CNN) is used to predict the direction and step size starting from each endpoint of the graph. The tracking-CNN receives a 3D image patch at the seed point location and predicts the direction of the coronary artery in the form of a binary output at discrete evenly spaced locations on the unit sphere. In order to obtain a new point, the predicted direction is entered with a step size corresponding to the vessel radius prediction. To prevent backward direction prediction, the predicted direction class that is close to the follow-up direction is masked. Except for the number of output nodes in the seventh layer, the architecture of the tracking-CNN is the same as that of the seed-CNN and opening-CNN (as described in step 2802). The tracking-CNN uses 501 output channels: 500 direction classes and one additional channel for radius regression. A softmax activation layer is then used to combine the outputs for the direction classes.

[0231] Only endpoints are tracked, i.e. nodes with less than two edges. Of these nodes, only those for which the uncertainty of the direction prediction is below a certain threshold are tracked. Because nodes not connected to an ostium are unlikely to be in a coronary artery, a different uncertainty threshold is used for nodes connected to an ostium than for all other nodes. The uncertainty is given by the entropy over the categories of the direction output.

[0232] Tracking all seed points simultaneously produces multiple sub-graphs. Therefore, these tracked sub-graphs are merged when they are superimposed. Fig.30An example of simultaneous tracking is provided in , where the tracking of two seed points (blue node (3001) and red node (3002)) is shown. Since the two tracked subgraphs (blue node and red node) overlap after 3 steps, they are merged into a single connected graph (green node, 3003) in step 4. Note that this merge avoids redundant tracking of the proximal node located above the bifurcation (3004). Therefore, a new node is created only if its position is not already occupied by another node (i.e., the node is not part of another subgraph). Node overlap is evaluated using the artery radius at each existing node predicted by the tracker. Fig.31 , 3101 shows the construction of a dendrogram, which includes the extraction of seeds and subsequent artery tracing.

[0233] exist Fig.28 In step 2804, the coronary tree is refined. During tree tracing as described in step 2803, high sensitivity is archived to obtain a complete coronary tree (2902). However, this may come at the expense of limited accuracy, i.e., false positive extractions representing veins or other tubular structures. Therefore, in order to remove false positives from the extracted tree, a collection of GCNs is used (2908). GCNs are a generalization of CNNs to arbitrary graph inputs. Therefore, GCNs enable learning a combined representation of node features and connectivity directly from the graph. Therefore, it is necessary to define features that can then be used by GCNs.

[0234] To enable refinement of the initially extracted tree, arterial segments are created by grouping adjacent centerline points of the dendrogram. These segments are characterized using a set of features that describe the segment's position, orientation, geometry, and image appearance.

[0235] The position of the segment is described by the Cartesian coordinates of the centerline points of the segment relative to the center of the left ventricular myocardium at the start, end, and each quarter of the segment length, as taught by Bruns et al. in "Deep learning from dual-energy information for whole-heart segmentation in dual-energy and single-energy non-contrast-enhanced cardiac CT" (described in Medical Physics 47, 2020, pp. 5048-5060). Two features describing the orientation of the segment are extracted. The first feature corresponds to the Cartesian coordinates of the normalized direction vector between the endpoints of the segment. The second feature consists of the Cartesian coordinates of the normalized direction vector between the first two centerline points in the segment. To describe the geometry of the blood vessels, the mean and standard deviation of the vessel radius of the centerline points in the segment (corresponding to the vessel size) are used.

[0236] The appearance of the segment is characterized by its texture, which is derived using the output of the tracking-CNN (2907, and described by step 2803) and the seed-CNN (2905, and described by step 2802). The characterization is described by the mean and standard deviation of the entropy values ​​of the centerline points in the segment. The entropy value corresponds to the uncertainty of the tracker at that location. Intuitively, this indicates how similar the segment is to a vessel. Similarly, the mean and standard deviation of the output values ​​of the seed-CNN (extracted from the output map at the location of the centerline points in the segment) are calculated.

[0237] In order to distinguish true coronary artery segments (positive class) from other vessel-like structures (negative class), binary classification was performed using GCN. GCN enables learning a combined representation of node features and connections using the initially extracted dendrogram directly as input for classification. To increase robustness to potentially missing segments, an ensemble of GCNs was used and applied to multiple graphs at different resolutions.

[0238] In the Graph Attention Network (GAT), the weights for aggregating features from neighboring nodes are learned end-to-end using an attention subnetwork. The aggregation function of GAT is defined as in, is the local neighborhood first transformed by the weight matrix W The input features of the nodes in . σ represents the nonlinearity, for which the Leaky ReLU function is used. In order to determine the weighting coefficient α ij , a decay subnet (multi-head) is used, which consists of a weight vector The parameterization is as follows:

[0239]

[0240] Among them, || represents cascade.

[0241] Since attention mechanisms enable GATs to express the importance of neighboring segments to each other, they may be a suitable choice for encoding local segment neighborhoods to refine our extracted coronary artery trees.

[0242] In order to exploit information about the geometry of the coronary artery tree, a sufficiently large receptive field is required. However, as Wu et al. teach in "A Comprehensive Survey on Graph Neural Networks" (IEEE Transactions on Neural Networks and Learning Systems 32, 2021, pp. 4–24), GCNs typically have limited receptive fields. Therefore, the effective receptive field is increased by using a coarser graph at a lower resolution as input. To create a coarse graph, adjacent nodes of the initially extracted tree (fine graph) are grouped into segments.

[0243] Although grouping nodes increases the receptive field, it may also group true and false nodes into the same segment (e.g., artery leaking into vein), leading to label ambiguity (see Fig.32 , 3201). Therefore, multiple rough graphs (3202) are created by grouping the centerline nodes into segments in two different ways. For the first process graph, we apply the conventional definition taught by Hampe et al. in "Graph attention networks for segment labeling incoronary artery trees" (Medical Imaging 2021: Image Processing), in which multiple segments are separated by bifurcations. In order to enable the subdivision of these segments, an additional graph is created by dividing the segments from the above undivided graph into smaller segments with lower resolution. In order to enable information flow between all segments adjacent to the bifurcation, the edges of the graph are established by combining all pairs of segments that may be involved. In addition, the information flow from each node to itself is achieved by inserting an edge from each node to itself.

[0244] In order to combine the benefits of reduced label ambiguity (graphs with small segments) and the benefits of large receptive fields (graphs with large segments), a multi-resolution graph integration strategy is adopted. To do this, the predictions of all graphs with different resolutions are back-projected to the fine graph. The integration is performed by taking the average of the output probabilities of all GCNs on the fine graph.

[0245] Although the GCNs in the set are applied to graphs with different resolutions, Fig.33 The network architecture schematically shown in is the same for all GCNs. It consists of three GAT layers with dense connections from the input features to all consecutive GAT layers. In each GAT layer, four heads (each with eight encodings) are used in combination with residual connections and discards. In addition, the input features are cascaded to the input of each consecutive GAT layer. Finally, a linear layer followed by a softmax activation function is used to predict the binary output probabilities corresponding to the real segments of the coronary artery tree and other vascular-like structures. Fig.31 Figure 3102 provides an illustration of the results of the tree refinement step.

[0246] Finally, in Fig.28 At step 2805, anatomical coronary artery labeling is performed. Anatomical labels are assigned to the segments of the extracted coronary artery tree (2904). Similar to tree extraction, a collection of GCNs trained on graphs with different resolutions for anatomical labeling is adopted (2909). Segment labeling is proposed as a multi-class classification task. By using GCN, the model is able to learn information about each segment and about the relationship between these segments. In addition, the multi-resolution integration method alleviates the problems associated with label ambiguity caused by missing bifurcations, while ensuring that the receptive field of the method is large enough to model the relationship between segments. Graphs of different resolutions are obtained by grouping centerline points into segments by a method similar to that described at step 2804. The network architecture for anatomical labeling is the same as the network architecture for tree refinement (step 2804), except that ten output categories are used for anatomical labeling and one output category is used for each coronary artery region segment label (from the reference standard 2806). In addition, the same input features are used. Fig.34 A diagram of the result of step 2805 is provided; it shows the extracted coronary artery tree in which each coronary artery segment is anatomically labeled.

[0247] The reference standard (2806) is a database containing data of multiple patients. Each set in the database contains, for each patient, a) a contrast-enhanced CT image dataset (2801 represents the reference image set during the training phase) and a corresponding coronary artery centerline tree, wherein each centerline point in the tree is assigned a lumen radius and an anatomical label, such as according to the model introduced by Austen et al. in "A reporting system on patients evaluated for coronary artery disease. Report of the Ad Hoc Committee for Grading of Coronary Artery Disease, Council on Cardiovascular Surgery, American Heart Association" (Circulation 1975; 51, 5-40).

[0248] Experimental setup

[0249] All GCNs were trained for 500 epochs using the reference standard (2806) by using the Adam optimizer. Unless otherwise specified, the learning rate was set to 0.001 for the first 300 epochs and to 0.0001 for the remaining 200 epochs. For dropout, a probability of 0.2 was used.

[0250] Tree initialization (2802) and tree tracking (2803): All three CNNs (2905, 2906, and 2907) were trained using the reference standard (2806) as described by Wolterink et al. in "Coronary artery centerline extraction incardiac CT angiography using a CNN-based orientation classifier" (Med Image Anal. 2019 Jan;51:46-60). For initialization of tracking, 400 seed points were used. Such a large number of seed points was chosen to ensure high sensitivity. In addition, redundancy was reduced and it was ensured that seed points were not generated too close to each other by maintaining a minimum distance of 3 mm between seed points. For artery tracking, the input to the tracking-CNN was a 19×19×19 image patch and the number of output directions was 500. If the entropy (i.e., uncertainty) of the output direction is below 0.9, nodes connected to the opening are tracked, and if the entropy is below 0.7, nodes disconnected from the opening are tracked. To avoid backward tracking, we masked directions that were within less than 60 degrees of the previously tracked direction. Tracking terminates if there are no nodes to track or if the maximum number of 40 steps determined in preliminary experiments is reached, thereby preventing a large number of false positive extractions. Compared to redundantly tracking the proximal portion, the proposed graph tracking reduces the number of tracked nodes from an average of 7667 to 4476 (42%). After tracking is completed, all single nodes are removed because they present untracked seed points. Given that the extracted seed points provide starting points for constructing the tree, incorrect seed points and leakage due to incorrect centerline direction predictions may lead to false positives (usually in arterial structures similar to veins). Figures 3501 and 3502 of Figure 35 show that the seed-CNN outputs high values ​​for the coronary arteries and lower values ​​for the coronary veins. Figure 3501 shows an axial slice of the CCTA image dataset, showing the coronary arteries (3503) and veins (3504) side by side. Image 3502 shows the output map of the seed-CNN, which has high probability values ​​for the coronary arteries and lower probability values ​​for the coronary veins. When the tracker is trained on relatively small image patches around the centerline voxels, it tracks the coronary arteries (3505), but it also tracks other vascular structures, such as the coronary veins (3506). Image 3505 shows the output of the tracking-CNN for a small patch around the seed point in the coronary arteries (the location in the CCTA marked by circle 3503). The probability output is visualized as an activation on the unit sphere, and the size of the activation corresponds to the probability amplitude. Image 3506 shows the output of the tracking-CNN for a small patch around the seed point in the coronary veins (the location in the CCTA marked by the dotted circle 3504).Therefore, after tree tracing, the tree is then refined by removing false positives (2804).

[0251] Tree refinement (2804): To reduce label ambiguity ( Fig.32 ) and still learn from a large receptive field, we used a graph ensemble strategy based on coarse graphs with different resolutions. Specifically, in addition to the unpartitioned graph, we created a graph with predetermined centerline points (e.g., 10 centerline points) per segment.

[0252] Anatomical Labeling (2805): In order to learn the robustness of the anatomical labeling to potential errors from tree extraction, we use the automatically extracted trees (the result of step 2804) as the input of the GCN for training, instead of using the reference trees from the reference standard (2806) as the input of the GCN for training. To train the set for anatomical labeling, the automatically extracted trees are represented by the same coarse graph as the tree extraction. Preferably, multiple graphs are created where each segment has a plurality of predetermined centerline points (e.g., 5, 10, 20, and 30 centerline points). Therefore, in order to set the reference for training the GCN, we project the anatomical labels from the reference tree onto the automatically extracted coarse graph. During training of the anatomical labeling, nodes that exist in the automatically extracted tree but not in the reference tree (false positive nodes) are not backpropagated.

[0253] The present disclosure mainly describes that the organ of interest is the myocardium and the blood vessel is the coronary artery. The skilled person will understand that the teaching can be extended to other organs in the same manner. For example, the organ of interest can be a kidney perfused by a renal artery, or a brain (or part thereof) perfused by an intracranial artery. In addition, the present disclosure relates to CCTA data sets (in several forms). The skilled person will appreciate that the teaching can be extended to other imaging modes in the same manner (e.g., rotational angiography, MRI, SPECT, PET, ultrasound, X-ray, etc.).

[0254] Embodiments of the present disclosure may be used on a stand-alone system, or included directly in, for example, a computed tomography (CT) system. Fig.36 An example of a high-level block diagram of a computed tomography (CT) system is shown. In the block diagram, the embodiment is included as an example of how the embodiment can be integrated into such a system.

[0255] Portions of the system, as defined by various functional blocks, may be implemented with dedicated hardware, analog and / or digital circuits, and / or one or more processors operating program instructions stored in a memory.

[0256] The most common form of computed tomography is X-ray CT, but there are many other types of CT, such as dual-energy CT, multi-energy spectrum CT, or photon counting CT. In addition, positron emission tomography (PET) and single photon emission computed tomography (SPECT) may be present or combined with any of the previous forms of CT.

[0257] Fig.36 The CT system of the present invention describes an X-ray CT system. In an X-ray CT system, the X-ray system moves around the patient in a gantry and acquires images. By using digital processing, a three-dimensional image is constructed from a large number of two-dimensional angiographic images taken around a single axis of rotation.

[0258] For a typical X-ray CT system 120, an operator positions a patient 1200 on a patient table 1201 and provides input for the scan using an operator console 1202. The operator console 1202 typically includes a computer, keyboard / foot pedal / touch screen, and one or more monitors.

[0259] The operator control computer 1203 uses the operator console input to instruct the gantry 1204 to rotate and sends instructions to the patient table 1201 and the X-ray system 1205 to perform the scan.

[0260] Using the selected scan protocol selected in the operator console 1202, the operator control computer 1203 sends a series of commands to the gantry 1204, the patient table 1201, and the X-ray system 1205. The gantry 1204 then reaches and maintains a constant rotation speed throughout the scan. The patient table 1201 reaches the desired starting position and maintains a constant speed throughout the scanning process.

[0261] The X-ray system 1205 includes an X-ray tube 1206 having a high voltage generator 1207 that produces an X-ray beam 1208 .

[0262] The high voltage generator 1207 controls the X-ray tube 1206 and transmits power to the X-ray tube 1206. The high voltage generator 1207 applies a high voltage across a vacuum gap between a cathode of the X-ray tube 1206 and a rotating anode.

[0263] Due to the voltage applied to the X-ray tube 1206, electron transfer occurs from the cathode to the anode of the X-ray tube 1206, resulting in an X-ray photon generation effect, also known as Bremsstrahlung. The generated photons form an X-ray beam 1208 which is directed to an image detector 1209.

[0264] X-ray beam 1208 includes photons having an energy spectrum ranging up to a maximum value determined by the voltage and current submitted to X-ray tube 1206 .

[0265] The X-ray beam 1208 then passes through a patient 1200 lying on a moving table 1201. The X-ray photons of the X-ray beam 1208 penetrate the patient's tissue to varying degrees. Different structures in the patient 1200 absorb different portions of the radiation, modulating the beam intensity.

[0266] The modulated X-ray beam 1208' emitted from the patient 1200 is detected by an image detector 1209 located opposite the X-ray tube.

[0267] The image detector 1209 may be an indirect or direct detection system.

[0268] In the case of an indirect detection system, the image detector 1209 includes a vacuum tube (X-ray image intensifier) ​​that converts the outgoing X-ray beam.

[0269] 1208' is imaged as an amplified visible light image. The amplified visible light image is then transmitted to a visible light image receiver (such as a digital camera) for image display and recording. This results in a digital image signal.

[0270] In the case of a direct detection system, the image detector 1209 comprises a flat panel detector. The flat panel detector converts the X-ray outgoing beam 1208' directly into a digital image signal.

[0271] The digital image signal generated by the image detector 1209 is passed to the image generator 1210 for processing. Typically, the image generation system includes a high-speed computer and a digital signal processing chip. The acquired data is pre-processed and enhanced before being sent to the display device 1202 for operator viewing and to the data storage device 1211 for archiving.

[0272] In the gantry, the X-ray system is positioned in such a way that the patient 1200 and the moving table 1201 are located between the X-ray tube 1206 and the image detector 1209 .

[0273] In contrast-enhanced CT scanning, the injection of contrast media must be synchronized with the scanning. The contrast media injector 1212 is controlled by the operation control computer 1203.

[0274] For FFR measurement, an FFR guidewire 1213 is present and adenosine is also injected into the patient via syringe 1214 to induce a maximal hyperemic state.

[0275] The embodiments of the present application are Fig.18The X-ray CT system 120 is implemented as follows. A clinician or other user acquires a CT scan of a patient 1200 by selecting a scan protocol using an operator console 1202. The patient 1200 lies on an adjustable table 1201 that moves at a continuous speed throughout the scan controlled by an operator control computer 1203. A gantry 1204 maintains a constant rotation speed throughout the scan.

[0276] A plurality of two-dimensional X-ray images are then generated using the high voltage generator 1207, X-ray tube 1206, image detector 1209 and digital image generator 1210 as described above. The images are then stored on hard drive 1211. Using these X-ray images, a three-dimensional image is constructed by the image generator 1210.

[0277] The general processing unit 1215 uses the three-dimensional image to perform the classification as described above.

[0278] Several embodiments of methods and apparatus for automatically identifying patients with functionally significant stenosis based on information extracted from only a single CCTA image have been described and illustrated herein.

[0279] While particular embodiments of the present application have been described, it is not intended that the application be limited thereto, as it is intended that the application be as broad in scope as the art will allow and the specification be read likewise.

[0280] For example, a multi-phase CCTA data set can be used, functional assessment of the renal artery associated with the perfused kidney can be assessed based on the disclosed method, and data processing operations can be performed offline on images stored in a digital memory (such as a PACS or VNA in a DICOM (Digital Imaging and Communications in Medicine) format commonly used in the field of medical imaging). Therefore, those skilled in the art will understand that other modifications may be made to the provided application without departing from the spirit and scope of the claimed application.

[0281] The embodiments described herein may include various data stores as discussed above as well as other memories and storage media. These may reside in various locations, such as on any or all of the storage media local to one or more of the computers (and / or residing in one or more of the computers) or remote from the computer on a network. In a particular set of embodiments, the information may reside in a storage area network ("SAN") familiar to those skilled in the art.

[0282] Similarly, any necessary files for carrying out the functions attributed to the computers, servers or other network devices may be stored locally and / or remotely, as appropriate.

[0283] Where the system includes computer devices, each such device may include hardware elements that may be electrically coupled via a bus, including, for example, at least one central processing unit ("CPU" or "processor"), at least one input device (e.g., a mouse, keyboard, controller, touch screen, or keypad), and at least one output device (e.g., a display device, printer, or speaker). Such a system may also include one or more storage devices (such as disk drives), optical storage devices, and solid-state storage devices (such as random access memory ("RAM") or read-only memory ("ROM")), as well as removable media devices, memory cards, flash memory cards, and the like.

[0284] Such a device may also include a computer-readable storage medium reader, a communication device (eg, a modem, a (wireless or wired) network card, an infrared communication device, etc.), and a working memory as described above.

[0285] A computer-readable storage medium reader may be connected to or configured to receive a computer-readable storage medium, which represents a remote, local, fixed and / or removable storage device and a storage medium for temporarily and / or more permanently containing, storing, transmitting and retrieving computer-readable information. The system and various devices will also typically include a plurality of software applications, modules, services or other elements located within at least one working memory device including an operating system and an application program (such as a client application or a web browser).

[0286] It should be understood that alternative embodiments may have many variations from the above-described embodiments. For example, customized hardware may also be used, and / or specific elements may be implemented in hardware, software (including portable software such as applets), or both.

[0287] Additionally, connection to other computing devices such as network input / output devices may be employed.

[0288] Various embodiments may also include receiving, sending or storing instructions and / or data implemented according to the foregoing description on a computer-readable medium. Storage media and computer-readable media for containing code or portions of code may include any suitable media known or used in the art (including storage media and communication media, such as but not limited to volatile and non-volatile, removable and non-removable media implemented in any method or technology for storing and / or transmitting information (such as, computer-readable instructions, data structures, program modules or other data) (including RAM, ROM, electrically erasable programmable read-only memory ("EEPROM"), flash memory or other memory technology, compact disk read-only memory ("CD-ROM"), digital versatile disk (DVD) or other optical storage, cassettes, magnetic tapes, magnetic disk storage or other magnetic storage devices, or any other media that can be used to store desired information and can be accessed by system devices)). Based on the disclosure and teachings provided herein, those of ordinary skill in the art will understand other ways and / or methods of implementing various embodiments.

[0289] Fig.37 A graphical representation of a machine in the example form of a computer system 18000 is shown, within which a set of instructions for causing the machine to perform any one or more of the methods, processes, operations, or schemes discussed herein may be executed. In an optional embodiment, the machine operates as a standalone device or may be connected (e.g., networked) to other machines. In a networked deployment, the machine may operate in the capacity of a server or client machine in a server-client network environment, or as a peer machine in a peer (or distributed) network environment. The machine may be a personal computer (PC), a tablet PC, a personal digital assistant (PDA), a cellular phone or smart phone, a Web device, or any machine capable of executing a set of instructions (sequentially or otherwise) specifying an action to be taken by the machine. In addition, although only a single machine is shown, the term "machine" should also be deemed to include any combination of machines that individually or jointly execute a set (or multiple sets) of instructions to perform any one or more of the methods discussed herein. Example embodiments may also be practiced in a distributed system environment where both local and remote computer systems that are linked through a network (e.g., by hardwired, wireless, or a combination of hardwired and wireless connections) perform tasks. In a distributed system environment, program modules may be located in both local and remote memory storage devices (see below).

[0290] The example computer system 18000 includes a processor 18002 (e.g., a central processing unit (CPU), a graphics processing unit (GPU), or both), a main memory 18001, and a static memory 18006 that communicate with each other via a bus 18008. The computer system 18000 may also include a video display unit 18010 (e.g., a liquid crystal display (LCD) or a cathode ray tube (CRT)). The computer system 18000 also includes an alphanumeric input device 18012 (e.g., a keyboard), a user interface (UI) cursor controller 18014 (e.g., a mouse), a disk drive unit 18016, a signal generating device 18018 (e.g., a speaker), and a network interface device 18020 (e.g., a transmitter).

[0291] The disk drive unit 18016 includes a machine-readable medium 18022 on which is stored one or more sets of instructions 18024 and data structures (e.g., software) embodying one or more methods or functions shown herein (or used by one or more methods or functions shown herein). The software may also reside completely or at least partially in the main memory 18001 and / or in the processor 18002 during execution by the computer system 18000, the main memory 18001 and the processor 18002 also constituting machine-readable media.

[0292] Instructions 18024 may also be sent or received over network 18026 via network interface device 18020 using any of a number of well-known transmission protocols (e.g., HTTP, Session Initiation Protocol (SIP)).

[0293] The term "machine-readable medium" shall be deemed to include a single medium or multiple media (e.g., a centralized or distributed database, and / or associated caches and servers) that store one or more sets of instructions. The term "machine-readable medium" shall also be deemed to include any medium that is capable of storing, encoding, or carrying a set of instructions for execution by a machine and causing the machine to perform any of one or more of the methods presented herein. Thus, the term "machine-readable medium" shall be deemed to include, but not be limited to, solid-state memories and optical and magnetic media.

[0294] The method embodiments shown herein may be computer-implemented. Some embodiments may include a computer-readable medium encoded with a computer program (e.g., software), which includes instructions operable to cause an electronic device to perform the methods of various embodiments. Software implementations (or computer-implemented methods) may include microcode, assembly language code, or higher-level language code, which may also include computer-readable instructions for performing various methods. The code may form part of a computer program product. In addition, the code may be tangibly stored on one or more volatile computer-readable media or non-volatile computer-readable media during execution or at other times. These computer-readable media may include, but are not limited to, hard disks, removable disks, removable optical disks (e.g., compact disks and digital video disks), cassettes, memory cards or sticks, random access memories (RAM), read-only memories (ROM), etc.

[0295] Accordingly, the specification and drawings are to be regarded in an illustrative rather than a restrictive sense. It will, however, be readily understood that various modifications and changes may be made thereto without departing from the broader spirit and scope of the application as set forth in the claims.

[0296] Within the spirit of the present disclosure, there are other changes. Therefore, although the disclosed technology is susceptible to various modifications and optional configurations, some of its illustrated embodiments are shown in the accompanying drawings and have been described in detail above. However, it should be understood that it is not intended to limit the present application to one or more specific forms disclosed, but on the contrary, it is intended to cover all modifications, optional configurations and equivalents that fall within the spirit and scope of the present application as defined in the appended claims.

[0297] Unless otherwise noted herein or clearly contradicted by context, the use of the terms "a" and "an" and "the" and similar referents in the context of describing the disclosed embodiments (especially in the context of the appended claims) are to be interpreted as covering both the singular and the plural. The terms "comprising," "having," "including," and "containing" are to be interpreted as open-ended terms (i.e., meaning "including, but not limited to") unless otherwise noted. The term "connected," when unmodified and referring to a physical connection, is to be interpreted as partially or fully contained therein, attached to, or joined together, even if there are some intervening pieces.

[0298] Unless otherwise indicated herein, recitation of ranges of values ​​herein are merely intended to serve as a shorthand method of referring individually to each separate value falling within the range, and each separate value is incorporated into the specification as if it were individually recited herein. Unless otherwise indicated or contradicted by context, use of the term "set" (e.g., "set of items") or "subset" will be construed to include a non-empty set of one or more members.

[0299] Furthermore, unless otherwise noted or contradicted by context, the term "subset" of a corresponding set does not necessarily mean a proper subset of the corresponding set, but rather a subset and a corresponding set may be equivalent.

[0300] Unless otherwise specified herein or clearly contradictory to the context, the operations of the processes described herein may be performed in any suitable order. The processes described herein (or variations and / or combinations thereof) may be performed under the control of one or more computer systems configured with executable instructions, and may be implemented as code (e.g., executable instructions, one or more computer programs, or one or more applications) that is jointly executed by hardware or a combination thereof on one or more processors. The code may be stored on a computer-readable storage medium, for example, in the form of a computer program that includes a plurality of instructions that may be executed by one or more processors. The computer-readable storage medium may be non-transitory.

[0301] Preferred embodiments of the present disclosure including the best mode known to the inventor for implementing the present application are described herein. After reading the foregoing description, variations of those preferred embodiments may be readily understood by those of ordinary skill in the art. The inventor expects those skilled in the art to appropriately adopt these variations, and the inventor intends to practice the embodiments of the present disclosure in a manner other than that specifically described herein.

[0302] Accordingly, the scope of this disclosure includes all modifications and equivalents of the subject matter recited in the appended claims as permitted by applicable law.In addition, unless otherwise indicated herein or clearly contradicted by context, the scope of this disclosure encompasses any combination of the above-described elements in all possible variations thereof.

[0303] Additional references are listed below:

[0304] · Wong DTL, Ko BS, Cameron JD, Nerlekar N, Leung MCH, Malaiapan Y et al., Transluminal attenuation gradient in coronary computed tomography angiography is a novel noninvasive approach to the identification of functionally significant coronary artery stenosis: a comparison with fractional flow reserve. J Am Coll Cardiol. (2013) 61:1271–9. doi:10.1016 / j.jacc.2012.12.029

[0305] · Ko BS, Wong DTL, BL, Leong DP, Cameron JD, Gaur S et al., Diagnostic performance of transluminal attenuation gradient and noninvasive fractional flow reserve derived from 320-detector Row CT angiography to diagnose hemodynamically significant coronary stenosis: an NXT substudy. Radiology. (2016) 279:75–83. doi:10.1148 / radiol.2015150383

[0306] · Diaz-Zamudio M, Dey D, Schuhbaeck A, Nakazato R, Gransar H, Slomka PJ, et al., Automated quantitative plaque burden from coronary CT angiography noninvasively predicts hemodynamic significance by using fractional flow reserve in intermediate coronary lesions. Radiology. (2015) 276:408 - 15. doi:10.1148 / radiol.2015141648

[0307] · Otaki Y, Han D, Klein E, Gransar H, Park RH, Tamarappoo B, et al., Value of semiquantitative assessment of high-risk plaque features on coronary CT angiography over stenosis in selection of studies for FFRct. J Cardiovasc Comput Tomogr. (2021) 16:27–33. doi:10.1016 / j.jcct.2021.06.004

[0308] · Gould KL, Lipscomb K, Calvert C. Compensatory changes of the distal coronary vascular bed during progressive coronary constriction. Circulation. (1975) 51:1085–94. doi:10.1161 / 01.CIR.51.6.1085

[0309] · Dey D, Achenbach S, Schuhbaeck A, Pflederer T, Nakazato R, Slomka PJ, et al. Comparison of quantitative atherosclerotic plaque burden from coronary CT angiography in patients with first acute coronary syndrome and stable coronary artery disease. J Cardiovasc Comput Tomogr. (2014) 8:368–74. doi:10.1016 / j.jcct.2014.07.007

[0310] · Hell MM, Dey D, Marwan M, Achenbach S, Schmid J, Schuhbaeck A. Noninvasive prediction of hemodynamically significant coronary artery stenoses by contrast density difference in coronary CT angiography. Eur J Radiol. (2015) 84:1502–8. doi:10.1016 / j.ejrad.2015.04.024

[0311] · Ko BS, Wong DTL, Cameron JD, Leong DP, Soh S, Nerlekar N, et al. The ASLA score: a CT angiographic index to predict functionally significant coronary stenoses in lesions with intermediate severity–diagnostic accuracy. Radiology. (2015) 276:91–101. doi:10.1148 / radiol.15141231

[0312] · Dey D, Gaur S, Ovrehus KA, Slomka PJ, Betancur J, Goeller M, et al., Integrated prediction of lesion - specific ischaemia from quantitative coronary CT angiography using machine learning: a multicentre study. Eur Radiol. (2018) 28:2655–64. doi:10.1007 / s00330 - 017 - 5223 - z

[0313] · Yang S, Koo BK, Hoshino M, Lee JM, Murai T, Park J, et al., CT angiographic and plaque predictors of functionally significant coronary disease and outcome using machine learning. JACC Cardiovascular imaging. (2021) 14:629–41. doi:10.1016 / j.jcmg.2020.08.025

[0314] · Ghanem AM, Hamimi AH, Matta JR, Carass A, Elgarf RM, Gharib AM, et al., Automatic coronary wall and atherosclerotic plaque segmentation from

[0315] 3D coronary CT angiography. Sci Rep. (2019) 9:47. doi:

[0316] 10.1038 / s41598 - 018 - 37168 - 4

[0317] · Loshchilov, I., Hutter, F., 2019. Decoupled Weight Decay Regularization, in: International Conference on Learning Representations – ICLR 2019

[0318] All references, including publications, patent applications, and patents, cited herein are hereby incorporated by reference to the same extent as if each reference were individually and specifically indicated to be incorporated by reference and were set forth in its entirety herein.

[0319] Listed terms:

[0320] The enumerated clauses are now provided for the purpose of illustrating some possible embodiments that may be provided according to the present disclosure. The set of clauses provided below is for illustration and should not be construed as limiting, exclusive, or exhaustive. Features described in one set of clauses may be utilized and incorporated into one or more of the other sets of clauses. In any one or more of the following sets of clauses, an embodiment may provide a computer-implemented method.

[0321] Clause Set A1:

[0322] Embodiments disclosed herein may provide methods and systems for assessing occlusion of a vessel of interest in a patient, the methods and systems involving one, some, or all of the following operations:

[0323] obtaining a volume image dataset for a blood vessel of interest;

[0324] analyzing the volumetric image data set to extract data representing an axial trajectory of a vessel of interest;

[0325] generating a multiplanar reformatted (MPR) image based on the volumetric image dataset and data representing an axial trajectory of a vessel of interest;

[0326] generating first characteristic data characterizing the presence of zero or more bifurcations or side branches along an axial trajectory of the vessel of interest;

[0327] supplying the MPR image and the first feature data to a first machine learning network, the first machine learning network outputting, given the MPR image and the first feature data, i) a plurality of latent space encodings characterizing features of the vessel of interest along the axial trajectory of the vessel of interest and ii) additional feature data of additional features of the vessel of interest along the axial trajectory of the vessel of interest; and

[0328] The plurality of latent space encodings and additional feature data output by the first machine learning network are supplied to a second machine learning network, which outputs data characterizing the FFR pullback of the vessel of interest given the input data.

[0329] A2. The method of clause A1, further comprising displaying or outputting data characterizing FFR pullback of the vessel of interest.

[0330] A3. A method according to clause A1 or A2, wherein:

[0331] The first characteristic data is generated by analyzing the MPR image; and / or

[0332] The first characteristic data is generated by analyzing the volume image data set; and / or

[0333] The first feature data is generated by a coronary artery centerline tree derived from a volumetric image dataset.

[0334] A4. A method according to clause A1, A2 or A3, wherein:

[0335] The additional features represented by the additional feature data output by the first machine learning network include at least one feature related to lumen properties of the vessel of interest along an axial trajectory of the vessel of interest, such as lumen area and / or lumen attenuation.

[0336] A5. A method according to clause A1, A2 or A3, wherein:

[0337] The additional features represented by the additional feature data output by the first machine learning network include at least one feature related to plaque characteristics of the vessel of interest along the axial trajectory of the vessel of interest (such as calcium plaque area, soft plaque area, mixed plaque area).

[0338] A6. A method according to any one of clauses A1 to A5, the method also involving generating myocardial characteristic data characterizing a local portion of the myocardium associated with the vessel of interest, and supplying the myocardial characteristic data as input to a second machine learning network for generating data characterizing the FFR pullback of the vessel of interest.

[0339] A7. A method according to any one of clauses A1 to A6, wherein:

[0340] The first machine learning network includes a variational autoencoder having an encoder portion that generates a plurality of latent space encodings, wherein the encoder portion is trained using unsupervised learning.

[0341] A8. A method according to clause A7, wherein:

[0342] The first feature data is input to the convolution block of the encoder part.

[0343] A9. A method according to clause A7, wherein:

[0344] The first machine learning network includes at least one auxiliary decoder portion that is supplied with a subset of the latent space encodings generated by the encoder portion and is configured to generate additional feature data given the subset of the latent space encodings as input.

[0345] A10. A method according to clause A7, wherein:

[0346] At least one auxiliary decoder portion is trained by supervised learning using training data including reference annotations based on measurements or extractions of corresponding features for a plurality of patients.

[0347] A11. A method according to any one of clauses A1 to A10, wherein:

[0348] The second machine learning network is trained by supervised learning using training data including reference annotations based on measurements of FFR pullback or FFR dropoff associated with vessel centerline points for multiple patients.

[0349] A12. A method according to any one of clauses A1 to A11, wherein:

[0350] The second machine learning network is further configured to output an FFR value of the blood vessel; and

[0351] The second machine is trained by supervised learning using training data including reference annotations based on measurements of FFR values ​​associated with blood vessels for a plurality of patients.

[0352] A13. A method according to any one of clauses A1 to A12, wherein:

[0353] The second machine learning network is further configured to output data representing a prediction of the presence of functionally significant stenosis; and

[0354] A second machine learning network is trained by supervised learning using training data including reference annotations for a plurality of patients indicating the presence of functionally significant stenosis.

[0355] A14. A method according to any one of clauses A1 to A13, wherein:

[0356] The second machine learning network includes a convolutional neural network, which is trained by supervised learning using training data, and the training data includes reference annotations for output data of the second machine learning network.

[0357] A15. A method according to clause A14, wherein:

[0358] The reference annotations are derived by manually segmenting the corresponding volumetric image data and / or automatically segmenting the corresponding volumetric image data.

[0359] A16. A method according to clause A14, wherein:

[0360] The convolutional neural network of the second machine learning system includes a regression head and an output stage, wherein the regression head generates an FFR drop along an axial trajectory of the vessel of interest, and the output stage generates an FFR pullback output by the second machine learning system.

[0361] A17. A method according to clause A14, wherein:

[0362] The convolutional neural network of the second machine learning system also includes a first classification head, which outputs data representing the FFR value of the blood vessel.

[0363] A18. A method according to clause A14, wherein:

[0364] The convolutional neural network of the second machine learning system also includes a second classification head that outputs data representing a prediction of the presence of functionally significant stenosis.

[0365] A19. A method according to any one of clauses A1 to A18, wherein:

[0366] The vessel of interest includes a coronary artery or a coronary arterial tree.

[0367] A20. A method according to any one of clauses A1 to A19, wherein:

[0368] The volume image dataset includes CCTA image data.

[0369] A21. A system for assessing obstruction of a blood vessel of interest in a patient, the system comprising:

[0370] At least one processor configured to perform some or all of the operations described in clauses A1 to A20 when executing program instructions stored in the memory.

[0371] A22. A system according to clause A21, further comprising:

[0372] The imaging acquisition subsystem is configured to acquire a volumetric image dataset.

[0373] A23. A system according to clause A22, further comprising:

[0374] The display subsystem is configured to display data representative of the severity of anatomical lesions of the blood vessel of interest.

[0375] A24. A non-transitory program storage device tangibly embodying a program of instructions executable on a machine to perform any or some of the operations for assessing occlusion of a vessel of interest in a patient according to clauses A1 to A20.

[0376] Clause Set B1:

[0377] Embodiments disclosed herein may provide methods and systems involving simulating or planning an interventional treatment of an occlusion of a vessel of interest of a patient, the methods and systems comprising one, some, or all of the following operations:

[0378] obtaining a volume image dataset for a blood vessel of interest;

[0379] analyzing the volumetric image data set to extract data representing an axial trajectory of a vessel of interest;

[0380] generating a multi-planar reformatted (MPR) image based on the volumetric image dataset and data representing an axial trajectory of the vessel of interest;

[0381] supplying the MPR image to a first machine learning network, the first machine learning network outputting, given the MPR image, i) a plurality of latent space encodings characterizing features of the vessel of interest along the axial trajectory of the vessel of interest and ii) additional feature data characterizing additional features of the vessel of interest along the axial trajectory of the vessel of interest; and

[0382] adjusting some or all of the additional feature data based on a simulated or planned treatment of the vessel of interest;

[0383] The plurality of latent space encodings and adjusted additional feature data output by the first machine learning network are supplied to a second machine learning network, which, given the input data, outputs data characterizing the FFR pullback of the vessel of interest taking into account a simulated or planned treatment of the vessel of interest.

[0384] The operation of clause B1 may be followed by any or some of the operations according to clauses A1 to A20 above.

[0385] Clause Set B2:

[0386] Embodiments disclosed herein may provide methods and systems involving simulating or planning an interventional treatment of an occlusion of a vessel of interest of a patient, the methods and systems comprising one, some, or all of the following operations:

[0387] obtaining a volume image dataset for a blood vessel of interest;

[0388] analyzing the volumetric image data set to extract data representing an axial trajectory of a vessel of interest;

[0389] generating a multiplanar reformatted (MPR) image based on the volumetric image dataset and data representing an axial trajectory of a vessel of interest;

[0390] Adjusting the MRP image based on simulated or planned treatment of the vessel of interest;

[0391] supplying the adjusted MPRG image to a first machine learning network, which, given the MPR image, outputs i) a plurality of latent space encodings characterizing features of the vessel of interest along the axial trajectory of the vessel of interest and ii) additional feature data of additional features of the vessel of interest along the axial trajectory of the vessel of interest; and

[0392] The plurality of latent space encodings and additional feature data output by the first machine learning network are supplied to a second machine learning network, which, given the input data, outputs data characterizing the FFR pullback of the vessel of interest taking into account a simulated or planned treatment of the vessel of interest.

[0393] The operation of clause B2 may be followed by any or some of the operations according to clauses A1 to A20 and / or B1 above.

[0394] B3. The method of clause B1 or B2, further comprising displaying or outputting data characterizing FFR pullback of the vessel of interest taking into account a simulated or planned treatment of the vessel of interest.

[0395] B4. The method according to clause B1 or B2 or B3 further involves generating first feature data characterizing the presence of zero or more bifurcations or side branches along the axial trajectory of the vessel of interest, and supplying the first feature data for input to a first machine learning system for generating multiple latent space encodings and additional feature data.

[0396] B5. A method according to clause B4, wherein:

[0397] The first characteristic data is generated by analyzing the MPR image; and / or

[0398] The first characteristic data is generated by analyzing the volume image data set; and / or

[0399] The first feature data is generated by a coronary artery centerline tree derived from a volumetric image dataset.

[0400] B6. A method according to any one of clauses B1 to B5, wherein:

[0401] The additional features represented by the additional feature data output by the first machine learning network (and capable of being adjusted by the method in B1) include at least one feature related to the luminal properties of the vessel of interest along the axial trajectory of the vessel of interest (such as luminal area and / or luminal attenuation).

[0402] B7. A method according to any one of clauses B1 to B6, wherein:

[0403] The additional features represented by the additional feature data output by the first machine learning network (and capable of being adjusted by the method in B1) include at least one feature related to the plaque characteristics of the vessel of interest along the axial trajectory of the vessel of interest (such as calcium plaque area, soft plaque area, mixed plaque area).

[0404] B8. A method according to any one of clauses B1 to B7, the method further involving generating myocardial characteristic data characterizing a local portion of the myocardium associated with the vessel of interest, and supplying the myocardial characteristic data as input to a second machine learning network for generating data characterizing the FFR pullback of the vessel of interest.

[0405] B9. A method according to any one of clauses B1 to B8, wherein:

[0406] The first machine learning network includes a variational autoencoder having an encoder portion that generates a plurality of latent space encodings, wherein the encoder portion is trained using unsupervised learning.

[0407] B10. A method according to clauses B1 to B9, wherein:

[0408] The first feature data of clause B3 is input to the convolution block of the encoder part.

[0409] B11. A method according to clauses B1 to B9, wherein:

[0410] The first machine learning network includes at least one auxiliary decoder portion that is supplied with a subset of the latent space encodings generated by the encoder portion and is configured to generate additional feature data given the subset of the latent space encodings as input.

[0411] B12. A method according to clauses B1 to B9, wherein:

[0412] At least one auxiliary decoder portion is trained by supervised learning using training data including reference annotations based on measurements or extractions of corresponding features for a plurality of patients.

[0413] B13. A method according to any one of clauses B1 to B12, wherein:

[0414] The second machine learning network is trained by supervised learning using training data including reference annotations based on measurements of FFR pullback or FFR dropoff associated with vessel centerline points for multiple patients.

[0415] B14. A method according to any one of clauses B1 to B13, wherein:

[0416] The second machine learning network is further configured to output an FFR value of the blood vessel; and

[0417] The second machine is trained by supervised learning using training data including reference annotations based on measurements of FFR values ​​associated with blood vessels for a plurality of patients.

[0418] B15. A method according to any one of clauses B1 to B14, wherein:

[0419] The second machine learning network is further configured to output data representing a prediction of the presence of functionally significant stenosis; and

[0420] A second machine learning network is trained by supervised learning using training data including reference annotations for a plurality of patients indicating the presence of functionally significant stenosis.

[0421] B16. A method according to any one of clauses B1 to B15, wherein:

[0422] The second machine learning network includes a convolutional neural network, which is trained by supervised learning using training data, and the training data includes reference annotations for output data of the second machine learning network.

[0423] B17. A method according to clause B16, wherein:

[0424] The reference annotations are derived by manually segmenting the corresponding volumetric image data and / or automatically segmenting the corresponding volumetric image data.

[0425] B18. A method according to clause B16 or B17, wherein:

[0426] The convolutional neural network of the second machine learning system includes a regression head and an output stage, wherein the regression head generates an FFR drop along an axial trajectory of the vessel of interest, and the output stage generates an FFR pullback output by the second machine learning system.

[0427] B19. A method according to clauses B16 to B18, wherein:

[0428] The convolutional neural network of the second machine learning system also includes a first classification head, which outputs data representing the FFR value of the blood vessel.

[0429] B20. A method according to clauses B16 to B19, wherein:

[0430] The convolutional neural network of the second machine learning system also includes a second classification head that outputs data representing a prediction of the presence of functionally significant stenosis.

[0431] B21. A method according to any one of clauses B1 to B20, wherein:

[0432] The vessel of interest includes a coronary artery or a coronary arterial tree.

[0433] B22. A method according to any one of clauses B1 to B21, wherein:

[0434] The volume image dataset includes CCTA image data.

[0435] B23. A system for assessing obstruction of a vessel of interest in a patient, the system comprising:

[0436] At least one processor is configured to perform any or some of the operations described in clauses B1 to B22 when executing program instructions stored in the memory.

[0437] B24. A system according to clause B23, further comprising:

[0438] The imaging acquisition subsystem is configured to acquire a volumetric image dataset.

[0439] B25. A system according to clause B23, further comprising:

[0440] The display subsystem is configured to display data representative of the severity of anatomical lesions of the blood vessel of interest.

[0441] B26. A non-transitory program storage device tangibly embodying a program of instructions capable of being executed on a machine to perform any or some of the operations described in clauses B1 to B22 involving simulation or planning of treatment of an occlusion of a vessel of interest of a patient.

[0442] Clause Set C1:

[0443] Embodiments disclosed herein may provide methods and systems for extracting a coronary artery tree from volumetric image data of a vessel of interest of a patient, the methods and systems comprising one, some, or all of the following operations:

[0444] obtaining a volume image dataset for a blood vessel of interest;

[0445] Track multiple seed points in an image dataset;

[0446] using multiple seed points to extract an initial representation of the coronary artery tree in an image dataset;

[0447] Inputting the initial representation of the coronary artery tree to a first set of graph convolutional neural networks to generate a refined representation of the coronary artery tree; and

[0448] A second set of graph convolutional neural networks is used to generate labels for segments of the refined representation of the coronary artery tree.

[0449] C2. A method according to clause C1, wherein:

[0450] Initial and Refined Representations of the Coronary Artery Tree The coronary artery tree is represented as an undirected dendrogram, where each point in the centerline of a coronary artery segment corresponds to a node in the dendrogram, and connections between centerline points are represented by undirected edges in the dendrogram.

[0451] C3. A method according to clause C1 or C2, wherein:

[0452] An initial representation of the coronary artery tree is constructed by tracing the coronary artery centerlines from seed points.

[0453] C4. A method according to clause C1 or C2 or C3, wherein:

[0454] An initial representation of the coronary artery tree was derived by predicting the locations of seed points and coronary artery ostia using two convolutional neural networks.

[0455] C5. A method according to any one of clauses C1 to C4, wherein:

[0456] An initial representation of the coronary artery tree was derived by adding new nodes to the tree using a convolutional neural network configured to predict direction and step length through one or more terminal nodes of the graph (nodes with fewer than two edges) to generate synthetic subgraphs, and merging subgraphs when there was overlap.

[0457] C6. A method according to any one of clauses C1 to C5, wherein:

[0458] The initial representation of the coronary artery tree was derived by grouping adjacent centerline points of the dendrogram to create segments.

[0459] C7. A method according to clause C6, wherein:

[0460] The segment is characterized using a set of features selected from the group consisting of: a location of the segment, an orientation of the segment, a geometric shape of the segment, or an image appearance of the segment.

[0461] C8. A method according to any one of clauses C1 to C7, wherein:

[0462] The first set of graph convolutional neural networks is configured to perform binary classification to distinguish true coronary artery segments (positive class) from other vessel-like structures (negative class),

[0463] C9. A method according to any one of clauses C1 to C8, wherein:

[0464] The first set of graph convolutional neural networks is configured to adopt a multi-resolution graph aggregation strategy in which predictions for multiple graphs at different resolutions are back-projected to a fine graph.

[0465] C10. A method according to any one of clauses C1 to C9, wherein:

[0466] The second set of graph convolutional neural networks is trained on graphs with different resolutions for anatomical labeling.

[0467] C11. The method according to any of clauses C1 to C10, further comprising displaying or outputting a refined representation of the coronary artery tree and / or labels for segments of the coronary artery tree.

[0468] C12. A system for extracting a coronary artery tree from volumetric image data of a vessel of interest of a patient, the system comprising:

[0469] At least one processor configured to perform any or some of the operations described in clauses C1 to C12 when executing program instructions stored in the memory.

[0470] C13. A system according to clause C3, further comprising:

[0471] The imaging acquisition subsystem is configured to acquire a volumetric image dataset.

[0472] C14. A system according to clause C13, further comprising:

[0473] A display subsystem configured to display according to any of clauses C1 to C10 further relates to displaying or outputting a refined representation of the coronary artery tree and / or labels for segments of the coronary artery tree.

[0474] C15. A non-transitory program storage device tangibly embodying a program of instructions executable on a machine to perform any or some of the operations described in clauses C1 to C11 to extract a coronary artery tree from volumetric image data.

[0475] All features appearing in the above clause groups may be combined between them and with any features appearing in the attached claims.

Claims

1. A method for assessing obstruction of a blood vessel of interest in a patient, comprising: obtaining a volume image dataset for the blood vessel of interest; analyzing the volumetric image dataset to extract data representing an axial trajectory of the vessel of interest; generating a multi-planar reformatted (MPR) image based on the volumetric image dataset and the data representing the axial trajectory of the vessel of interest; supplying the MPR image as input to a first machine learning network, the first machine learning network outputting feature data given the MPR image, the feature data characterizing a plurality of features of the vessel of interest along an axial trajectory of the vessel of interest; generating additional data, by separate and distinct analysis from the first machine learning network, the additional data characterizing at least one additional feature of the vessel of interest along an axial trajectory of the vessel of interest; as well as The data output by the first machine learning network and the additional data are supplied as input data to a second machine learning network, which outputs data characterizing the severity of the anatomical lesion of the blood vessel of interest given the input data.

2. The method according to claim 1, further comprising: Data representative of the severity of the anatomical lesion of the vessel of interest is displayed or output.

3. The method according to claim 1 or 2, wherein: The additional data is generated by analyzing the MPR image; and / or The additional data is generated by analysis of the volumetric image dataset; and / or The additional data is generated by deriving a coronary artery centerline tree from the volumetric image dataset.

4. A method according to any one of the preceding claims, wherein: The additional data characterizes at least one of side branches and bifurcations along an axial trajectory of the vessel of interest.

5. A method according to any one of the preceding claims, wherein: The additional data characterizes at least one of a soft plaque area, a mixed plaque area, or other characteristic features along an axial trajectory of the vessel of interest.

6. A method according to any one of the preceding claims, wherein: The additional data also characterizes a localized portion of the myocardium associated with the vessel of interest.

7. A method according to any one of the preceding claims, wherein: The data output by the second machine learning network includes a fractional flow reserve (FFR) value for the entire vessel of interest; and The second machine learning network is trained by supervised learning using training data including reference annotations based on measurements of FFR values ​​for a plurality of patients.

8. A method according to any one of the preceding claims, wherein: The data output by the second machine learning network includes fractional flow reserve (FFR) values ​​for centerline points along the vessel of interest; and The second machine learning network is trained by supervised learning using training data including reference annotations based on measurements of FFR values ​​associated with vessel centerline points for a plurality of patients.

9. A method according to any one of the preceding claims, wherein: The data output by the second machine learning network represents a prediction of the presence of functionally significant stenosis; and The second machine learning network is trained by supervised learning using training data including reference annotations for a plurality of patients indicating the presence of functionally significant stenosis.

10. A method according to any one of the preceding claims, wherein: The plurality of features represented by the feature data output by the first machine learning network include at least one feature related to luminal properties of the vessel of interest (such as luminal area and / or luminal attenuation) along an axial trajectory of the vessel of interest.

11. A method according to any one of the preceding claims, wherein: The multiple features represented by the feature data output by the first machine learning network include at least one feature related to the plaque characteristics of the vessel of interest (such as calcium plaque area, soft plaque area, mixed plaque area) along the axial trajectory of the vessel of interest.

12. A method according to any one of the preceding claims, wherein: The first machine learning network comprises a convolutional neural network trained using training data comprising reference annotations for the plurality of features represented by feature data output by the first machine learning network.

13. The method according to claim 12, wherein: The reference annotations are derived by manually segmenting the corresponding volume image data and / or automatically segmenting the corresponding volume image data.

14. A method according to any one of the preceding claims, wherein: The second machine learning network comprises a convolutional neural network trained using training data comprising volumetric image data and corresponding reference annotations for output data characterizing a severity of an anatomical lesion of the vessel of interest.

15. The method according to claim 14, wherein: The reference annotations are derived by manually segmenting the corresponding volume image data and / or automatically segmenting the corresponding volume image data.

16. The method of claim 14, wherein: The convolutional neural network of the second machine learning system includes a regression head that outputs a fractional flow reserve (FFR) value.

17. The method of claim 16, wherein: The convolutional neural network of the second machine learning system also includes an accumulator that outputs fractional flow reserve (FFR) values ​​for centerline points along the vessel of interest.

18. A method according to any one of claims 14 to 17, wherein: The convolutional neural network of the second machine learning system includes a classification head that outputs data representing a prediction of the presence of functionally significant stenosis.

19. A method according to any one of the preceding claims, wherein: The vessel of interest includes a coronary artery or a coronary arterial tree.

20. A method according to any one of the preceding claims, wherein: The volume image dataset includes CCTA image data.

21. A system for assessing occlusion of a blood vessel of interest in a patient, the system comprising: At least one processor is configured to perform some or all of the operations according to claims 1 to 20 when executing program instructions stored in the memory.

22. The system of claim 21, further comprising: The imaging acquisition subsystem is configured to acquire the volume image dataset.

23. The system of claim 22, further comprising: The display subsystem is configured to display data representing the severity of the anatomical lesion of the blood vessel of interest.

24. A non-transitory program storage device tangibly embodying a program of instructions executable on a machine to perform some or all of the operations for assessing occlusion of a vessel of interest in a patient according to claims 1 to 20.

Images