Machine Learning Approach for 3D Reconstruction of Coronary Arteries from X-ray Angiography Images

Abstract

A method for performing 3D vascular tree reconstruction includes providing segmented binary angiographic images, applying distance transformations to the images, and generating distance-transformed binary angiographic images. The set of distance-transformed binary angiographic images is provided to a trained 3D vascular tree reconstruction machine learning model capable of reconstructing 3D vessels. The 3D vascular tree reconstruction machine learning model includes a multi-stage convolutional neural network, which includes a multi-stage architecture having (i) a vascular centerline stage and (ii) a radius reconstruction stage. The resulting 3D reconstructed vascular tree can be used to perform clinical assessments of coronary artery health and occlusion.

Description

Technical Field

[0001] The present invention generally relates to three-dimensional (3D) image reconstruction of objects, and more specifically, to the 3D reconstruction of coronary artery trees using angiographic images and multi-layer neural network machine learning models.

Background Art

[0002] The description of the background art provided herein is for the purpose of generally presenting the context of the present disclosure. The research of the inventors named herein as of the filing date, and aspects of the description that may not otherwise be considered prior art at the time of filing, are not admitted to be prior art to the present disclosure, either expressly or implicitly.

[0003] The three-dimensional (3D) reconstruction of structures and objects is widely used across a broad range of industries, including, among other things, materials research, urban mapping, gaming, virtual environments, medicine, and medical research. Each application of 3D reconstruction has its own difficulties and drawbacks. For example, urban mapping may use satellite or drone imagery that lacks the desired depth resolution to properly reconstruct smaller structures, or in gaming, the movement of people or actors may be reconstructed in 3D with its own unique difficulties.

[0004] Specifically, 3D reconstruction can be useful in developing physiologically based quantitative approaches for determining organ function, the state of vascular function, diagnosing one or more diseases or disorders, or monitoring organs and structures during treatment, intervention, or surgery. Coronary artery disease (CAD) is one of the leading causes of death in the United States, affecting over 15 million Americans. CAD is characterized by the accumulation of plaque due to atherosclerotic changes in the coronary arteries, which can cause narrowing (also known as stenosis) or occlusion of the coronary arteries and potentially lead to symptoms such as angina and myocardial infarction.

[0005] In this specification, “vascular occlusion” refers to what is generally understood as CAD, for example, narrowing (localization or diffusion) of the epicardial coronary arteries as seen in imaging studies and characterized by either anatomical or functional indicators. Conversely, “microvascular disease” refers to a disease of the coronary microcirculation characterized by loss of vasodilation capacity.

[0006] The primary invasive diagnostic method for CAD is coronary angiography, in which a contrast agent is injected into the patient's blood vessels via a catheter and imaged to characterize the severity of the stenosis. This method relies on the visualization of anatomical abnormalities, and the visual inspection is semi-quantitative as it merely estimates the percentage reduction in lumen area. A diameter reduction of 70% or more often leads to further evaluation or revascularization, such as coronary stenting.

[0007] A more physiologically-based quantitative approach to assessing coronary artery stenosis involves calculating the fractional flow reserve (FFR), a metric defined as the ratio of the engorged blood flow in the diseased artery to the expected engorged blood flow in the same artery without stenosis. For example, in the coronary arteries, FFR can be expressed as the ratio of distal coronary pressure to proximal coronary pressure. For instance, an FFR of less than 0.80 indicates the presence of severe stenosis requiring revascularization because blood flow to the distal vascular bed is impaired. Decisions regarding revascularization incorporating FFR have been shown to improve outcomes compared to angiography alone. FFR determination is robust to various patient geometries, taking into account, for example, the contribution of collateral vessels to the geometry of blood flow and lesions.

[0008] Despite its advantages, the invasive nature of using catheter-based pressure measurements means that healthcare professionals often do not measure patients' FFR. Some have found that physicians choose not to perform FFR in nearly two-thirds of all cases due to the risks to the patient, lack of resources, and additional costs. Another drawback of FFR is its variability due to the varying hemodynamic states within patients.

[0009] More accurate and less invasive techniques are needed to diagnose CAD. More specifically, there is a need for non-invasive and user-independent approaches to FFR measurement that would not pose a risk to the patient.

[0010] Recently, some have proposed non-invasive computational workflows to determine FFR and assess the severity of CAD. These efforts employ one of two different approaches. The first approach relies on computed tomography angiography (CTA) data to reconstruct the 3D geometry of the coronary artery tree. While this non-invasive approach has shown promising results in clinical trials, the use of CTA is heavily influenced by imaging artifacts due to calcification, making it difficult to depict the boundaries of the vascular lumen. Due to its lower spatial resolution compared to angiography, CTA also has limitations in its ability to detect small continuous lesions or capture fine vascular geometry. Finally, because CTA data does not provide information about blood flow, the boundary conditions (BCs) for hemodynamic CFD analysis are usually not patient-specific, as they rely on morphometric or population data.

[0011] A second approach, also based on a non-invasive computational workflow, reconstructs the 3D geometry of the vascular tree by relying on multiplane angiography data before performing a physics-based blood flow simulation. The main advantages of using angiography are its superior ability to detect vascular lumen boundaries in the presence of calcified stenosis and its higher spatial resolution compared to CTA, which improves the sensitivity of the geometric reconstruction algorithm. However, angiography-based approaches for FFR quantification have a fundamental challenge: reconstructing the 3D geometry of the vessel of interest from a series of 2D images acquired at various positions on the patient table over time. Furthermore, all angiography-based approaches for FFR quantification have created workflows that require substantial input from the operator to identify and assist in segmentation of the vessel of interest. Finally, all angiography-based approaches have considered 3D reconstruction of a single coronary artery or modeled the physics of blood flow using a highly simplified method. These drawbacks effectively negate the advantages of 3D reconstruction of the vascular tree using high-resolution angiography data, which is the most commonly performed procedure in CAD diagnosis.

[0012] Regardless of the approach, all calculation-derived FFR methods have shown poor predictive performance around the critical diagnostic value of FFR=0.8 due to the aforementioned limitations in image data quality, lack of information on blood flow, the need for operator input, and computational modeling assumptions. Therefore, there is currently no pipeline for accurate 3D reconstruction-based FFR calculation that can be effectively deployed to any hospital nationwide.

[0013] There is a critical need for more accurate and less invasive techniques for CAD diagnostics, specifically for 3D object reconstruction for use in medical diagnosis, using a non-invasive approach for fully automated FFR measurement. [Overview of the project]

[0014] A technique is provided for performing 3D reconstruction of the coronary vascular tree. The 3D reconstruction is generated using a trained multi-stage neural network. The accuracy of the obtained 3D reconstructed coronary vascular tree can be used to perform anatomical and functional assessments of coronary artery disease (CAD) using machine learning and computational modeling techniques. The technique described herein addresses the shortcomings of projection-based approaches and other types of 3D reconstruction, such as other machine learning methods. The obtained accurate 3D reconstruction may also be useful for non-invasive fractional flow reserve (FFR) quantification, relying on accurate physics-based computational fluid dynamics (CFD) simulations performed using the obtained 3D reconstructed vessels and vascular tree, based on angiographically derived anatomical and hemodynamic data.

[0015] In exemplary embodiments, the technology provides a process for training and utilizing a multi-stage neural network ML model to perform 3D reconstruction of a coronary vascular tree. The ML model is trained by generating a synthetic coronary vascular tree derived from a statistical model of 3D image data, such as magnetic resonance angiography and / or computed tomography angiography image data; generating binarized angiographic images from different conical projections of the synthetic vascular tree; and using the binarized angiographic images to train the ML model to perform 3D reconstruction from the binarized angiographic images (e.g., clinical angiographic images).

[0016] For example, a computer implementation method for training a vascular tree reconstruction system includes selecting, for each synthetic vascular tree in a set of synthetic vascular trees, a set of three or more sets of binary angiographic images corresponding to three or more different projection angles. The method includes applying a Euclidean distance transform to the binary angiographic images to encode the two-dimensional (2D) diameter of each branch, thereby generating a Euclidean distance transformed angiographic image. Finally, using the Euclidean transformed angiographic image, a 3D vascular tree reconstruction machine learning model is trained that can reconstruct 3D vessels, and the training of the 3D vascular tree reconstruction machine learning model includes a multi-stage convolutional neural network comprising a multi-stage architecture having (i) a vascular centerline stage and (ii) a radius reconstruction stage.

[0017] Another example is a method for performing 3D vascular tree reconstruction, comprising: providing segmented binary angiographic images to a preprocessing module; the preprocessing module applying a Euclidean distance transform to the binary angiographic images to encode the two-dimensional (2D) diameter of each branch to generate a Euclidean distance transformed angiographic image; and feeding the set of Euclidean distance transformed angiographic images to a 3D vascular tree reconstruction machine learning model capable of reconstructing 3D vessels, wherein the trained 3D vascular tree reconstruction machine learning model comprises a multi-stage convolutional neural network comprising a multi-stage architecture having (i) a vascular centerline stage and (ii) a radius reconstruction stage. [Brief explanation of the drawing]

[0018] This patent or application file includes at least one drawing made in color. A copy of this patent or patent application publication containing the color drawing will be provided by the United States Patent and Trademark Office upon request and payment of the required fees.

[0019] The drawings described below illustrate various aspects of the systems and methods disclosed herein. Each drawing shows an embodiment of a particular aspect of the disclosed systems and methods, and it should be understood that each drawing is intended to correspond to a possible embodiment. Furthermore, wherever possible, the following description refers to the reference numerals included in the following drawings, and features shown in multiple drawings are designated by consistent reference numerals.

[0020] [Figure 1] This is a schematic diagram of an exemplary system for performing anatomical and functional assessments of coronary artery disease (CAD) using a machine learning framework to generate a 3D reconstruction of the vascular tree, as an example. [Figure 2] This is a schematic diagram of a 3D reconstruction multi-stage neural network machine learning algorithm and training module framework, which may be implemented in the system shown in Figure 1, as an example. [Figure 3A] This is an example of a process for generating a synthetic vascular tree and using it to train a 3D reconstructed multi-stage neural network machine learning model. [Figure 3B] This document illustrates the process for training a 3D reconstruction multi-stage neural network machine learning model as an example. [Figure 4] This document illustrates the process for performing a 3D reconstruction of the coronary vascular tree using a trained ML model, which may be executed by the system shown in Figure 1. [Figure 5] This example illustrates both high-fidelity or 3D representations and low-fidelity or 1D representations of coronary vascular trees. [Figure 6] This is a schematic diagram of an exemplary implementation architecture of a neural network with angiography input and output 3D reconstructed vascular trees, as an example. [Figure 7] An example of a synthetic 3D coronary artery tree generator is provided below. [Figure 8] This example illustrates a process of generating binary angiographic images by performing multiple projections of a synthetic vascular tree. [Figure 9] This paper illustrates an example of a vascular geometry with 70% stenosis, as well as a comparison of single-stage and multi-stage model reconstructions. [Figure 10] This paper presents a performance comparison between multi-stage neural network reconstruction and projection-based methods in the reconstruction of a given set of sample 3D synthetic blood vessels and corresponding projection images (distant from 15 degrees), using one example. [Figure 11] This example shows several examples of the centerlines of the coronary artery trees and the corresponding ground truth centerlines reconstructed from synthetic trees. [Figure 12] This example shows a radial reconstruction along the normalized midline position of the main right coronary artery, without stenosis, single stenosis, or two stenosis. [Figure 13] We summarize the results of ground truth and reconstructed coronary tree pressure measurements using one example. [Modes for carrying out the invention]

[0021] A technique is provided for performing three-dimensional (3D) reconstruction of an object using a multi-stage neural network machine learning (ML) model. In the specific example described, 3D reconstruction may be implemented to perform anatomical and functional assessments of the state of occlusion within a blood vessel (e.g., to assess coronary artery disease (CAD)). Multi-stage neural network 3D reconstruction of a coronary artery tree performs reconstruction from uncalibrated two-dimensional (2D) radiographic angiography images. Multiple images of a single vascular tree are taken, each image at a different angle or viewpoint of the vascular tree. The images are typically binarized without knowledge of image acquisition parameters (e.g., camera distance from the vessel, viewpoint at a specific angle, relative angle of the images, etc.), as required by other 3D reconstruction techniques such as projection-based reconstruction, and the reconstruction is performed from the binarized images.

[0022] The input to the multi-stage neural network (ML) model is two, three, or more binarized angiograms for a given coronary artery tree, to which a Euclidean distance transformation is applied to create a smooth field that implicitly encodes the vessel radius. To train the ML model, the binarized angiograms were created by a synthetic angiography generator, also known as a synthetic data generator. Since no angular or distance information is provided to the neural network as input, the demonstrated 3D reconstruction method does not require image calibration or parameter correction algorithms.

[0023] The disclosed 3D reconstruction method utilizes a single backbone network and separate steps for the reconstruction of separate vascular centerlines and vascular radii. The ML model outputs an analytical matrix representation of the coronary artery tree, which can be used for further analysis and application, such as hemodynamic modeling of local vascular stenosis (i.e., stenosis). As used herein, the state of vascular occlusion refers to the determination of vascular health, including an assessment of vascular health based on anatomical features (determined, e.g., from angiographic images) and functional features (determined, e.g., using physical models and including flow data). Thus, as used herein, the state of occlusion should be interpreted as including, but not limited to, various hemodynamic indicators such as the degree of stenosis, the length of stenosis, FFR, and combinations thereof.

[0024] The described method involves training a neural network ML model using a dataset of synthetic coronary artery trees from an angiogenizer that utilizes both clinical imaging data (e.g., MRA and CTA image data) and literature values ​​regarding coronary artery anatomy. While training is described as being performed using synthetic angiography, the ML model may be trained using images of clinically acquired images of the angiography trees and validated using 3D reconstruction of clinically acquired images. The described multi-stage neural network ML model can achieve sub-pixel accuracy in anatomy radius reconstruction with a root mean squared error (RMSE) of 0.16 ± 0.07 mm and a mean absolute error (MAE) of 0.27 ± 0.18 mm, which is a very important feature often used in diagnosis. Furthermore, the multi-stage neural network ML model reduces anatomy centerline reconstruction error by 52% and 38%, respectively, compared to single-stage neural network methods and projection geometry-based methods. The described method is robust to challenges faced by other 3D reconstruction methods, such as anatomy shortening and overlap in input images. The described ML model is a step toward automated analysis of the severity of anatomical and functional diseases in the coronary arteries.

[0025] In some examples, systems and methods are provided for training an ML model using binary angiographic images to reconstruct a vascular tree. Training involves using images from multiple views of a synthetic vascular tree, applying a Euclidean distance transform to encode the 2D diameter of each branch of the vascular tree, and training the ML model by a multi-stage convolutional neural network. The multi-stage convolutional neural network includes at least a vascular tree centerline stage and a radius reconstruction stage. The multi-stage convolutional neural network may include three stages: (i) a first stage containing a classification convolutional neural network backbone; (ii) a centerline stage containing a multilayer perceptron (MLP) with ReLU activation and inter-layer batch normalization of the MLP for centerline reconstruction; and (iii) a radius reconstruction stage containing an MLP with ReLU activation and inter-layer batch normalization of the MLP for radius reconstruction. The output of the second stage is Mi ×N i is a 3×3 matrix, where 3 represents the dimension of the centerline coordinates in the orthogonal coordinates for each point N i of each branch M i of each 3D vascular tree i. The third stage is performed for each branch M i and the output of the third stage is a N i *1 matrix of the radius for each point N i of each centerline.

[0026] The disclosed method further includes performing vascular tree reconstruction by utilizing an ML model trained according to the method described herein. For example, a user or system may provide a plurality of 2D angiography images of a vascular tree, each image corresponding to a different projection angle of the vascular tree, and the ML model may generate a dataset indicative of a 3D reconstruction of the vascular tree. The dataset indicative of the 3D reconstruction of the vascular tree may include one or more M i ×N i ×4 matrices, where M i is the number of branches within the vascular tree, where each branch has a branch centerline, and N i is the number of points on each branch centerline, and 4 is the dimension of the data encoded at each point of the branch centerline, specifically including three spatial dimensional coordinates and a radius.

[0027] ​​In Figure 1, the CAD evaluation system 100 includes a computing device 102 (or "signal processor" or "diagnostic device") configured to collect angiographic image data from a patient 120 via an angiography imaging device 124 by performing the functions of the disclosed embodiments. The CAD evaluation system may be used to carry out the training and implementation of the multi-stage neural network 3D reconstruction method described herein. As shown in the figure, the system 100 may be implemented on the computing device 102, specifically on one or more processing units 104 which may represent a central processing unit (CPU), and / or on one or more graphics processing units (GPUs) including a cluster of CPUs and / or GPUs, any of which may be cloud-based. The features and functions described for the system 100 may be stored in and implemented from one or more non-temporary computer-readable media 106 of the computing device 102. The computer-readable media 106 may include, for example, an operating system 108 and a CAD machine learning (deep learning and / or neural network) framework 110 having elements corresponding to elements of the deep learning framework described herein. More generally, the computer-readable medium 106 may store trained deep learning models, including vascular segmentation machine learning models, flow extraction machine learning models, low-dimensional models based on graph theory or other neural networks, and executable code, used to implement the techniques of this specification. Additionally, the computer-readable medium 106 may store executable instructions for training ML models such as deep learning models and / or neural network models as described herein. The computer-readable medium 106 and the processing unit 104 may store image data, segmentation models or rules, fluid dynamic classifiers, datasets showing 3D reconstruction of vascular trees, synthetic vascular trees, and other data specified herein in one or more databases 112.As discussed in the examples herein, a CAD machine learning framework 110 applying the techniques and processes herein (e.g., various different neural networks) may generate 3D and / or 1D segmented vascular dendritic models, FFR and other fluid dynamic assessments, vascular occlusion status data (such as degree of stenosis), and / or microvascular disease data.

[0028] The computing device 102 includes a network interface 114 that is communicably coupled to a network 116 for communicating with and / or other computing devices such as portable personal computers, smartphones, electronic documents, tablets, and / or desktop personal computers. The computing device further includes an I / O interface 115 connected to devices such as a digital display 118 and a user input device 122. As described herein, the computing device 102 generates a display of a subject CAD, which may include vascular conditions in the vascular system, such as anatomical and functional vascular occlusion (via FFR calculation, iFR calculation, or QFR calculation) and microvascular disease prediction (by comparing changes in peripheral resistance when two hemodynamic conditions are recorded), as an electronic document that can be accessed and / or shared on the network 116.

[0029] In the illustrated example, the computing device 102 is communicably connected to the electronic medical record (EMR) database 126 via the network 116. The EMR 126 may be a network-accessible database or a dedicated processing system. In some examples, the EMR 126 contains data for each of one or more patients. The EMR data may include vital sign data (e.g., hemoglobin oxygen saturation derived from pulse oximetry, heart rate, blood pressure, respiratory rate), laboratory data such as whole blood count (e.g., mean platelet volume, hematocrit, hemoglobin, mean corpuscular hemoglobin, mean corpuscular hemoglobin concentration, mean corpuscular hemoglobin amount, white blood cell count, platelets, red blood cell count, and red blood cell distribution width), and laboratory data such as basal metabolic panel (e.g., blood urea nitrogen, potassium, sodium, glucose, chloride, CO2, calcium). EMR data may include calculated values ​​including blood glucose (m, creatinine), demographic data (e.g., age, weight, race and sex, zip code), less common laboratory data (e.g., bilirubin, partial thromboplastin time, international normalized ratio, lactate, magnesium and phosphite), and other suitable patient indicators currently in existence or to be developed in the future (e.g., O2, use of Glasgow Coma Score or its components, and urine output over the past 24 hours, antibiotic administration, blood transfusion, fluid administration, etc.), as well as shock index and mean arterial pressure. EMR data may additionally or alternatively include chronic medical and / or surgical conditions. EMR data may include historical data collected from the patient's previous examinations, including historical FFR, iFR, or QFR data. Stenosis, prediction of vascular disease, vascular resistance, CFD simulation data, and determination of other data are produced by the techniques of this specification. EMR 126 may be updated when new data is collected from the angiography imaging device 124 and evaluated using the computing device 102. In some cases, the technology may provide ongoing training for the EMR126.

[0030] In conventional angiography imaging applications, angiographic images are captured by a medical imaging device and then sent to an EMR for further processing, including storage and, in some cases, image processing before being sent to a medical professional. In this technology, the state of occlusion, stenosis, and microvascular disease can be determined on a computing device based on the angiographic images, without first offloading those images to the EMR126 for processing. Overall, the technology proposed here can significantly reduce the analysis time for cardiologists, partly by bypassing the EMR126 for processing. The EMR126 may simply be polled for data during analysis by the computing device 102 and may be used for storage of state determinations and other calculations generated by the technology herein. Certainly, there are many advantages to the faster and more automated analysis resulting from current technologies. For example, modeling and vascular occlusion / disease state analysis can be performed sequentially and individually for vessels corresponding to either the left or right coronary artery tree, but the results for cardiologists can still be produced in minutes using, for example, a 3D modeler or a 1D modeler, as described herein.

[0031] In the illustrated example, system 100 is implemented on a single server. However, the functionality of system 100 may be implemented across distributed devices connected to each other via communication links. In other examples, the functionality of system 100 may be distributed across any number of devices, including the portable personal computer, smartphone, electronic document, tablet, and desktop personal computer devices shown. In other examples, the functionality of system 100 may be cloud-based, such as one or more connected cloud CPUs or computing systems labeled 105, customized to perform machine learning processes and the computing techniques described herein. Network 116 may be a public network such as the Internet, a private network such as a research institution or corporate private network, or any combination thereof. The network may include local area networks (LANs), wide area networks (WANs), cellular, satellite, or other network infrastructure, whether wireless or wired. The network may utilize communication protocols, including packet-based and / or datagram-based protocols such as Internet Protocol (IP), Transmission Control Protocol (TCP), User Datagram Protocol (UDP), or other types of protocols. Furthermore, network 116 may include a number of devices that facilitate network communication and / or form the hardware infrastructure of the network, such as switches, routers, gateways, access points (such as wireless access points as shown), firewalls, base stations, repeaters, and backbone devices.

[0032] The computer-readable medium 106 may include executable computer-readable code stored thereon for programming a computer (including, for example, a processor and a GPU) to the technology of this specification. Examples of such computer-readable storage media include hard disks, CD-ROMs, digital multipurpose disks (DVDs), optical storage devices, magnetic storage devices, ROMs (read-only memory), PROMs (programmable read-only memory), EPROMs (erasable programmable read-only memory), EEPROMs (electrically erasable programmable read-only memory), and flash memory. More generally, the processing unit of the computing device 102 may represent a CPU-type processing unit, a GPU-type processing unit, a field-programmable gate array (FPGA), another class of digital signal processor (DSP), or other hardware logic components that can be driven by a CPU.

[0033] While the exemplary deep learning frameworks and neural networks described herein have been presented as comprising exemplary machine learning architectures, it should be noted that any number of suitable convolutional neural network architectures can be used. Generally speaking, the deep learning frameworks herein can implement any appropriate statistical model (e.g., a neural network or other model implemented through a machine learning process) applied to each of the received images. As discussed herein, the statistical model can be implemented in a wide variety of ways. In some examples, the machine learning model may take the form of a neural network, a support vector machine (SVM), or other machine learning process, and may be trained using images or multidimensional datasets to develop a model for 3D reconstruction of the vascular tree. Once these models are properly trained on a series of training images from different viewpoints, the statistical model can be used in real time to decide whether to perform 3D reconstruction of the vascular tree, and further, to analyze subsequent angiography image data provided as input to the statistical model for determining the presence of CAD and the state and disease of vascular occlusion.

[0034] In some cases, when a statistical model is implemented using a neural network, the neural network can be configured in a wide variety of ways. In some cases, the neural network can be a deep neural network and / or a convolutional neural network. In some cases, the neural network can be a distributed and scalable neural network. Neural networks can be customized in a wide variety of ways, such as by providing a specific top layer, such as a logistic regression top layer. A convolutional neural network can be thought of as a neural network containing a set of nodes to which parameters are associated. A deep convolutional neural network can be thought of as having a stacked structure of multiple layers. Neural networks or other machine learning processes can include a variety of sizes, number of layers, and levels of connectivity. Some layers can correspond to stacked convolutional layers (optionally followed by contrast normalization and max pooling) followed by one or more fully connected layers. This technology may be implemented so that machine learning training can be performed using small datasets, such as 20,000 images, 15,000 images, fewer than 10,000 images, fewer than 1,000 images, or fewer than 500 images. In one example, approximately 1,500 images were used (with 5,000 vascular trees, 3 images per vascular tree).

[0035] Figure 2 illustrates an exemplary multi-stage neural network model 200 for training an ML model and for performing a 3D reconstruction of a vascular tree using the trained ML model. Figure 3A shows a process 300 for training an ML model to perform a 3D reconstruction of a coronary vascular tree. Process 300 may be performed by system 100, and further, process 300 may be performed according to the schematic diagram of the neural network model in Figure 2. For clarity, process 300 will be described with simultaneous reference to the elements of Figures 1 and 2.

[0036] 3D image data 202 is acquired by a medical imaging device and is available in process 302 through the EMR system 126 in Figure 1. In this example, the 3D image data includes images of a vascular tree which may be acquired by 3D computed tomography (CTA) or 3D magnetic resonance angiography (MRA). The 3D image data 202 is then stored in memory and may be used to train an ML model or to provide a trained ML model to perform a 3D reconstruction of the vascular tree.

[0037] The 3D image data 202 includes an image of a vascular tree containing multiple vascular tree branches. In the example, the 3D image data 202 may include images of the left coronary artery tree (LCA) and / or the right coronary artery tree (RCA), each tree having any number of branches. The synthetic vascular tree generator 204 then determines the statistical branch centerlines and branch radii from one or more vascular trees and / or vascular tree branches in process 304. For each branch centerline, the synthetic vascular tree generator 204 can produce a distribution of control points and standard deviations for each control point in 3D space. When generating the synthetic vascular tree, the synthetic vascular tree generator 204 can randomly generate the length of the branch centerlines, the statistical distribution of branch centerline control, the most proximal radius of each branch centerline, and a radius tapering coefficient over the length of the branch, the radius tapering coefficient representing the ratio of the diameters at the most proximal and most distal locations of the vascular branch.

[0038] As further illustrated in Figure 7 with reference to a specific application, in one example, multiple 3D images of a vascular tree branch (upper left of pane A in Figure 7) may be used to determine a statistical centerline from multiple branches. The multiple images may be from the same vascular tree, such as the right main coronary artery, the sharp marginal branch of the sinoatrial node, or another coronary vascular tree branch, and each image may be acquired from a different patient or from a single patient at different times. The centerline control point distribution is a generated representation of the determined branch centerline. The synthetic vascular tree generator 203 further generates radii for each centerline point from the multiple 3D images of the vascular tree branch.

[0039] Next, the synthetic vascular tree generator 204 generates multiple synthetic vascular tree branches and synthetic vascular trees by introducing deformations to the determined vascular tree branch centerlines and radii. For example, the synthetic vascular tree generator 203 can generate different synthetic vascular trees from a single vascular tree branch centerline by providing linear tapering to one or more branches or sections of branches of a vascular tree. Additionally, the synthetic vascular tree generator 204 can generate various synthetic vascular trees and vascular tree branches by introducing stenosis, shearing, rotation, inversion, resizing, warping, or other geometric transformations or operations to one or more branches or sections of branches. The synthetic vascular tree generator 204 can apply one or more stenosis profiles to one or more branches of one or more synthetic vascular trees to simulate vessels with stenosis. Thus, an ML model can be used to perform a 3D reconstruction of a vascular tree with or without stenosis along one or more of its branches. Stenosis profiles may include various geometric profiles that narrow the radius of one or more vascular trees along at least a portion of one or more vessels in the vascular tree. For example, a stenosis profile may be a Gaussian or sinusoidal profile applied to one or more vessels in a vascular tree. One or more stenosis profiles may be derived from or include one or more of the following: parametric stenosis location data (e.g., location of stenosis along the vessel), stenosis severity data (e.g., degree of narrowing or degree of stenosis, rate of narrowing of the vessel over distance along the vessel), and stenosis length data (e.g., length of stenosis along the vessel).

[0040] The synthetic vascular tree generator 204 can further generate a 3D volume including a coronary vascular tree. i Regarding each branch of the vascular tree, M i A coronary vascular tree can be generated by lofting a contour with radius r (see Figure 5 for further explanation). The coronary vascular tree generated from the contour loft defines the 3D analytical representation of the outer boundary or surface of each coronary vascular tree. In the example, non-uniform b-splines (NURBS) or another spline representation of the surface may be used when generating the 3D analytical surface representation.

[0041] To generate a synthetic vascular tree, a processor such as processing unit 104 may generate a vascular tree matrix representing each individual vascular tree in process 306. In the example implementation described herein, the vascular tree matrix is ​​M i xN i It is an x4 matrix, where i is the number exponent assigned to a given vascular tree (i.e., 1, 2, 3, etc.). i N is the number of branches in the synthetic vascular tree i, where each branch has a branching centerline. i is the number of points on each branching centerline. The final constant dimension of 4 is the number of dimensions of the data encoded at each point on the branching centerline, including the three spatial dimensions and the radius value.

[0042] After one or more vascular tree matrices have been generated, process 300 may further include reconstructing the coronary vascular tree as a 3D volume from the vascular tree matrices. The reconstructed or generated coronary vascular tree i is then processed in process 308. i It has branching. The preprocessor 212 may then generate a set of binary angiographic images from the coronary vascular tree in process 310. The preprocessor 212 may include one or more processors, such as processing unit (2) 104 in Figure 1. The preprocessor 212 may perform cone-beam projection using a set of different projection angles to generate binary angiographic images from different angle viewpoints of the coronary vascular tree.

[0043] Next, in process 312, at least two sets of binary angiographic images are selected, each representing a different viewpoint of the coronary tree. Each image provides a different viewpoint due to the various projection angles used to generate the binary angiographic image. In the example described herein, three images are used to train the ML model for performing 3D reconstruction, but it should be understood that two, four, or more images may be used for both training the ML model and for performing reconstruction of 3D objects such as the coronary tree. In the example, clinically acquired 2D binary angiographic images may be used for training, or a synthetic angiographic tree may be generated from MRA and CTA image data as described herein. The various binary angiographic images may then be stored in a computer-readable medium 106 for further use when training the 3D reconstruction ML model.

[0044] Next, in process 314, the preprocessor 212 applies a Euclidean distance transformation to each of the binary angiographic images, encoding the 2D diameter of each branch of the vascular tree into each binary angiographic image. Using the Euclidean distance transformation, the preprocessor generates a Euclidean distance transformed angiographic image from each binary angiographic image.

[0045] Next, the ML training module 210 trains a 3D vascular tree reconstruction ML model that can reconstruct a 3D vascular tree in process 316 using Euclidean distance-transformed angiographic images. The training of the ML model in process 316 is described in more detail with reference to Figure 3B using training process 320. The ML training module 210 includes three stages for training the ML model. The first stage ("stage 1") 210a includes a classical convolutional neural network backbone for image classification in process 322. The ML training module 210 further includes a second stage ("stage 2") 210b, referred herein as the vascular centerline stage, and a third stage ("stage 3") 210c, referred herein as the radius reconstruction stage. Although described as separate stages, it should be understood that the neural network backbone used to train the first stage and perform the convolution can be used to further train at least one MLP in stage 2 210b and stage 3 210c, or in stage 2 210b and / or stage 3 210c. Alternatively, each MLP in each stage, and each stage itself, can be trained independently of each other and each MLP.

[0046] The vascular centerline reconstruction stage includes a multilayer perceptron (MLP) with rectified linear unit (ReLU) activation in process 324. The vascular centerline reconstruction stage further includes interlayer batch normalization of the MLP to perform centerline reconstruction. The vascular centerline reconstruction stage takes in Euclidean distance-transformed angiographic images and at least M i xN i Output the x3 matrix, M i This represents the number of branches, N i represents the number of points at each branch, and 3 represents the number of points at each branch M i Center line point N i This represents the number of dimensional coordinates. The centerline coordinates of a point can be Cartesian coordinates, polar coordinates, or point N i It could be another coordinate system that shows the spatial coordinates. Step 2 210b is performed for each vascular tree, and each vascular tree i M i xN i Output a 3x matrix.

[0047] The radial reconstruction stage in process 326 includes an MLP with ReLU activation accompanied by interlayer batch normalization of the MLP to perform radial reconstruction. Each branch M i The radius reconstruction stage is performed. The radius reconstruction stage takes the Euclidean distance-transformed angiographic image as input, and the radius reconstruction stage uses the radius value N i Output the x1 matrix, and each radius value is the respective center line point N i Corresponds to the radius value N. i The x1 matrix is ​​M i xN i The x3 matrix is ​​connected to each center line point N i Output M having spatial coordinates and corresponding radius for i xN i Forms an x4 matrix.

[0048] Process 300, in process 328, M i xN i A reconstructed vascular tree may be generated from the x4 matrix. The reconstructed vascular tree may include a low-fidelity vascular tree that includes branches formed from centerline points and associated radii at each centerline point. The reconstructed vascular tree may include a high-fidelity vascular tree that includes volumes defined by creating surface splines between the outer radii of each centerline point radius at each branch of the vascular tree. The high-fidelity vascular tree may be referred to as a 3D representation, while the low-fidelity vascular tree may be referred to herein as a 1D representation.

[0049] To train the 3D reconstructed ML model 220, the ML training model 210 may, in process 330, implement a loss function that compares the synthetic vascular tree to the corresponding reconstructed vascular tree in order to train a multi-stage convolutional neural network. The loss function includes one or more mean square error calculations that compare the mean square error between the data associated with the set of synthetic vascular trees and the reconstructed vascular tree. The ML model 220 may be trained by reducing the mean square error of the synthetic vascular tree and the reconstructed vascular tree. The data associated with the set of synthetic vascular trees and the reconstructed vascular tree may include data corresponding to vascular tree length, one or more vessel diameters, vessel tortuosity, and / or stenosis patterns. The loss function may include normalization terms that prioritize or provide weights to optimize the loss function based on some of the data and parameters more than others.

[0050] As illustrated in Figure 2, the trained ML model 220 includes three stages: stage 1 220a, stage 2 220b, and stage 3 220c. Stage 1 220a includes the trained convolutional neural network backbone, stage 2 220b is referred herein to as the trained vascular centerline stage, and stage 3 220c is referred herein to as the trained radius reconstruction stage.

[0051] Figure 4 shows process 400 for performing a 3D reconstruction of the coronary vascular tree using a trained ML model. Process 400 may be performed by system 100, and furthermore, process 400 may be performed according to the schematic diagram of the neural network model in Figure 2. For clarity, process 400 will be described with simultaneous reference to the elements of Figures 1 and 2.

[0052] Process 400 includes providing a segmented binary angiographic image to the preprocessor 212 in process 402. The segmented binary angiographic image may be provided to the preprocessor 212, or the clinical angiographic figure 207 may be provided to the preprocessor 212, and the preprocessor may perform segmentation of the clinical angiographic figure 207 in order to generate the segmented binary angiographic image. The preprocessor 212 then applies a Euclidean distance transformation in process 404 to encode the 2D diameter of each branch of the vascular tree into the binary angiographic image. The preprocessor 212 then generates a distance-transformed binary angiographic image.

[0053] In process 406, the preprocessor 212 provides the Euclidean distance-transformed angiographic images to a trained 3D vascular reconstruction ML model. The 3D vascular reconstruction machine learning model is a machine learning model trained to generate reconstructions of 3D vessels and vascular trees from angiographic images. As previously stated, the trained ML model 220 includes three stages for performing 3D vascular tree reconstruction. The first stage, stage 1 220a, in process 408 includes a trained classical convolutional neural network backbone for image classification. Although described as different stages, it should be understood that the neural network backbone used in stage 1 may be used to perform operations in stages 2 220b and 3 220c, or to perform at least one MLP in stages 2 220b and / or 3 220c. Alternatively, each MLP in each stage, and each stage itself, may perform each stage independently of each other and each MLP, using different neural networks or neural network backbones. Additionally, stages 220b and 320c may be executed independently and therefore simultaneously.

[0054] The trained vascular centerline reconstruction stage includes an MLP with ReLU activation in process 410. The trained vascular centerline reconstruction stage further includes interlayer batch normalization of the MLP to perform centerline reconstruction. In a particular example, in the centerline stage, the MLP consists of four hidden layers, with the first three layers having 1024 neurons and the last layer having 512 neurons. The trained vascular centerline reconstruction stage takes up an angiographic image transformed by Euclidean distance transformation and at least M i xN i Output the x3 matrix, M i This represents the number of branches, N i represents the number of points at each branch, and 3 represents the number of points at each branch M i Center line point N i This represents the number of dimensional coordinates. The centerline coordinates of a point can be Cartesian coordinates, polar coordinates, or point N i It could be another coordinate system that shows the spatial coordinates. Stage 2 220b is performed for each vascular tree, and each vascular tree i M i xN i Output a 3x matrix.

[0055] The trained radius reconstruction stage in process 412 includes an MLP with ReLU activation accompanied by interlayer batch normalization of the MLP to perform radius reconstruction. Each branch M i A trained radius reconstruction stage is performed. The trained radius reconstruction stage takes a Euclidean distance-transformed angiographic image as input, and the trained radius reconstruction stage uses the radius value N i Output the x1 matrix, and each radius value is the respective center line point N i Corresponds to the radius value N. i The x1 matrix is ​​M i xN i The x3 matrix is ​​connected to each center line point N i Output M having spatial coordinates and corresponding radius for i xN i It forms an x4 matrix. In a particular example, the trained radius reconstruction stage is the sum M of the given vascular tree i. iRegarding MLPs, a separate MLP is included for each vascular branch. In one example, a radius MLP consists of three hidden layers, each with 128 neurons, with batch normalization and ReLU activation occurring between each hidden layer. The MLP for each branch can be trained individually to improve the network's ability to capture sudden reductions in vascular radius in stenotic areas. Without independent training of each MLP for each branch, stenosis may be overlooked, as it constitutes a small portion of the coronary tree.

[0056] Process 400, in process 412, M i xN i A vascular tree 230 reconstructed from an x4 matrix can be further generated. The reconstructed vascular tree 230 may include a low-fidelity vascular tree that includes branches formed from centerline points and associated radii at each centerline point. The reconstructed vascular tree may include a high-fidelity vascular tree that includes volumes defined by creating surface splines between the outer radii of the radii of each centerline point at each branch of the vascular tree. The low-fidelity vascular tree may be referred to herein as the 1D representation of the reconstructed vascular tree, while the high-fidelity vascular tree may be referred to as the 3D representation. To generate the high-fidelity representation, a surface can be formed using B-splines, or non-uniform rational B-splines, and M i xN i The 3D volume can also be defined from an x4 matrix.

[0057] In some embodiments, the reconstructed vascular tree 230 may be used for the medical diagnosis or analysis of the vascular tree. For example, process 400 may further include identifying vascular examination regions of the reconstructed vascular tree 230 and determining the state of vascular occlusion from either a low-fidelity or high-fidelity representation of the vascular tree. The vascular examination regions may be regions of the reconstructed vascular tree 230 corresponding to regions of the reconstructed vascular tree that include the vascular tree structure. The vascular examination regions may be determined from a clinical angiographic image 207, a segmented binary angiographic image, or another image used in process 400 to perform 3D reconstruction of the vascular tree. Further details on using the reconstructed 3D vascular tree to determine and analyze vascular occlusion can be found in U.S. Patent No. 11,386,563, originally filed November 23, 2020, which is incorporated herein by reference in its entirety.

[0058] Figure 5 shows examples of both high-fidelity or 3D vascular tree representations and low-fidelity or 1D vascular tree representations. The low-fidelity, or 1D, vascular tree model representation shows each branch M i Each point N i The values ​​are given by the centerline coordinates and radius. The high-fidelity reconstruction representation includes a volume bounded by a smooth analytical surface formed between the radii of each centerline coordinate point, and the volume is given by all centerline points N i It includes. In the example given, the vascular tree i is the tree matrix M i ×N i It is expressed by ×4, where M i is a vascular tree i The number of branches within the N i is the number of points on each branching centerline, and 4 is the numerical dimension of the data encoded by each point on the branching centerline, specifically its three-dimensional spatial coordinates (x, y, z) and radius r.

[0059] Figure 6 is a schematic diagram of an exemplary implementation architecture of a neural network with angiographic input and output 3D reconstructed vascular trees. The multi-layer neural network was designed to reconstruct both the vascular centerlines (layer 2) and radii (layer 3) of each branch of the coronary artery tree. Both the centerline layer and the radius layer were trained with ML models using a convolutional neural network backbone to prioritize relevant features of the coronary artery tree from the input image and to accurately reconstruct them. As an example, the ResNet101 backbone was used as the backbone neural network. While convolutional layers can learn image-based features related to vascular geometry, MLPs may be better at solving regressions to identify the 3D coordinates of vascular centerline points and corresponding radii. Therefore, the final layer of the backbone network contained separate MLPs for the centerline layer and the radius layer.

[0060] In the central line section, the final fully connected layer of the backbone network was replaced with an MLP with ReLU activation and interlayer batch normalization. The MLP consisted of four hidden layers, with the first three layers having 1024 neurons and the last layer having 512 neurons. The output of the central line MLP was shown in the binarized angiography. i For each branch, N i M including the center line point i xN i It was a linear layer of x3. This output vector was reassembled into a matrix before calculating the loss for training.

[0061] On the other hand, the radial stage replaces the final layer of the backbone with a separate MLP for each vascular branch, totaling M iWe designated the radius MLP as such. Each radius MLP consisted of three hidden layers, each with 128 neurons, and between each hidden layer, there was batch normalization and ReLU activation. The output of each MLP was a radius vector of dimension N. We trained the MLP for each branch individually to improve the network's ability to capture sudden decreases in vascular radius in stenotic regions. Without this step, stenosis could be overlooked, as it constitutes a small portion of the points in the coronary artery tree.

[0062] The radial stage was trained using the mean squared error as the loss function (Equation 1).

number

number

number

number

[0063] We trained both centerline reconstruction and radial stages using an Adaptive Moment Estimation (ADAM) optimizer with a learning rate of 5e-4, weight decay, L2, and normalization. We set the batch size to 8 and trained the multistage network for 300 epochs. For radial stage MLPs, we froze the backbone after the first 300 epochs and trained each branch MLP for an additional 50 epochs.

[0064] For comparison, a single-stage neural network model was developed as a counterpart to the multi-stage network described above. The single-stage network architecture consisted of a backbone network and a single MLP that output an M×N×4 matrix containing both centerline points and their associated radii, as shown in Figure 1. The single-stage network used a single MLP to determine both centerline reconstruction and radius reconstruction in a single stage. The loss function for the single-stage network was the weighted mean squared error loss function, Equation 3.

number

[0065] In equation 3, y i and

number

number

[0066] To train the proposed multi-stage neural network, hundreds to thousands of ground truth 3D coronary artery trees can be used along with their corresponding segmented 2D angiographic images. In practice, this means having to identify thousands of patients with both 3D CTA data and 2D X-ray angiography, which is typically impractical for many medical centers and many studies. Another challenge of using clinical image data as input to train ML models is that the coronary arteries deform in each frame of an X-ray angiography series due to cardiac contraction. This requires temporal registration of frames from multiple angiography series to create a valid set of input images for 3D coronary artery tree reconstruction. To produce a sufficiently large dataset and eliminate external sources of error such as temporal registration, a method was devised to produce a sufficiently large training dataset consisting of a set of 5,000 static 3D coronary artery tree shapes and their corresponding 2D projections. Although synthetic data is used to train and validate the 3D reconstruction multi-stage neural network described herein, the use of synthetic projection images as input does not preclude future clinical applications. Using segmentation algorithms or neural networks such as AngioNet, clinical angiograms acquired during future routine patient care can be converted into public input for the described 3D reconstructed neural network ML model.

[0067] Because the large anatomical variations in the left coronary artery make reconstruction more difficult, we provide an example that focuses on 3D reconstruction of the right coronary artery tree. A method for generating a set of synthetic angiographic images is implemented. This method includes the following two steps: 1) a synthetic 3D coronary artery tree generator, and 2) a projection algorithm for creating a set of segmented angiographic images.

[0068] For the first step, the synthetic 3D coronary artery tree generator generates one or more synthetic vascular trees. Figure 7 illustrates an overview of the synthetic 3D coronary artery tree generator. Patient-specific centerline distributions were obtained from 10 CTAs of the four major branches of the right coronary artery: the right coronary artery (RCA), sinoatrial node branch (SA), acute marginal branch (AM), and post-descending artery (PDA). The posterolateral ventricular branch (PLV) is implicitly included as part of the RCA, which branches into the PDA and PLV branches. Using this data, the statistical distribution of control points and their standard deviations in 3D space for each branch of the coronary artery tree was identified, as illustrated in Figure 7A. From this distribution, new vessels were generated using uniform random sampling. Linear tapering of the radius was assigned to each vascular branch, and stenosis with a Gaussian profile was randomly introduced (Figure 7B). The branches were joined to form a tree and augmented with random rotation, shear, and / or distortion (Figure 7C). The synthetic tree generator algorithm was refined by iterating with qualified intervention cardiologists to generate realistic trees until cardiologists could not distinguish between actual images of vascular trees and the synthetically generated vascular trees.

[0069] The second step in the method for generating synthetic angiographic images involved projecting the generated synthetic vascular tree. The cone-beam projection for each coronary artery tree was generated from five views to mimic the X-ray angiography acquisition process. The image acquisition angles were randomly sampled from a 20-degree window around commonly used clinical values. Three of the five views were randomly selected for training. Figure 8 illustrates how multiple projections of the vascular tree were performed to generate binary angiographic images. Since binary images lack distance information, a Euclidean distance transformation was applied to the projected images. The resulting distance-transformed binary angiographic images were then used to train a neural network. The synthetic vascular tree generated by the described steps was divided into 4,500 coronary artery trees and their corresponding projections for training, and 500 for validation, resulting in 90–10 training divisions.

[0070] Several methods were employed to evaluate the performance of the proposed method. To demonstrate the advantages of a multi-stage approach, the first reconstruction error of our multi-stage network was compared with that of the aforementioned single-stage neural network. Next, a direct comparison was made between the multi-stage neural network reconstruction method and the projection method. Both comparisons were performed on a single vascular geometry for simplicity. Then, the performance of the method in the reconstruction of the right coronary artery tree was considered. Finally, an analysis was conducted to determine how the geometric reconstruction error affects hemodynamics.

[0071] A distinguishing feature of the disclosed 3D reconstruction method is its multi-stage nature, which considers the tasks of centerline and radius reconstruction separately. The importance of separate reconstruction of centerline and radius is illustrated through a comparison between single and multi-stage networks. The performance of both networks was evaluated using 10 different RCA synthetic vessels. For each synthetic vessel, stenosis between 20% and 90% was introduced in 5% increments, resulting in a total of 150 unique vessel geometries. For each geometry, the reconstruction error was evaluated. An example of a vessel geometry with 70% stenosis and its single-stage and multi-stage reconstructions are shown in Figure 9. On average, the root mean square error (RMSE) of the multi-stage network centerline was 52% lower than that of the single-stage centerline RMSE (0.83 ± 0.29 mm vs. 1.73 ± 0.42 mm), as shown in the right panel (Panel B) of Figure 9.

[0072] Regarding the accuracy of radius reconstruction, the mean absolute error (MAE) at the time of stenosis was evaluated rather than the RMSE of the entire vessel in order to accurately predict the severity of stenosis. The MAE across the entire vessel was 0.117 ± 0.068 mm for the multi-stage network and 0.927 ± 0.436 mm for the single-stage network. In both networks, the MAE increased with the severity of stenosis. The single-stage network, in the left panel of Figure 9, failed to effectively detect stenosis at all levels and resulted in a very large MAE error (greater than 1.5 mm) for severe stenosis with a diameter reduction of more than 80%.

[0073] Next, the performance of a multi-stage neural network was compared with a projection-based method in the simplest case of single vessel centerline reconstruction from two projection images. In the projection-based reconstruction method, a point cloud approach was used to automatically identify up to 10 possible corresponding points for each centerline point in the reference projection image. The set of possible corresponding points was further refined using reprojection error and order-constrained cost functions. The matching points were then back-projected from both images and interpolated with b-splines to create a 3D centerline. Figure 10 presents the synthetic single vessel used for comparison, along with the binarized projection image and comparison results. For the dataset of synthetic projection images for this comparison, five synthetic RCA vessels were initially generated. The projection in top panel A of Figure 10 uses a spherical coordinate system (θ,φ), where θ and φ are the azimuth and elevation angles, respectively. For a given vessel, the first projection image for all pairs was fixed at θ=-45° and φ=0. The second set of images was defined using six different intervals Δθ∈[15°, 90°] for a total of 30 pairs of different projection images (five different vessels, six different angles between the projection images of each vessel). The reconstruction error was compared between both methods as a function of the angle Δθ between the projection images.

[0074] As seen in panel B of Figure 10, the centerline RMSE and standard deviation were lower with the neural network approach compared to the projection-based method for all test angles and vessels: 2.12 ± 0.05 mm and 3.49 ± 0.44 mm, respectively. Panel C of Figure 10 shows two examples of the same vessel reconstructed using two pairs of projection images with different Δθ. Ground truth centerlines (blue), neural network centerlines (red), and projection-based centerlines (brown) are shown. The neural network centerlines followed the average path of the ground truth centerlines but failed to capture some of the bends along that path (red arrows). Brown arrows, on the other hand, indicate areas of gap or uncertainty in the centerlines reconstructed using the projection-based method.

[0075] The accuracy of the proposed coronary artery tree reconstruction method was analyzed using the aforementioned validation set of 500 synthetic coronary artery trees and binarized image generation for synthetic vascular trees. Figure 11 shows several examples of the reconstructed coronary artery tree centerlines (red) and corresponding ground truth (blue). It was observed that the neural network learned the mean centerline path of each vessel in the tree, but, as with single vessels, the predicted vessels were not as tortuous as the ground truth centerlines. The RMSE between the ground truth and the predicted centerline points in the validation set was 2.57 ± 0.78 mm. The MAE of vessel length was 8.83 ± 4.81 mm. The optimal value for vessel length reconstruction was obtained with a normalized length parameter λ = 0.1 (see Equation 2), which resulted in a 47% reduction in vessel length error (16.36 ± 2.88 mm) compared to the same network trained without length normalization of the loss function. A larger value of λ resulted in excessive restriction of vessel length and inaccurate representation of different branching pathways, while a smaller value of λ resulted in weaker vessel length.

[0076] Next, errors in vessel radius along the midline were considered for all 2,000 branches. The RMSE of vessel radius was 0.16 ± 0.07 mm, which corresponds to sub-pixel resolution. The radius error was large, especially in cases of severe stenosis where the diameter exceeded 70%, when comparing the minimum stenosis diameter (MAE = 0.27 ± 0.18 mm), and was consistent with the behavior reported for single vessel reconstructions in projection-based reconstruction comparisons. Figure 12 shows examples of radius reconstructions (red dots) along the normalized midline position of the main RCA (blue) in a vessel without stenosis (top panel), two cases of single stenosis (middle panel), and a case of two stenosis (bottom panel). Panel A in Figure 12 demonstrates that the neural network accurately captured the tapering of the vessel along its length. Panel B shows that the neural network accurately predicted the severity and location of the stenosis, although some oscillations are evident in the region outside the stenosis, or the location of the stenosis is slightly altered. Panel C shows that the neural network approximates the continuous constriction as a single, longer constriction, with an error of 0.3 mm in constriction severity. Some oscillations remain evident in the region outside the constriction.

[0077] In the aforementioned example, the performance of a multi-stage neural network for reconstructing the centerlines and radii of the coronary artery trees was compared with other reconstruction methods to evaluate accuracy. Below, we consider the functional assessment of vascular disease using 3D reconstruction of vascular trees with a neural network ML model. To evaluate the performance of 3D reconstruction in the assessment of vascular disease, we used the physics of blood flow and pressure in the reconstructed coronary artery trees using computational fluid dynamics (CFD) simulations and compared the results with known ground truth CFD data. Specifically, we compared the pressure distribution of RCA vessels in ground truth coronary artery trees (known synthetic geometric shapes) with their reconstructions to evaluate the overall performance as a diagnostic method. The pressure field is used as the basis for calculating established functional metrics of CAD such as FFR, iFR, and QFR.

[0078] CFD simulations with consistent inflow and outflow boundary conditions were performed using the validated open-source software CRIMSON. Two examples were considered: 1) a healthy coronary tree without stenosis, and 2) a diseased tree with one stenosis. Figure 13 summarizes the pressure results for the ground truth and reconstructed coronary trees. The results for the healthy coronary tree are shown in the upper panel of Figure 13. The left column shows the pressure solution map calculated by CFD analysis of the ground truth and reconstructed geometric shapes. The center and right columns show radius versus pressure, respectively, for the normalized centerline position of the ground truth (blue) and reconstructed (red) geometric shapes. In the upper panel, there is no stenosis and the pressure change under the RCA is small. The pressure gradients over the ground truth and reconstructed RCA vessels are DPGT = 9.8 mmHg and DPR = 8.4 mmHg, and consequently ΔP = ΔP GT -ΔP R This results in a difference in pressure gradient estimates of 1.4 mmHg (or only 1.2% of the ground truth inflow pressure).

[0079] In the lesioned coronary artery tree, the pressure gradients over the ground truth and the reconstructed RCA vessel were DPGT = 18.8 mmHg and DPR = 14.3 mmHg, resulting in an error of ΔP = 4.5 mmHg. Additionally, the pressure in the reconstructed shape was approximately 10 mmHg lower than the pressure in the ground truth. This is due to the difference in the reconstructed radius in the proximal portion of the vessel and the reduced tortuosity (and therefore reduced resistance) of the reconstructed coronary artery tree compared to the ground truth.

[0080] In cases of stenosis, the pressure error is larger compared to healthy cases, which may be due to the radius reconstruction error during stenosis. Given a vessel with radius a, its resistance R is a force of 4 on that radius (R ~ 1 / a 4) is inversely proportional. Therefore, relatively small errors in radius reconstruction have a substantial effect on vascular resistance and thus result in a pressure drop across the entire vessel. The ground truth stenosis geometry had continuous 51% and 57% stenoses. However, the neural network approximated these two stenoses as a single, longer, 53% stenosis corresponding to a radius error of 0.07 mm (see middle column 13). The pressure ratio on both sides of the stenosis (surrogate of FFR) was 0.886 for ground truth and 0.895 for the reconstructed vessel. Therefore, despite a 10 mmHg difference in inlet pressure, there was only a 1% error in the clinical volume of interest.

[0081] As described above, the presented 3D reconstruction multi-layer neural network ML model can accurately reconstruct the vascular tree from a set of uncalibrated X-ray angiographic images at sub-pixel resolution of the vascular radius. The disclosed method and system demonstrate that the reconstruction error is at an acceptable level for accurately modeling hemodynamic quantities such as pressure gradients across vessels. Additionally, the proposed method surpasses other existing reconstruction methods, such as the use of single-layer ML models and projection-based reconstruction.

[0082] Additional aspects Embodiment 1. A computer implementation method for training a vascular tree reconstruction system, comprising: selecting a set of three or more sets of binary angiography images corresponding to three or more different projection angles for each synthetic vascular tree in a set of synthetic vascular trees; applying a Euclidean distance transformation that encodes the two-dimensional (2D) diameter of each branch to the binary angiography images to generate Euclidean distance transformed angiography images; and training a 3D vascular tree reconstruction machine learning model capable of reconstructing 3D vessels based on the set of Euclidean distance transformed angiography images, wherein the training of the 3D vascular tree reconstruction machine learning model includes a multi-stage convolutional neural network having a multi-stage architecture having (i) a vascular centerline stage and (ii) a radius reconstruction stage.

[0083] Embodiment 2. A multistage convolutional neural network having a three-stage architecture, wherein the 3D vascular tree reconstruction machine learning model is a multistage convolutional neural network having a three-stage architecture, the second stage being a centerline stage which includes a multilayer perceptron (MLP) and involves ReLU activation and inter-layer batch normalization of the MLP for centerline reconstruction, and the third stage being a radius reconstruction stage which includes an MLP and involves ReLU activation and inter-layer batch normalization of the MLP for radius reconstruction, the output of the second stage includes an M_i × N_i × 3 matrix where 3 represents the number of dimensions of the centerline coordinates in orthogonal coordinates for each point Ni of each branch Mi of each 3D vascular tree i, the third stage is performed for each branch Mi, and the output of the third stage is an Ni*1 matrix of radii for each point Ni of each centerline.

[0084] Embodiment 3. The computer implementation method according to Embodiment 1, further comprising: generating a set of synthetic vascular trees, where each synthetic vascular tree i is represented by a tree matrix M_i×N_i×4, where Mi is the number of branches in the synthetic vascular tree i, each branch having a branch centerline, Ni is the number of points on each branch centerline, and 4 is the dimension of the data encoded at each point on the branch centerline, including three spatial dimension coordinates and a radius; generating a three-dimensional (3D) volume from the M_i×N_i×4 structure, which includes a coronary vascular tree containing Mi branches; and generating a set of binary angiographic images from each synthetic vascular tree by a cone beam projecting each respective synthetic vascular tree using a plurality of projection angles.

[0085] Embodiment 4. The computer implementation method according to Embodiment 3, further comprising generating a set of synthetic vascular trees for both the left coronary artery tree (LCA) and the right coronary artery tree (RCA), each consisting of any number of branches.

[0086] Apparatus 5. Creating a set of synthetic vascular trees is equivalent to obtaining 3D medical imaging data (such as 3D computed tomography angiography (CTA) or 3D magnetic resonance angiography (MRA)) of the target coronary vascular tree, The computer implementation method according to embodiment 3, comprising: identifying branch centerlines from 3D medical imaging data; and generating a statistical distribution of control points in three-dimensional space and a corresponding standard deviation for each branch centerline.

[0087] Embodiment 6. The computer implementation method according to Embodiment 5, comprising randomly generating for each branch centerline (i) the length of the centerline, (ii) the distribution of centerline control points, (iii) the nearest radius of each branch centerline, and (iv) a radius tapering coefficient over the length of the branch.

[0088] Embodiment 7. The computer implementation method according to Embodiment 5, comprising imposing one or more stenotic profiles on one or more branches of each synthetic vascular tree.

[0089] Embodiment 8. The computer implementation method according to Embodiment 7, wherein one or more constriction profiles include a specific analysis profile such as a Gaussian profile or a sinusoidal profile.

[0090] Embodiment 9. The computer implementation method according to Embodiment 7, wherein each of one or more stenosis profiles includes at least one of parametric stenosis location data, stenosis severity data, and stenosis length data.

[0091] Embodiment 10. The computer implementation method according to Embodiment 5, further comprising generating a 3D volume including the coronary vascular tree by lofting a contour having the radius r of each branch Mi of the vascular tree for each point Ni, and defining the analytical surface representation of the boundary of each coronary vascular tree.

[0092] Embodiment 11. A computer implementation method according to Embodiment 2, further comprising utilizing a loss function to train a multi-stage convolutional neural network, wherein the loss function comprises one or more mean square errors between data associated with a set of vascular trees, which include one or more data relating to vascular tree length, diameter, vascular tortuosity, and stenosis patterns, and the reconstructed vascular trees, and further comprising a normalization term that prioritizes certain features of the trees, including but not limited to vascular tree length, diameter, vascular tortuosity, and stenosis patterns.

[0093] Embodiment 12. The computer implementation method according to Embodiment 2, wherein the classification convolutional neural network backbone used to train the first stage is also used to train at least one of the second stage MLP and the third stage MLP.

[0094] Embodiment 13. The computer implementation method according to Embodiment 2, wherein each MLP is trained independently of the others and each stage is trained independently.

[0095] Embodiment 14. A method for performing 3D vascular tree reconstruction, comprising: providing segmented binary angiographic images to a preprocessing module; applying a Euclidean distance transformation to the binary angiographic images to encode the two-dimensional (2D) diameter of each branch, thereby generating a Euclidean distance-transformed angiographic image; and supplying a set of Euclidean distance-transformed angiographic images to a 3D vascular tree reconstruction machine learning model capable of reconstructing 3D vessels, wherein the trained 3D vascular tree reconstruction machine learning model comprises a multi-stage convolutional neural network having a multi-stage architecture having (i) a vascular centerline stage and (ii) a radius reconstruction stage.

[0096] Embodiment 15. The method according to Embodiment 14, wherein the dataset showing the 3D reconstruction of the vascular tree includes an M_i × N_i × 4 matrix, where Mi is the number of branches in vascular tree i, each branch having a branch centerline, Ni is the number of points on each branch centerline, and 4 is the dimension of the data encoded at each point on the branch centerline, specifically including three spatial dimension coordinates and a radius.

[0097] Embodiment 16. The method according to Embodiment 14, wherein the trained machine learning model is a multi-stage convolutional neural network having a three-stage architecture, the trained machine learning model having (i) a first stage comprising a classification convolutional neural network backbone, (ii) a centerline stage comprising a multilayer perceptron (MLP) with ReLU activation and inter-layer batch normalization of the MLP for centerline reconstruction, and (iii) a radius reconstruction stage comprising an MLP with ReLU activation and inter-layer batch normalization of the MLP for radius reconstruction, the output of the second stage comprising an M_i × N_i × 3 matrix, where 3 represents the number of dimensions of the centerline coordinates in orthogonal coordinates for each point Ni of each branch Mi, the third stage is performed for each branch Mi, and the output of the third stage is an Ni*1 matrix of radii for each point of each centerline.

[0098] Embodiment 17. The method according to Embodiment 14, further comprising generating a low-fidelity vascular tree representation including a vascular tree i represented by a tree matrix M_i × N_i × 4, where Mi is the number of branches in the vascular tree i, each branch having a branch centerline, Ni is the number of points on each branch centerline, and 4 is the dimension of the data encoded at each point on the branch centerline, including three spatial dimension coordinates and radii.

[0099] Embodiment 18. The method according to Embodiment 17, further comprising generating a high-fidelity vascular tree representation including a three-dimensional (3D) volume including a coronary vascular tree including Mi branching from an M_i×N_i×4 structure.

[0100] Embodiment 19. The method according to Embodiment 18, further comprising generating a 3D volume by executing a 2D surface spline between the outer radii of adjacent points of each branch centerline.

[0101] Embodiment 20. The method according to Embodiment 18, further comprising identifying a vascular examination region of a vascular tree and determining the state of vascular occlusion of one or more vessels within the vascular examination region from a low-fidelity vascular tree representation or a high-fidelity vascular tree representation.

[0102] Throughout this specification, multiple examples may implement components, operations, or structures described as a single example. While individual operations of one or more methods are illustrated and described as separate operations, one or more of these operations may be performed simultaneously, and they do not need to be performed in the illustrated order. Structures and functions presented as separate components within an exemplary configuration may be implemented as a combined structure or component. Similarly, structures and functions presented as single components may be implemented as separate components. These and other variations, modifications, additions, and improvements are included within the scope of this specification.

[0103] Additionally, certain embodiments described herein include logic or a number of routines, subroutines, applications, or instructions. These may constitute either software (e.g., code embodied on a non-temporary machine-readable medium) or hardware. In hardware, routines, etc., are tangible units capable of performing a particular operation and may be configured or arranged in a particular manner. In exemplary embodiments, one or more computer systems (e.g., standalone, client, or server computer systems), or one or more hardware modules of a computer system (e.g., processors or groups of processors), may be configured as hardware modules that operate to perform a particular operation described herein by software (e.g., an application or part of an application, such as the contrast agent injection system shown in Figure 2).

[0104] In various embodiments, hardware modules can be implemented mechanically or electronically. For example, a hardware module may include a permanently configured, dedicated circuit or logic (e.g., a field-programmable gate array (FPGA) or an application-specific processor such as an application-specific integrated circuit (ASIC)) for performing a particular operation. A hardware module may also include a programmable logic or circuit (e.g., implemented in a general-purpose processor or other programmable processor) temporarily configured by software for performing a particular operation. It will be understood that the decision of whether to implement a hardware module mechanically, with a dedicated and permanently configured circuit, or with a temporarily configured circuit (e.g., configured by software) can be made considering cost and time.

[0105] Therefore, the term “hardware module” should be understood to encompass tangible entities that are physically constructed, permanently configured (e.g., embedded in hardware), or temporarily configured (e.g., programmed) to operate in a particular way or to perform a particular operation described herein. In consideration of embodiments where a hardware module is temporarily configured (e.g., programmed), each hardware module does not need to be configured or instantiated in any instance at any given time. For example, if a hardware module includes a general-purpose processor configured using software, that general-purpose processor can be configured as different hardware modules at different points in time. Thus, the software may configure the processor, for example, to configure one hardware module at one time and another hardware module at another time.

[0106] Hardware modules can provide information to other hardware modules and receive information from other hardware modules. Therefore, the hardware modules described herein can be understood as being communicatively coupled. When such hardware modules exist simultaneously, communication can be achieved through signal transmission (e.g., through appropriate circuits and buses) connecting the hardware modules. In embodiments where multiple hardware modules are configured or instantiated at different points in time, communication between such hardware modules can be achieved, for example, through the storage and retrieval of information in a memory structure accessible to the multiple hardware modules. For example, one hardware module may perform an operation and store the output of that operation in a memory device to which it is communicatively coupled. Further hardware modules can then access the memory device to retrieve and process the stored output. Hardware modules can also initiate communication with input or output devices to perform operations on resources (e.g., information gathering).

[0107] Various operations of the exemplary methods described herein can be performed, at least in part, by one or more processors that are temporarily (e.g., by software) or permanently configured to perform the relevant operations. Whether temporarily or permanently configured, such processors may constitute a processor implementation module that operates to perform one or more operations or functions. The modules referred to herein may include processor implementation modules in some exemplary embodiments.

[0108] Similarly, any method or routine described herein can be at least partially processor-implemented. For example, at least part of the operation of a method may be performed by one or more processors or processor-implemented hardware modules. Certain performance of an operation may reside not only within a single machine but also distributed among one or more processors deployed across several machines. In some embodiments, one or more processors may reside in a single location (e.g., in a home environment, a work environment, or as a server farm), while in other embodiments, the processors may be distributed across multiple locations.

[0109] Certain operational performance characteristics may reside not only within a single machine but also distributed among one or more processors deployed across several machines. In some exemplary embodiments, one or more processors or processor implementation modules may reside in a single location (e.g., in a home environment, a work environment, or a server farm). In other exemplary embodiments, one or more processors or processor implementation modules may be distributed across multiple locations.

[0110] Unless otherwise specified, any description in this specification using terms such as “processing,” “computing,” “calculating,” “determining,” “presenting,” and “displaying” may mean the operation or processing of a machine (e.g., a computer) that manipulates or transforms data expressed as physical (e.g., electronic, magnetic, or optical) quantities in one or more memories (e.g., volatile memory, non-volatile memory, or a combination thereof), registers, or other machine parts that receive, store, transmit, or display information.

[0111] Where used herein, any reference to “one embodiment” or “embodiment” means that certain elements, features, structures or characteristics described in conjunction with the embodiment are included in at least one embodiment. The phrase “in one embodiment” appearing in various places herein does not necessarily refer to the same embodiment.

[0112] Some embodiments can be described using the expressions “combined” and “connected” along with their derivatives. For example, some embodiments can be described using the term “combined” to indicate that two or more elements are in direct physical or electrical contact. However, the term “combined” can also mean that two or more elements are not in direct contact with each other but still cooperate or interact with each other. Embodiments are not limited to this context.

[0113] Those skilled in the art will recognize that a wide variety of modifications, changes, and combinations can be made with respect to the embodiments described above without departing from the scope of the present invention, and such modifications, changes, and combinations should be considered to fall within the scope of the concept of the present invention.

[0114] Therefore, although the present invention has been described in relation to specific examples, these examples are merely illustrative and not intended to limit the present invention, and it will be apparent to those skilled in the art that modifications, additions, or deletions can be made to the disclosed embodiments without departing from the spirit and scope of the invention.

[0115] The above explanation is provided solely for the purpose of clarifying understanding, and modifications within the scope of the present invention may be obvious to those skilled in the art; therefore, no unnecessary limitations should be inferred therefrom.

Claims

1. A computer implementation method for training a vascular tree reconstruction system, For each synthetic vascular tree in the set of synthetic vascular trees, select a set of three or more binary angiographic images corresponding to three or more different projection angles, The process involves applying a Euclidean distance transformation that encodes the two-dimensional (2D) diameter of each branch to the binary angiographic image to generate a Euclidean distance transformed angiographic image, A computer implementation method comprising training a 3D vascular tree reconstruction machine learning model capable of reconstructing 3D blood vessels based on a set of Euclidean distance-transformed angiographic images, wherein the training of the 3D vascular tree reconstruction machine learning model includes a multi-stage convolutional neural network comprising a multi-stage architecture having (i) a vascular centerline stage and (ii) a radius reconstruction stage.

2. The 3D vascular tree reconstruction machine learning model is a multi-stage convolutional neural network having a three-stage architecture, the first stage comprising a classification convolutional neural network backbone, the second stage being the centerline stage comprising a multilayer perceptron (MLP) and involving ReLU activation and interlayer batch normalization of the MLP for centerline reconstruction, and the third stage being the radius reconstruction stage comprising an MLP and involving ReLU activation and interlayer batch normalization of the MLP for radius reconstruction, wherein the output of the second stage is M i ×N i The matrix includes a 3x matrix, and the 3 is each 3D blood vessel tree i Each branch M i Each point N i This represents the number of dimensions of the centerline coordinates in the Cartesian coordinate system, and the third stage is for each branch M i The process is performed on the third stage, and the output of the third stage is the radius N for each point Ni on each center line. i *1 The computer implementation method according to claim 1, wherein the matrix is ​​a matrix.

3. To generate a set of synthetic vascular trees, each synthetic vascular tree i is represented by a tree matrix M i × N i × 4, where M i is the number of branches in the synthetic vascular tree i, each branch having a branch centerline, and N i is the number of points on each branch centerline, and 4 is the dimension of the data encoded at each point of the branch centerline, which includes three spatial dimension coordinates and a radius, and to generate M i ×N i From the ×4 structure, M i To generate a three-dimensional (3D) volume including a coronary vascular tree with branching, The computer implementation method according to claim 1, further comprising generating a set of binary angiographic images from each synthetic vascular tree using a cone beam that projects each respective synthetic vascular tree using a plurality of projection angles.

4. The computer implementation method according to claim 3, further comprising generating a set of the synthetic vascular trees for both the left coronary artery tree (LCA) and the right coronary artery tree (RCA), each consisting of any number of branches.

5. To generate the aforementioned set of synthetic blood vessel trees, This involves obtaining 3D medical imaging data of the target coronary vascular tree (such as 3D computed tomography angiography (CTA) or 3D magnetic resonance angiography (MRA)), Identifying the branching centerline from the aforementioned 3D medical imaging data, The computer implementation method according to claim 3, comprising generating a statistical distribution of control points in three-dimensional space and the corresponding standard deviation for each branching centerline.

6. The computer implementation method according to claim 5, comprising randomly generating for each branch centerline (i) the length of the centerline, (ii) the distribution of centerline control points, (iii) the nearest radius of each branch centerline, and (iv) a radius tapering coefficient over the length of the branch.

7. The computer implementation method according to claim 5, comprising imposing one or more stenotic profiles on one or more branches of each synthetic vascular tree.

8. The computer implementation method according to claim 7, wherein the one or more constriction profiles include a specific analysis profile such as a Gaussian profile or a sinusoidal profile.

9. The computer implementation method according to claim 7, wherein each of the one or more stenosis profiles includes at least one of parametric stenosis location data, stenosis severity data, and stenosis length data.

10. Each branch M of the aforementioned vascular tree i Each point N i The computer implementation method according to claim 5, further comprising generating a 3D volume including the coronary vascular tree by lofting a contour having radius r, and defining an analytical surface representation of the boundary of each coronary vascular tree.

11. The computer implementation method according to claim 2, further comprising using a loss function to train the multi-stage convolutional neural network, wherein the loss function comprises one or more mean square errors between data associated with a set of vascular trees, which comprises one or more data relating to vascular tree length, diameter, vascular tortuosity, and stenosis patterns, and a normalization term that prioritizes certain features of the trees, which include but are not limited to vascular tree length, diameter, vascular tortuosity, and stenosis patterns.

12. The computer implementation method according to claim 2, wherein the classification convolutional neural network backbone used to train the first stage is also used to train at least one of the second stage MLP and the third stage MLP.

13. The computer implementation method according to claim 2, wherein each MLP is trained independently of the others, and each stage is trained independently.

14. A method for performing 3D vascular tree reconstruction, comprising providing segmented binary angiographic images to a preprocessing module, The aforementioned preprocessing module applies a Euclidean distance transformation that encodes the two-dimensional (2D) diameter of each branch to the binary angiography image, thereby generating a Euclidean distance transformed angiography image. A method comprising: feeding a set of Euclidean distance-transformed angiographic images to a trained 3D vascular tree reconstruction machine learning model capable of reconstructing 3D vessels, wherein the trained 3D vascular tree reconstruction machine learning model comprises a multi-stage convolutional neural network comprising a multi-stage architecture having (i) a vascular centerline stage and (ii) a radius reconstruction stage.

15. The dataset showing the 3D reconstruction of the vascular tree is M i ×N i The formula includes a 4x matrix, and in the formula, M i is the number of branches in the vascular tree i, where each branch has a branching centerline, and N i The method according to claim 14, wherein is the number of points on each branching centerline, and 4 is specifically the dimension of the data encoded at each point on the branching centerline, including three spatial dimensional coordinates and a radius.

16. The trained machine learning model is a multi-stage convolutional neural network having a three-stage architecture, the first stage comprising a classification convolutional neural network backbone, the second stage being the centerline stage comprising a multilayer perceptron (MLP) and involving ReLU activation and interlayer batch normalization of the MLP for centerline reconstruction, and the third stage being the radius reconstruction stage comprising an MLP and involving ReLU activation and interlayer batch normalization of the MLP for radius reconstruction, wherein the output of the second stage is M i ×N i The matrix includes a 3x matrix, and the 3 is each branch M i Each point N i This represents the number of dimensions of the centerline coordinates in Cartesian coordinates, and the third stage is performed for each branch, and the output of the third stage is the radius N for each point of each centerline. i *The method according to claim 14, wherein the matrix is ​​1.

17. tree matrix M i ×N i The method further includes generating a low-fidelity vascular tree representation that includes a vascular tree i represented by ×4, where M i This is the number of branches in the vascular tree i, where each branch has a branching centerline, and N i The method according to claim 14, wherein is the number of points on each branching centerline, and 4 is the dimension of the data encoded at each point on the branching centerline, including three spatial dimension coordinates and a radius.

18. M i ×N i From the ×4 structure, M i The method according to claim 17, further comprising generating a high-fidelity vascular tree representation including a three-dimensional (3D) volume including a coronary vascular tree including branches.

19. The method according to claim 18, further comprising generating the 3D volume by executing a 2D surface spline between the outer radii of adjacent points of each branch centerline.

20. Identifying the vascular examination region of the aforementioned vascular tree, The method according to claim 18, further comprising determining the state of vascular occlusion of one or more blood vessels within the vascular examination area from the low-fidelity vascular tree representation or the high-fidelity vascular tree representation.

Images