Systems and methods for coronary vascular diagnosis and prognosis using biomechanical stress profiling indices

Abstract

A computer-implemented method is proposed that utilizes medical imaging (i.e., invasive coronary angiography, invasive optical coherence tomography, or other catheter-based imaging and / or non-invasive computed tomography) to generate a 3D computer model of the arteries / vasculature to provide a risk score that predicts future changes in coronary artery disease based on artificial intelligence and biomechanical simulation. The computer-implemented method preferably analyzes images from one or more imaging systems to extract data regarding arterial structure and function, including augmenting or simulating missing information, and generates the 3D geometry and predicted outcomes of the artery or arterial system.

Description

[Technical Field]

[0001] The present invention relates to a system and method for generating a predictive model of a patient's arteries / vasculature based on artificial intelligence and biomechanical simulation, which provides a risk analysis to predict future coronary artery disease evolution and suggest optimal treatment pathways. [Background technology]

[0002] Cardiovascular disease is the leading cause of death worldwide, accounting for approximately 30% of all deaths in 2019. Often, a person has no symptoms of underlying vascular disease until they experience a heart attack or stroke.

[0003] Furthermore, even when using medical imaging, it is difficult to determine which coronary artery disease or which plaque will progress to a critical state. This may therefore lead to the patient having a recurrence of heart attack. Approximately one in five people will have another heart attack within five years. This therefore increases hospitalization and medical costs, resulting in an economic impact on society. The cost of cardiovascular disease is estimated to be approximately US$1.1 trillion annually across the United States, the EU, and the United Kingdom, and is projected to increase in the coming years.

[0004] Several systems have been developed in the published prior art to model coronary artery physiology and predict coronary artery disease and plaque buildup in a patient's circulatory system.

[0005] Published prior art includes U.S. Patent Application Publication No. 20210153945 (CHOI et al.) entitled "Systems and methods for predicting coronary plaque vulnerability from patient-specific anatomical image data." The system filed by CHOI et al. generally relates to a method for reporting coronary plaque vulnerability from patient-specific anatomical image data. The disclosed method includes acquiring anatomical image data of a patient's vasculature, determining hemodynamic and biochemical characteristics, and predicting vulnerability of plaque present in the patient's vasculature based on one or more determined feature vectors.

[0006] Other prior art publications known to the applicant include U.S. Patent No. 6,047,080, entitled "Method and apparatus for three-dimensional reconstruction of coronary vessels from angiographic images," U.S. Patent Application Publication No. 20210228094, entitled "Systems and methods for vascular diagnosis using blood flow magnitude and / or direction," and U.S. Patent Application Publication No. 20210161384, entitled "System and methods for estimation of blood flow characteristics using reduced order model and machine learning."

[0007] Additionally, there are systems with predictive capabilities currently in use within the field, such as the system used by HeartFlow™, which involves AI applied to CT coronary angiography imaging (CTCA) to determine blood pressure reduction, which involves incorporating fluid dynamics into the calculations.

[0008] Another system currently in use within the field is manufactured by Artrya™ and involves the use of AI to detect "vulnerable" plaque, and uses non-invasive computed tomography coronary angiography (CTCA) imaging.

[0009] It is recognized by relevant experts in the field that non-invasive CTCA imaging systems can be used as a tool to identify patients with severe arterial disease who require invasive procedures. However, this imaging lacks predictive capabilities and understanding of physiology, which led to the development of the prior art discussed above.

[0010] Additionally, several systems exist that provide continuous data to physicians during invasive procedures, such as invasive angiograms. One such system uses a fractional flow reserve (FFR) pressure wire to analyze the blood pressure drop across the plaque. Typically, if the blood pressure drop falls below 0.8, the physician will intervene (i.e., use a stent). An FFR system is used for the PressureWire™ X Guidewire, sold in the United States by Abbott Laboratories.

[0011] There are also systems for determining blood pressure reduction without the need for additional invasive wires. One such system involves the virtual quantitative flow ratio (QFR) developed by Medis™ (i.e., it eliminates the need for wires by using only angiographic images). However, the system disclosed by Medis™ cannot distinguish between plaque composition (i.e., whether the stenosis is caused by fatty or calcified plaque) and lacks predictive capabilities.

[0012] Those skilled in the art recognize that assessing anatomy or physiology from angiographic images is not straightforward. The difficulty of obtaining good quality views increases radiation exposure to the patient, requires manual processes that are highly susceptible to artifacts, fail to provide information about plaque composition, and offer no predictive capabilities. Furthermore, critically ill patients, such as those suffering from renal failure, cannot be exposed to such radiation, limiting the usefulness of the approach by requiring several views and prolonging the procedure time.

[0013] Invasive imaging systems exist that visualize cross-sectional anatomy and plaque formation. One such system is optical coherence tomography (OCT). An OCT system is used in the Dragonfly OPTIS™ imaging catheter sold by Abbott Laboratories. However, such systems have limitations, including an inability to see behind fatty plaque due to the tissue penetration limitations of light-based imaging (signal attenuation). These systems also lack predictive capabilities and face the same limitations as the invasive wires mentioned above.

[0014] Published prior art discloses the development of various systems for evaluating individual biomechanical parameters related to the diagnosis of dynamic cardiac angiography (4D) images and for modeling vascularity and stress. These published documents include the article by Wu, X. et al., "Angiography-Based 4-Dimensional Superficial Wall Strain and Stress: A New Diagnostic Tool in the Catheterization Laboratory," Frontiers in Cardiovascular Medicine, 2021, Vol. 8, Article 667310, pp. 1-13. This study suggests that angiography-based superficial wall dynamics may identify coronary artery segments at high risk for plaque rupture and the failure site of implanted stents. Frictional stresses generated by blood flow acting on the wall are calculated for static models of coronary arteries. Future developments include the integration of high-speed computational techniques to enable online availability of superficial wall strain and stress in the catheterization laboratory.

[0015] Another prior art document, U.S. Patent Application Publication No. 2019 / 0192012 (Cathworks Ltd), discloses automated determination of parameters based on vascular images used to calculate a vascular disease score. The score calculator is configured to determine a Vascular Condition Scoring Tool ("VSST") score for each potential lesion based on at least one of the size of the potential lesion, the distance from a branch point in a plurality of vascular segments to the potential lesion, and the distance from one potential lesion to an adjacent potential lesion. An exemplary device also includes a user interface configured to display the VSST score for the potential lesion.

[0016] Another US patent application by Cathworks Ltd (US Patent Application Publication No. 2020 / 0126229) discloses the use of vascular scores as well as generating simulations and virtual stents. Some prior art publications, including WO 2021 / 257906 (Univ Northwestern et al.) and WO 2020 / 146905 (Lightlab Imaging Inc), suggest the use of machine learning.

[0017] It should be recognized that the discussion of any prior art throughout this specification is included solely for the purpose of providing a context for the present invention, and should not be construed as an admission that such prior art was widely known or formed part of the common general knowledge in the art as it existed prior to the priority date of this application. [Prior art documents] [Patent documents]

[0018] [Patent Document 1] US Patent Application Publication No. 20210153945 [Patent Document 2] U.S. Patent No. 6,047,080 [Patent Document 3] US Patent Application Publication No. 20210228094 [Patent Document 4] US Patent Application Publication No. 20210161384 [Patent Document 5] U.S. Patent Application Publication No. 2019 / 0192012 [Patent Document 6] US Patent Application Publication No. 2020 / 0126229 [Patent Document 7] International Publication No. 2021 / 257906 [Patent Document 8] International Publication No. 2020 / 146905 [Non-patent literature]

[0019] [Non-Patent Document 1] "Angiography-Based 4-Dimensional Superficial Wall Strain and Stress: A New Diagnostic Tool in the Catheterization Laboratory", Frontiers in Cardiovascular Medicine,2021,Vol.8,Article 667310,p.1-13. Summary of the Invention

[0020] It is an object of the present invention to provide a method for generating an advanced visualization and predictive model of at least a portion of a patient's arterial / vasculature system to provide a risk score predicting future changes in coronary artery disease, based on artificial intelligence and biomechanical simulation. It is an object of the present invention to overcome at least some of the problems mentioned above, or at least to provide the public with a useful alternative. The foregoing objects should not be considered necessarily cumulative, and various aspects of the present invention may satisfy one or more of the above objects.

[0021] The present invention can be understood to generally include computer-implemented methods and computer systems on which the methods are implemented.

[0022] The computer-implemented method utilizes medical imaging (i.e., invasive coronary angiography, invasive optical coherence tomography, or other catheter-based imaging and / or non-invasive computed tomography) to generate a 3D computer model of the arteries / vasculature to provide a risk score that predicts future changes in coronary artery disease based on artificial intelligence and biomechanical simulation.

[0023] The computer-implemented method preferably analyzes images from one or more imaging systems to extract data regarding, but not limited to, the arterial centerline, lumen wall, plaque layer (fatty or calcified layer) of the arterial wall, and bifurcation regions to generate a 3D geometry of the artery or arterial system.

[0024] The computer-implemented method preferably analyzes image and measurement data to extract inputs including, but not limited to, patient history, medication use, clinical symptoms, biochemical signatures, and to determine physiological characteristics including, but not limited to, blood flow velocity, blood pressure, heart rate, arterial dynamic motion, and microvascular resistance.

[0025] The computer-implemented method preferably can incorporate manual user input from a trained technician or clinician.

[0026] The computer-implemented method preferably applies these physiological inputs and 3D geometry to perform one or more artificial intelligence and / or biomechanical simulations in real time to suggest likely outcomes for the plaque and / or artery and / or patient, and whether the patient requires or would benefit from a detailed simulation evaluation.

[0027] The aforementioned methods should be considered "Level 1" analysis and are displayed through a user interface for real-time interactive visualization, allowing the user to better assess the patient's condition.

[0028] Upon suggestion from the "Level 1" analysis and / or request from the clinician, the computer-implemented method preferably performs detailed artificial intelligence-embedded biomechanical simulations to analyze up to 69 personalized markers.

[0029] The computer-implemented method then selectively combines and / or omits metrics in combination with the "Level 1" data through a machine learning decision-making process to provide a continuous multidimensional Biomechanical Stress Profiling Index (BSPI) across the plaque / plaques and / or artery / arteries and / or for the entire patient.

[0030] It should be recognized that the BSPI is continuous and multidimensional in a way that suggests the likelihood of some change, suggesting that not all markers are necessary at all times and not just providing a numerical value (e.g., blood pressure reduction) or just the entire endpoint.

[0031] The computer-implemented method preferably presents the BSPI in some format, including, but not limited to, a written report, a data spreadsheet, or an interactive visualization that directly compares with similar demographic data present in a computer system database.

[0032] The aforementioned methods should be considered "Level 2" analysis and are displayed through a user interface for real-time interactive visualization, allowing the user to better assess the patient's condition.

[0033] The computer-implemented method preferably also allows the user to query individual markers individually through an interactive visualization.

[0034] The computer-implemented method preferably uses a machine learning process to suggest likely or unlikely treatment paths based on the BSPI, including, but not limited to, using balloon angioplasty to restore blood flow and visualizing the optimal location for the procedure, inserting a stent to hold the artery open and visualizing the optimal location for the procedure, suggesting stent patency or malapposition and visualizing locations where adjustment is needed, performing a coronary artery bypass graft (CABG), using aggressive medical therapy, and lifestyle modifications.

[0035] The computer-implemented method may preferably provide a data output that is specific to the type of imaging system used.

[0036] The computer-implemented method preferably integrates imaging data from different imaging systems, where available, into a single enhanced user interface.

[0037] In one aspect of the present invention, although not necessarily its broadest or only aspect, there is proposed a computer-implemented method for generating advanced visualization and prediction models to provide individualized biomechanical stress profiling metrics for a patient, the method comprising: a. acquiring images, data, and characteristics related to a patient; b. constructing a vasculature model of at least a portion of an artery of the patient; c. extracting or calculating physiological information from the acquired images, data, and features of the patient; d. Using the captured data, conducting a lightweight "Level 1" artificial intelligence and / or biomechanical assessment; e. Using the "Level 1" results to suggest an optimal route or the need for "Level 2" analysis; f. Generating an enhanced visual display and / or report of the "Level 1" results to assist clinician decision making; g. Conducting "Level 2" artificial intelligence and / or biomechanical simulations for comprehensive patient assessment using several metrics; h. generating a continuous, multi-dimensional biomechanical stress profiling index of one or more plaques and / or arteries using the metrics, acquired images, data, and features of the patient; i. retraining and updating the "Level 1" analysis using results from the biomechanical stress profiling index; j. utilizing biomechanical stress profiling indices to highlight vulnerable plaque areas, plaque synthesis, risk of future growth and destabilization, and / or vascular changes over time; k. generating an enhanced visual display and / or report of the indicators to assist clinician decision-making in making treatment decisions for the patient; Includes.

[0038] In one form, step "a." may include obtaining imaging information from one or more invasive catheter-based imaging systems, such as coronary optical coherence tomography.

[0039] In another aspect, step "a." may include acquiring imaging information and gantry orientation and gating information from one or more planes in an invasive coronary angiogram and / or ventriculogram.

[0040] In another form, step "a." may include obtaining the imaging information from non-invasive computed tomography imaging.

[0041] In another form, step "a." may include obtaining continuous measurements such as heart rate, blood pressure, and electrocardiogram, including associated data from wearable technology and patient characteristics.

[0042] In yet another form, step "a." may include obtaining manual input values ​​from a trained technician or clinician.

[0043] In one form, step "b." also automatically: i. pre-processing the 2D intravascular imaging stack on a computational medium such as a central processing unit or a graphical processing unit (CPU or GPU); ii. scaling and axially stacking the pre-processed and segmented image data into slices in three dimensions; iii. communicating the pre-processed and segmented data from the local systems over a secure network to a centralized cloud computer or containerized instance; iv. Segmenting the pre-processed image stack on a CPU or GPU using machine learning, such as a temporal neural network or a 3D neural network, to identify vascular structures; v. classifying and segmenting the vascular scaffold(s), if present, in the two-dimensional frame and generating a three-dimensional map of the scaffold on a GPU using a generative machine learning model and knowledge of the scaffold design prior to insertion; vi. Implementing a deep physics-informed neural network on a GPU using knowledge of tissue continuity, vascular structure, blood pressure, and image properties to reconstruct the medial and adventitial layers in the attenuated region; vii. Classifying and segmenting plaque components using a 3D neural network machine learning algorithm; viii. Interpolating the segmented data slices to voxelize them; ix. In another form, step "vii." may include feeding the segmented slice data into a neural field to generate a three-dimensional differentiable density map; x. Generating an adaptable mesh from the voxelized / density structure appropriate for the 3D user interaction and simulation process; xi. communicating the processed steps back to the local system over a secure network; may include:

[0044] In another aspect, step "b." also automatically: i. pre-processing, on a CPU or GPU, one or more temporal angiogram and / or ventriculogram acquisitions or image sequences; ii. communicating the pre-processed and segmented data from the local systems over a secure network to a centralized cloud computer or containerized instance; iii. Segmenting the epicardial vasculature using numerical and / or machine learning based algorithms; iv. Inputting the pre-processed image sequence(s) and segmented vascular structure(s) and gantry orientation(s) metadata into an Angiographic Neural Radiance Field (ANeRF); v. generating, on the GPU, a three-dimensional density map of vascular and / or ventricular structures using the angiographic neural radiance field; vi. Creating an adaptive mesh from the 3D density map appropriate for 3D user interaction and simulation processes; vii. communicating the processed steps back to the local system over a secure network; may include:

[0045] In yet another aspect, step "b." also automatically: i. pre-processing, on a CPU or GPU, a stack or stacks of computed tomography images and / or associated metadata such as axial, coronal, and sagittal planes, and bolus administration time; ii. communicating the pre-processed and segmented data from the local systems over a secure network to a centralized cloud computer or containerized instance; iii. Segmenting vascular and ventricular structures using numerical and / or machine learning based algorithms; iv. identifying and segmenting vascular and ventricular structures using numerical and / or machine learning based algorithms; v. identifying and segmenting plaque components using numerical and / or machine learning based algorithms; vi. Interpolating and voxelizing the segmented data stack; vii. Creating an adaptive mesh suitable for the 3D user interaction and simulation process; viii. communicating the processed steps back to the local system over a secure network; and may include:

[0046] In one form, step "c." also includes: i. acquiring and processing a temporal range of images, rather than a single image frame; ii. analyzing the acquired or processed temporal image data using probabilistic programming and / or machine learning based algorithms; iii. Extracting relevant image features as a 5-dimensional feature set; may include:

[0047] In another form, step "c." also includes: i. acquiring and processing a time range of patient data or patient features rather than static data points; ii. analyzing the acquired or processed temporal data using probabilistic programming and / or machine learning based algorithms; iii. extracting relevant data features as a multidimensional feature set; may include:

[0048] In one form, step "d." also automatically: i. matching the acquired or extracted data to a feature set or multiple feature sets; ii. generating an extended set of boundary conditions to simulate cardiac or vascular loading of the patient; iii. analyzing the feature set and augmented boundary conditions using a physics-informed machine learning model to obtain a lightweight subset of biomechanical metrics estimated in real time; iv. Analyzing the feature set(s) using computational statistical models and generative machine learning models; v. Visualizing for each data point the obtained feature set(s), computational statistical model and approach; vi. generating a report or dataset for storage in a local or cloud-based electronic medium; may include:

[0049] In one form, step "e." also includes: i. analyzing the feature set(s) from step "d." using computational statistics, probabilistic programming, and / or generative machine learning models; ii. Presenting the feature set(s) and the underlying computational model(s) to a user; iii. Incorporating manual user input from a skilled clinician / technician, including but not limited to selecting or adding appropriate data and computational models appropriate for the patient; iv. predicting the general risk profile of the patient; v. Generating probabilistic scenarios of various treatment option(s) and presenting the scenario(s) in a progression from the strongest option to the weakest option; vi. Recommending or not recommending the use of detailed "Level 2" simulations using a generic risk profile and probabilistic scenario(s); vii. generating a report or dataset for storage in a local or cloud-based electronic medium; may include:

[0050] In one form, step "f." also includes: i. accessing the report and / or dataset from the preceding step from an electronic medium; ii. loading a user profile or incorporating manual input and formatting the visual display to suit the pre-configured settings; iii. Adding the report and / or dataset(s) of steps "a." through "e." to a visual display; and iv. automatically highlighting or presenting statistically significant or important probabilistic data points in a visually conspicuous manner; v. Augmenting the display with five-dimensional (three-dimensional space, time, and other metrics) data from one or more acquired datasets; vi. interactively highlighting areas of interest throughout the vasculature to the user using color, shape markers, or other visually relevant methods; vii. user interactions to modify or enhance the display, including opening or closing additional data displays or adding / removing data points from the five-dimensional display; may include:

[0051] In one form, step "g." also automatically: i. Capturing a user command to proceed to a "Level 2" simulation process; ii. packaging all of the data from steps "a." through "f." and communicating the packaged data to a centralized cloud computer or containerized instance over a secure network; iii. generating coarse and fine meshes of vascular structures, including but not limited to the lumen, plaque components, vessel wall, and epicardial structures; iv. Defining patient-specific boundary conditions for the mesh structure, including but not limited to blood properties and blood profiles, displacement profiles, and electrophysiology profiles; v. Using the acquired and calculated patient data, performing simulations at one time point and / or one heartbeat and / or several heartbeats to determine engineering-based vasculature stress metrics using continuum mechanics principles such as fluid-structure interaction techniques, fluid-structure-electrophysical interaction techniques, computational fluid dynamics techniques, and solid mechanics techniques; may include:

[0052] In one form, step "h." also includes: i. constructing a feature set from "Level 2" engineering-based stress criteria; ii. applying probabilistic programming and machine learning based decision approaches to the "Level 1" and "Level 2" feature sets; iii. calculating continuous and multidimensional biomechanical stress profiling metrics on the coarse mesh from step "g.iii." using step "ii."; and iv. Extracting feature sets of likely patient, vessel, and plaque level(s) outcomes at various time intervals from step "iii." using a generative method; v. adding the "Level 1" and "Level 2" feature set(s) to a secure cloud-based electronic storage medium; vi. communicating the processed steps back to the local system over a secure network; may include:

[0053] In one form, step "i." also includes: i. Retrieving the "Level 1" and "Level 2" feature set(s) from a secure cloud-based electronic storage medium; ii. Using a centralized cloud computer or containerized instance, calculating the variance and / or error of the "Level 2" and "Level 1" feature set(s); iii. incorporating manual input from a skilled technician if the variance / error exceeds a set threshold; iv. Retrieving all relevant patient feature sets from a secure cloud-based electronic storage medium; v. Retraining the machine learning based approach from steps "a.", "b.", "c.", and "d." and the "Level 1" analysis using the data retrieved from steps "i." and "iv."; vi. retraining the model from step "b.", preferably using a cross-imaging modality data augmentation approach; and vii. pushing the retrained hyperparameters and / or new machine learning model to a cloud-based machine learning operations (MLOps) pipeline; and viii. communicating the updated parameters to a local system over an electronic network; may include:

[0054] In one form, step "k." also includes: i. incorporating manual inputs to adapt the visualization to each user's preferences; ii. visualizing the two-dimensional image(s) stack from one or more imaging modalities; iii. visualizing the three-dimensional vasculature from one or more imaging modalities; iv. Identifying areas or data points of interest to the user according to shape or color or other visual markers; v. manual user interaction with the marker to display additional information, such as a graph or data of the prediction; vi. Automatically selecting and displaying the most important data to the user by using the most important pieces of information designated or extracted from the decision-making process of the previous embodiment, rather than static data points; vii. Presenting the data or metrics in both a 3D visualization, such as an "unwrapped" view, or a modified 2D visualization; viii. allowing interactive input to move, rotate or zoom the 2D and 3D visualizations of the vasculature in space and time; may include:

[0055] In another aspect of the present invention, a computer-implemented method for automating the processing and extraction of important features from intravascular images, including overcoming significant imaging system limitations, is proposed, the method comprising: a. Acquiring intravascular imaging pullback / image stacks and associated data at the time of acquisition, including but not limited to blood pressure, heart rate, and partial differential equations based on the physics of the imaging system; b. Pre-processing the image stack to remove unwanted regions, preferably pre-filtering noise or artifacts; c. Passing the pre-processed imaging stack and knowledge of the acquired data / physical properties to general-purpose computing hardware such as a suitable graphical processing unit; d. Segmenting the lumen using a spatiotemporal U-net machine learning architecture that leverages long short-term memory (LSTM) and attention mechanisms to enhance robustness and generalization capabilities for sparse and noisy real-world data; e. In another embodiment, the machine learning model also applies dynamic vertical layering to modify model layers during processing to improve segmentation. f. In yet a further embodiment, adaptive block shrinkage is applied to further reshape the encoder or decoder architecture during processing to improve segmentation; and g. Masking the preprocessed image stack with the resulting lumen segmentation map; h. feeding the masked and preprocessed image stack into a 3D U-Net based machine learning architecture to segment the middle layer of the vessel wall; i. passing the preprocessed image stack, the lumen segmentation map, and the intermediate layer segmentation maps to a preprocessing module of a modified deep physics-informed neural network architecture; j. processing the 3D pixel location, pixel multilayer color data, and multiple segmentation maps with the multilayer perceptron of stage 1; k. extracting global features from the multilayer perceptron and obtaining further data from a pre-processing module, including smoothed 3D vessel centerlines; l. Passing the local spatial information to a Stage 2 multilayer perceptron that has access to tissue continuity, nonlinear tissue properties, imaging physics-based properties, and partial differential equations governing pressure distribution; m. Imposing boundary or initial conditions on the network to aid convergence, such as by using the segmentation mask from steps "d." and "h."; n. Minimizing a network loss function to extract a segmented tissue map containing areas with significant imaging attenuation artifacts and information about tissue properties; o. performing the above steps within a single operation on a graphical processing unit for rapid and efficient processing; Includes.

[0056] In another aspect of the present invention, there is a computer-implemented method for automatically generating a three-dimensional density map or anatomical model (with only a single view) of the vasculature from an invasive coronary angiogram via angiographic neural radiance field (ANeRF) to minimize radiation exposure to the patient while increasing the information available to the clinician, the method comprising: a. obtaining at least one invasive angiogram of the vasculature comprising one or more images over a cardiac cycle; b. extracting C-arm orientation metadata from the acquired image data, including but not limited to primary and secondary angles, detector characteristics, x-ray characteristics, and source position relative to the patient / gantry isocenter and detector plane; c. Preprocessing the angiographic image or image stack using machine learning models or numerical methods to identify vascular structures; d. generating a multi-grid or sub-pixel representation of the vascular structures identified in step "c." to improve rendering resolution; e. In another embodiment, if there are multiple acquisitions from different primary or secondary angles, steps "a." through "d." are performed for each acquisition. f. In another embodiment, the multiple views are aligned using the C-arm coordinate metadata extracted in step "b."; or g. In yet another embodiment, these views are aligned using an energy minimization algorithm to overcome patient motion, including that resulting from acquisition setup errors, C-arm gantry motion, respiratory / cardiac motion, and from table or detector panning; h. Providing a multi-scale representation of the angiographic image(s), the binary mask, and the associated C-arm gantry orientation (after alignment via an energy minimization algorithm) as input to the angiographic neural radiance field; i. rendering a density field of the vasculature in three dimensions; j. In one embodiment, the three-dimensional density field is generated including a three-dimensional vascular connectivity filter to enhance vascular structures and reduce noise; k. In another embodiment, the density field may be processed into voxelized or mesh-based visualization techniques; l. interactively visualizing three-dimensional anatomical structures; Includes.

[0057] In another aspect of the invention, there is a computer-implemented method of acquiring temporal information from invasive coronary angiography imaging and the foregoing embodiments to determine virtual microvascular function, virtual vascular strain, virtual ejection fraction, and other functional measures without the need for additional testing or invasive wires, the method comprising: a. generating a three-dimensional density field of the vasculature using the immediately preceding aspect of the present invention to automatically generate a three-dimensional density map or anatomical model of the vasculature from an invasive coronary angiogram; b. Identifying background features throughout the angiography frame, including but not limited to ribs or vertebrae; c. applying a rigid body transformation to co-register background features across image frames to account for C-arm gantry or patient motion; d. In one embodiment, coregistration may generate a set of augmented images that represent a larger two-dimensional space than any individual image frame; e. In another embodiment, coregistration may generate a variable set of C-arm gantry orientations to account for motion artifacts across several image frames; f. mapping forward-looking and backward-looking images from one or more angiographic frames to a static three-dimensional density field; g. In another embodiment, the co-registered image stack may be used to generate a unique three-dimensional density field for each set of frames over time; h. In yet another embodiment, a static density field is encoded with continuity constraints and deformed over time to mimic a coregisted two-dimensional image stack; i. fitting a predetermined myocardial map to a three-dimensional density field; j. deforming the fitted myocardial map over one or more cardiac cycles to estimate ventricular function, such as ejection fraction; k. In one embodiment, a ventriculogram may be available and used to optimize a given myocardial map or ventricular estimate; l. Reprocessing the density field to extract volumetric changes in density of vascular structures over time; m. In another embodiment, the angiographic neural radiance field (ANeRF) may be preferably modified with additional multi-layer perceptron and Navier-Stokes equations and continuity-based loss function(s) to encode hemodynamics into the vessel density field; n. A step of calculating the dissipation or change in density of the blood vessel density field, and o. Mapping the changes in dissipation or density to a specific vessel segment or vessel segments, or segments of the myocardium, p. In another embodiment, non-vascular regions may be queried for changes in density in two or three dimensions; and q. In such embodiments, the identified dissipation or density changes may be graded and mapped to vasculature or myocardial segments as areas of "blush" or microvascular dysfunction; and Includes.

[0058] In another aspect of the invention, there is provided a computer-implemented method for providing novel intraluminal or intrastructural biomechanics-based metrics that are tailored to a particular patient, but that are generalizable and directly comparable across different patients, the method further comprising: a. generating an expanded set of boundary conditions based on patient characteristics; b. conducting a biomechanical simulation or machine learning implementation method to determine a continuum mechanics based tensor field in the fluid or structural domain using the extended boundary conditions; c. Computing an isosurface of a normalized metric of interest, which may include traditional or novel metrics, from several equally spaced units within the domain -1 to 1 or 0 to 1; d. Taking one or more plane-based slices of one or all of the isosurfaces from step "c." and determining the area contained within each plane-based isosurface slice; e. In another embodiment of "d.", from step "c.", taking one or more plane-based slices of one or all isosurfaces and determining the areas contained within the positive and negative isosurface-based regions; f. Determining the cross-sectional area of ​​the vessel in one or more planes used in steps "d." and "e." g. Calculating the ratio of the isosurface / slice base area to the lumen area, or the ratio of the area of ​​the positive isosurface slice to the area of ​​the negative isosurface slice, from one or more domain units; h. In another embodiment, calculating the extended variability of the isosurface and / or lumen planar area ratios across one or more domain units over the range of extended boundary conditions imposed from unit step "a."; i. generating a visual display or graph or report of metrics based on the expanded endoluminal biomechanics; Includes.

[0059] In yet another aspect of the invention, there is a computer-implemented method for selecting, distributing, and using available data to predict or identify an outcome or characteristic of a patient's vasculature, the method further comprising: a. Obtaining the various input metrics identified throughout the embodiments shown and described herein; b. Determining a statistical or probabilistic spatiotemporal distribution of the continuous metric; c. performing a multilevel discretization of the statistical or probabilistic spatiotemporal distribution to emphasize or improve the weighting of important locations or outcomes that might otherwise be overlooked or downplayed; d. Binning the discretized metrics or the entire metric in a multi-level, multivariate feature binning process; e. weighting or shifting bins using patient features to optimally capture data from one or more metrics; f. Implementing the bins as input or hidden layers in a fully connected network to capture non-linear features and interactions; g. automatically pruning connections through the network, preferably in a parallel process, but also in a serial process, to optimize feature propagation; h. Providing the likelihood of an outcome, the location or statistical probability of a particular feature being presented, or the probability or predicted success rate of one or more therapeutic intervention(s) or treatment pathway(s) for multiple parallel endpoints; Includes.

[0060] The visual display may be generated by a designated visualization tool or by designated hardware.

[0061] Preferably, one or more geometric / morphologically based metrics for a vessel or plaque may be selected for visualization or further analysis from the following group, including, but not limited to, volume; tortuosity: curvature; stenosis ratio; minimum lumen area; lesion diffusivity; lesion length; branching angle; opening location; plaque composition (lipidic, calcific, fibrotic, necrotic, complex); epicardial adipose tissue; plaque eccentricity; lipid volume; lipid length; calcium volume; fibrous cap thickness; presence of cholesterol crystals; presence of microchannels; macrophage index; presence of thrombus; presence of rupture; vessel wall thickness (intima, media, adventitia); and subsequent derived features from these metrics, such as atheroma volume fraction as outlined in the illustrated embodiments.

[0062] The geometry / morphology-based metrics may further be selected from a group that includes the temporal variation of each metric over one or more partial or complete cardiac cycles.

[0063] Function-based metrics can be calculated from angiogram images and measured electrocardiogram and blood pressure data, and can include data sources such as wearable sensors, eliminating the need to insert additional wires into the patient's circulatory system.

[0064] Preferably, the function-based metrics may be selected from the following group, including but not limited to: virtual microvascular function (vMF); virtual ejection fraction (vEF); virtual pulse wave velocity (vPWV); virtual arterial distensibility; virtual augmentation pressure; contrast agent pooling; virtual vascular strain (vVS); and subsequent derived features of these metrics, including temporal changes over one or more cardiac cycles as outlined in the illustrated embodiment.

[0065] Preferably, metrics can be derived from intravascular imaging, including, but not limited to, arterial wall properties (i.e., stiffness, Young's modulus, and nonlinear material modulus); stent strut malapposition; inflammatory or biological response; and subsequent derivation of these metrics from the illustrated embodiment or various intravascular catheter systems (i.e., those obtained from near-infrared fluorescence).

[0066] Preferably, the fluid mechanics-based metrics may be selected from the following group, including but not limited to: pressure drop; wall shear stress; velocity; helical flow; and subsequent deformation characteristics of these metrics, including wall shear stress gradient; transverse wall shear stress; cross-flow index; axial shear stress; secondary shear stress; wall shear stress divergence; critical point characteristics; wall shear stress exposure time; H1-H4 helical flow; and their deformation characteristics over one or more cardiac cycles.

[0067] Preferably, the fluid dynamics-based metric may further be selected from the following group: Invariant manifolds; Lagrangian coherent structures; the ratio of intraluminal flow to area; Intraluminal flow imbalance ratio; turbulent kinetic energy; and Fluid Strain Rate Includes.

[0068] Preferably, the solid mechanics based metrics may be selected from the following group, including but not limited to: displacement; principal stress; principal stress gradient; principal shear; principal strain; tensor divergence; and subsequent derived features of the Cauchy stress tensor, including temporal variations over one or more cardiac cycles.

[0069] Preferably, the solid mechanics based metric may further be selected from the following group: structural axial shear magnitude; structural secondary shear magnitude; the magnitude of structural radial shear; the magnitude of the axial principal stresses and the normalized misalignment (from the axial vector); the magnitude of the second principal stress and the normalized misalignment (from the second vector); the magnitude of the radial principal stress and the normalized misalignment (from the radial vector); Invariant manifolds; Includes.

[0070] Preferably, the metrics may further be selected from available patient characteristics including, but not limited to, the following clinical symptoms or records and lifestyle factors: stable or unstable patient; ST-elevation myocardial infarction (STEMI); non-ST-elevation myocardial infarction (NSTEMI); non-obstructive coronary artery myocardial infarction (MINOCA); occluded vessel(s); electrocardiogram (ECG) factors; heart rate; blood pressure; troponin; cholesterol; smoking status, body mass index (BMI), and gender.

[0071] In a preferred form, the steps of the method are contained within the algorithm of a software program.Accordingly, in another aspect of the invention, a software program is proposed for carrying out at least some of the steps of the above method.

[0072] The software program may be implemented as one or more modules for performing the steps of the invention on a computer system. The modules may package functional hardware units for use with other components or modules. The reader will recognize that the method steps may be performed using multiple central processing units (CPUs) or graphical processing units (GPUs), either in a single or multiple geographic locations, or in the cloud.

[0073] Associated application software may be stored on a computer-readable medium such as an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, mechanism, device, propagation medium, or computer memory. In one possible embodiment, the systems described herein include hardware coupled to a microprocessor, microcontroller, system-on-chip ("SOC"), or any other programmable device.

[0074] In yet another aspect of the present invention, an apparatus is proposed for carrying out any of the aspects of the above method, which may include embedded software or firmware along with corresponding hardware designed to perform one or more dedicated functions.

[0075] The apparatus may also include a processor(s) and a memory component(s) in which data is temporarily stored before being transmitted at predetermined intervals or queried by the device to retrieve the data. The memory component(s) may be non-volatile, flash, or cache storage device(s).

[0076] The processor(s) and memory component(s) cooperate with each other and with other components of the computer or computers to perform the functions described herein, some of the functionality described herein can be implemented in special purpose electronic devices that are hardwired to perform the described functions.

[0077] Communication between components of the apparatus may occur via long-range or short-range networks, such as, but not limited to, low-power wireless networks, microwave data links, 3G / 4G / 5G communication networks, BLUETOOTH®, BLUETOOTH® Low Energy (BLE), Wi-Fi, LoRa™, NB-IOT, Ethernet, Fibre Channel (FC), other types of wired or wireless networks, or may be connectable to devices utilizing such network(s).

[0078] Some of the components of the system may be connected via communications devices such as, but not limited to, a computer network such as a modem communications path, a local area network (LAN), the Internet, or a fixed cable.

[0079] Some embodiments of the system may communicate in real time through the aforementioned system to process one or more modules at one or more physical location(s), while a user interacts with one or more module(s) at another physical location(s). The device may utilize a cloud server and may include embedded software or firmware along with corresponding hardware designed to perform one or more dedicated functions of the present invention.

[0080] The specified software program may alternatively be stored on a computer-readable medium on a storage device such as a hard drive, a magneto-optical disk drive, a CD-ROM, an integrated circuit, a radio or infrared transmission channel between the computer and another device, a computer-readable card such as a PCMCIA card, a flash drive, or any number of other non-volatile storage devices, either as a stand-alone device or as part of a dedicated storage network such as a storage area network (SAN).

[0081] The foregoing are merely examples of relevant computer-readable media, other computer-readable media may be practiced without departing from the scope of the invention.

[0082] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments of the present invention and, together with the detailed description and claims, serve to explain the advantages and principles of the invention. [Brief explanation of the drawings]

[0083] [Figure 1] 1 is a flow chart of a system for providing evidence-based prognosis / prediction and visualization to clinicians. [Figure 2] Node-based flowchart showing the process at each node (i.e., clinic / hospital). [Figure 3]1 is a flowchart of a centralized cloud computer or containerized instance for performing detailed analysis based on data obtained from each node. [Figure 4] FIG. 1 is a schematic diagram of exemplary computer hardware and / or systems on which the embodiments described herein may be processed. [Figure 5] FIG. 1 is a schematic diagram of an intravascular machine learning approach to segment various features while overcoming limitations in imaging systems. [Figure 6] 1 is a summary of invasive coronary angiography (ICA) acquisition characteristics and pre-processing for embodiments described herein. [Figure 7] FIG. 1 is a schematic diagram of a machine learning workflow for segmenting and reconstructing 3D vasculature via angiographic neural radiance fields (ANeRF). [Figure 8] 1 illustrates a process for processing temporal information from an invasive angiogram that includes a virtual assessment of ventricular function. [Figure 9] FIG. 1 is a schematic diagram of a multi-level segmentation that can result from the previous embodiment. [Figure 10] 1 is a schematic diagram of a blood vessel illustrating contrast agent flow and dissipation characteristics for assessment of microvasculature and / or functional properties from angiographic images in accordance with disclosed embodiments. FIG. [Figure 11] FIG. 1 is a flowchart of a process for quantifying microvascular function in invasive coronary angiography without additional invasive wires, illustrating one of its applications in reinforcing boundary conditions for general measurement evaluation. [Figure 12] 1 shows a flowchart for coregistrating or augmenting multiple imaging modalities into a single spatiotemporal model for both analysis and visualization purposes. [Figure 13] Illustrates data or feature selection for "Level 1" and "Level 2" analysis. [Figure 14] FIG. 1 is a schematic diagram of recording multiple events or data entries over time for a single patient within the scope of the proposed embodiment. [Figure 15] Schematic of a machine learning decision-making approach. [Figure 16] 1 is an exemplary overview of intravascular imaging visualization and user interface. [Figure 17] 10 is a further example of a simplified user interface, including predictive and demographic comparisons. [Figure 18] 1 is an example of an enhanced user interface that includes data from multiple modalities, analysis, and / or prediction results in a single interface. [Figure 19] Examples of indicative performance are shown below. DETAILED DESCRIPTION OF THE INVENTION

[0084] Like reference characters indicate corresponding parts throughout the drawings. Dimensions of certain parts shown in the drawings may be modified and / or exaggerated for purposes of clarity or illustration.

[0085] Referring to Figure 1, a computer system is defined for implementing the methods described herein, which, in at least some embodiments, generates a predictive model of a patient's arteries / vasculature based on artificial intelligence and biomechanical simulations to present a risk analysis predicting future coronary artery disease changes and suggest optimal treatment pathways. The flowchart illustrates any number of connected nodes, Node 1 through Node N, which may operate as independent sites (such as clinics / hospitals), individually, or as connected services, and may or may not be connected to various third-party clouds or data systems, such as patient archiving and communication systems (PACS). Within each node, data is obtained from one patient or multiple patients from local data sources or from connected third-party clouds or data systems. The data is preprocessed on one or more computing devices and associated hardware, an embodiment of which is outlined in more detail in FIG. 4 , preferably by a dedicated software program or programs, which may reside locally or in cloud-based form. Preferably, such preprocessing includes quality checks (which may include handling missing or null data inputs, image visual quality assessment, various case filtering or correction, metadata extraction and logging, and / or de-identification). This data may be transferred to centralized computing instance(s) via a proxy server over a dedicated wide area network (WAN) for further analysis. Communication through the nodes to the centralized computing instances may also be performed through various other communication media or networks. The preprocessed data may also be displayed on the nodes, preferably locally using dedicated software as shown in the embodiments of FIGS. 16, 17, and 18, or by various display hardware, firmware, or dedicated techniques.The pre-processed data may also be passed to a data storage medium, either directly or via proxy server(s), either locally

[0107] or cloud-based [117 and 118].

[0086] The centralized cloud computer system preferably receives the preprocessed data via a WAN and also via other network interfaces and communication protocols. The data may be received via an application programming interface (API) server

[0108] . The API server

[0108] may be a dedicated server or may form part of a master / control node

[0109] . The master / control node

[0109] itself preferably comprises three or more control planes for preparing high-availability (HA) cluster services. The master / control plane and / or API server preferably validate input data and may prepare or configure object instances for compute nodes

[0111] , pods

[0112] , and other service component-level interactions, such as through a container management system including Kubernetes. In one embodiment, compute nodes may communicate with the master / control plane and receive instructions (computer programs or sets of instructions) to launch pods via different levels of general-purpose hardware (see FIG. 4). In another embodiment, the compute nodes may communicate with a master / control plane and receive instructions to launch several parallel pods or parallel pods on one or more compute nodes. In another embodiment, the management system(s) or compute platform(s) may include Docker, OpenShift, Amazon Web Services, Microsoft Azure, and related variations to manage and launch pods or container-based instances and pipelines. The master / control plane(s) and / or API server(s) preferably also communicate between the compute nodes, a WAN, and preferably data server(s)

[0113] , which launch a storage area network (SA) that can be configured to include a variety of volatile, non-volatile, or flash memory technologies, and electronic data storage devices

[0114] .The centralized computing system may also communicate with third-party clouds or data storage systems via various protocols. The centralized computing instances and SANs are accessible from authentication nodes where users, including clinicians, technicians, and patients, can access and visualize data, results, and reports, and instruct the system to perform further processes.

[0087] FIG. 2 shows a node-based flowchart clearly illustrating the process at each node (i.e., clinic / hospital). Preferably, data is acquired as patient-specific data from a local electronic network or a third party connected to a cloud-based system. This data includes, but is not limited to, manual input from a skilled technician or clinician, such as structural, functional, or chemical / biological imaging, blood pressure / blood flow velocity / catheter-based measurements, presentations (which may include ST-elevation myocardial infarction [STEMI], non-ST-elevation myocardial infarction [N-STEMI], non-obstructive coronary artery myocardial infarction [MINOCA]), and various other clinical notes and stable or unstable patients. The acquired data is preprocessed to address ambiguity, noise, and missing data values ​​using a computing device or multiple computing devices and associated hardware, an embodiment of which is outlined in more detail in FIG. 4, and a dedicated software program or programs, which may preferably reside locally or in cloud-based form. Preferably, such preprocessing may include quality checks, including, but not limited to, handling missing or null data inputs, image visual quality assessment, various filtering or corrections, metadata extraction and logging, and / or de-identification. Concurrently, in one embodiment, the preprocessed data is passed to a centralized or cloud-based computing instance via a proxy server over a communication network or WAN of one or various protocols. Communication to the centralized or cloud-based computing instance via the network or WAN and proxy server may also be based on local hardware capabilities and usage details, including, but not limited to, the number of parallel processors, system and cache memory, number of graphics processing units, and graphics or tensor cores and graphics cache details, and various related embodiments.

[0088] In another embodiment, in parallel with the aforementioned steps, the processed data is passed to a student machine learning model

[0206] , whose features and / or design and / or weights are obtained from the proxy server

[0204] and the centralized or cloud computing instance

[0205] via a machine learning operations (MLOp) pipeline. The student model can preferably be modified by the teacher model to optimize or meet the local hardware requirements passed to the proxy server and the centralized / cloud computing server through the aforementioned steps. The student model then performs a "Level 1" analysis

[0207] on local general-purpose hardware (referred to here as "local Level 1" analysis), whose hardware features and protocols are outlined in FIG. 4 and may include a general-purpose central processor or graph processing unit or accelerator to achieve real-time analysis. At the same time, the “Level 1” analysis is performed on a centralized or cloud-based compute instance

[0205] , which is referred to herein as an “advanced Level 1” analysis, preferably optimized to achieve all or some of the analysis results not possible on local hardware within the required timeframe (i.e., near real-time).

[0216] This “advanced Level 1” analysis is communicated to the local node

[0201] via a proxy server and concatenated with the “local Level 1” analysis.

[0208] In another embodiment, if local hardware or firmware requirements do not provide sufficient processing power, the entire “Level 1” analysis may be performed via a centralized or cloud-based compute instance. In another embodiment, the entire “Level 1” analysis may be performed locally. It should be appreciated that such an approach is designed to optimize available hardware resources, such as those illustrated in FIG. 4, across multiple sites or locations."Level 1" analysis may also preferably include processing one or more two-dimensional images, generating a three-dimensional map of the vasculature and its static and / or temporary anatomical structures, calculating a set of metrics using various embodiments, and using the set of metrics to provide one or more analytical and predictive models.

[0089] After the "Level 1" analyses are concatenated, a report and recommendations are generated, and a decision is made regarding the need for a more advanced "Level 2" analysis.

[0210] The "Level 1" analyses may preferably suggest or recommend the need for a "Level 2" analysis, or that decision may be made by an experienced user. If a decision is made to perform a detailed "Level 2" analysis, the preprocessed data and the "Level 1" analyses are passed through an electronic network and proxy server for processing on a centralized or cloud-based computing instance.

[0211] This step is further detailed in FIG. 3. Otherwise, the results are visualized or displayed to the user as described in the embodiments described herein.

[0212] Preferably, this display may include the "Level 1" or "Level 2" analyses, or both, and associated metrics or multidimensional visualizations. In another embodiment, this display preferably includes data from one or more imaging modalities, which may be different types of imaging, expanded into a single user interface and display. Prior to completion of the analysis

[0213] , the data is archived or stored on local electronic storage media or third-party cloud

[0215] and data storage systems

[0214] that can be accessed at any future stage by an authorized user or patient. If the analysis is incomplete, the process can proceed to obtain further data

[0202] .

[0090] Figure 3 is a flowchart of a centralized cloud computer and containerized instances for performing detailed analysis based on data acquired from each node. The centralized cloud computer or containerized instance(s) may simultaneously connect to one or more nodes, preferably via a proxy server

[0301] , and acquire data from the embodiment included in Figure 2, including preprocessed data and "Level 1" analysis

[0302] . The two parallel operations are performed on one or more pods or compute nodes and associated general-purpose hardware and firmware components, depending on scheduling by the master / control node. First, "Level 2" analysis is prepared based on the acquired node-based data and on a data server

[0313] and electronic storage medium

[0314] . These may preferably include features or weights of pre-trained machine learning models and empirical multidimensional physical and physiological laws. Information from structural imaging features, including various forms of 3D models, including 3D image stacks, 3D density fields, 3D adaptive meshes, or 3D point clouds, is first discretized by domain

[0306] . In one embodiment, domain discretization may preferably involve the process of dividing features into finite elements or finite volume elements. In another embodiment, boundary intersections or contact regions may be calculated based on other mesh-based descriptions of three-dimensional features, broadly encompassing the related fields of fluid mechanics, structural mechanics, electromechanical coupling, structural-fluid coupling, and chemical-mechanical coupling. Discretized domain properties are applied based on empirical multidimensional physics or multidimensional physiology, such as estimated nonlinear tissue properties extracted from imaging modalities (see the exemplary embodiment of FIG. 5 ), which is considered just one exemplary case. Constraints, such as boundary conditions or initial conditions, are also defined based on acquired data, which may preferably include measured or input data points from each specific patient, or may also include augmented data if null or missing inputs are detected from the aforementioned embodiments.After defining the constraints, the partial differential equations are solved, in one embodiment, by finite element or finite volume methods, or, in another embodiment, preferably by a neural network and associated loss function to generate one or more metrics. The calculated metrics are then used to determine a unique biomechanical stress profiling index (BSPI), which preferably uses all metrics to identify outcomes. In another embodiment, the BSPI identifies or predicts one or more independent or correlated outcomes using a subset of metrics selected using the embodiment described in FIG. 15. In parallel with this process, the student machine learning model and the teacher machine learning model are independently evaluated and passed to a deep variational autoencoder network along with the results from the aforementioned BSPI analysis and stored data from a local or cloud data server database. The autoencoder network preferably performs an unsupervised, low-dimensional latent representation of the detailed "Level 2" analysis and uses the error or variance between the teacher model and the student model to reconstruct a detailed teacher model and a lightweight student model suited to the requirements of the local node-based hardware or firmware. In another embodiment, the autoencoder may receive only the student model and the teacher model, and instead perform federated learning optimization using features from the local node-based student model and the global teacher model to reoptimize or reconstruct the local student model from the teacher model, again suited to the requirements of the local node-based hardware or firmware, without passing on patient details. The completed model(s), feature(s), trained weight(s), or other data are then passed via a data server

[0313] and associated networks and storage media

[0314] . The associated "Level 2" analysis is preferably passed back to the node for interactive visualization by the user via a proxy server and communication network.

[0091] In various embodiments, the systems and methods may preferably be implemented on or using the general-purpose computing components shown in FIG. 4 . In one embodiment, the computing component may include a central processing unit (CPU) 0401 with various levels of processor cache 0402 coupled to system memory 0404 via an input / output (I / O) bus 0403. In another embodiment, the computing component may also include a graphical processing unit (GPU) 0405 or acceleration component such as a tensor processing unit with various levels of graphical cache and memory 0406 that communicates with the system memory 0404 and other system components via the I / O bus 0403. The system memory is preferably configured to store data or code for rapid access to the CPU(s) and GPU(s) / accelerator(s) and may be configured to include volatile memory, non-volatile memory, or flash memory technology, and derivative features of such technologies. The components may also include I / O controllers 0408 that can access internal or external data storage media 0409 and / or various types of networks and connectivity devices 0410. In one such type, wired or wireless data communications to the storage network may include Ethernet or Fibre Channel (FC), low-power wireless networks, microwave data links, 3G / 4G / 5G communications networks, BLUETOOTH®, BLUETOOTH® Low Energy (BLE), Wi-Fi, LoRa™, and NB-IOT communications for communicating to computer-readable storage media such as electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, mechanisms, devices, propagation media, or computer memory components, preferably located and managed by data server(s) in a storage area network (SAN), which may include the aforementioned hardware.

[0092] In yet another embodiment, the computing components may include single-processor or multi-processor CPU

[0401] systems, or various or identical architectures of several processors capable of executing instructions or computations related to the embodiments described herein. In one embodiment, the multi-processor components may communicate via a message-passing interface (MPI)

[0407] , which may also preferably communicate between servers each including single or multiple CPU processors via various communication or network protocols. In other embodiments, other message-passing protocols may be used for parallel processing of instructions on one or more processors and / or servers. The computing components may also include one or more GPU or acceleration devices

[0405] of various or identical architectures for performing instructions, and similarly may communicate between devices and servers using one or more multi-GPU / accelerator components via various communication or network protocols.

[0093] Figure 5 is a schematic diagram of an intravascular machine learning approach that segments various features while overcoming limitations within imaging systems, including, but not limited to, the imaging system's limited tissue penetration depth, susceptibility to artifacts including residual blood from inadequate clearance and rotational distortion. Intravascular imaging pullbacks

[0501] are acquired as axially stacked 2D slices, and the entire stack of acquired images is passed to a spatiotemporal U-net machine learning architecture

[0502] that leverages long-short-term memory (LSTM) and attention mechanisms to enhance robustness and generalization capabilities for sparse and noisy real-world data. The encoder module

[0503] , visualized in a downsampled version of the architecture, is built on a ResNet backbone modified to incorporate 3D blocks to improve segmentation continuity across a series of 2D imaging slices. Vertical dynamic layering

[0505] enables the architecture to adapt to the dataset, overcoming the limitations of static network design. Auto-adaptive block shrinkage

[0507] further improves feature extraction and generalization by varying the convolutional blocks at each level as the dataset changes. 3D max-pooling

[0506] in the encoder downsamples the feature maps, while the decoder module upsamples them in 3D via trilinear interpolation

[0510] . Traditional skip connections correspond to the number of dynamically provisioned layers

[0508] . The 3D temporal convolutional decoder block

[0504] is constructed from dual 3D convolutions, 3D batch normalization, and a rectified linear activation function, passing to a single long- and short-term memory layer with visibility of the entire image stack, as well as an attention mechanism and a final activation function. The output segmentation map

[0511] generates a mask of the lumen, including bifurcation regions, across the entire imaging stack

[0512] .

[0094] Using the lumen segmentation map, the input image stack

[0513] is masked into a modified 3D DenseNet-based decoder architecture

[0514] to identify visible components in the hidden layer. The 3D decoder block

[0515] uses the same vertical dynamic layering

[0517] and 3D max-pooling

[0518] as the previous model, with similar cross-dense connections

[0519] and concatenation in the decoder block

[0516] . These expand the receptive field with large dilations, which consist of dual 3D convolutions, 3D batch normalization, and leaky rectified linear activation functions to improve segmentation of small but important and noisy hidden layer features. The output layer

[0521] produces a binary stack

[0522] , which is applied together with the lumen segmentation map and the original image stack for the final stage.

[0095] The original image stack

[0501] , lumen segmentation map

[0512] , and intermediate binary mask

[0522] are passed as inputs to a preprocessing block

[0525] before being applied to a modified deep physics-informed neural network architecture

[0523] . The preprocessing block determines mask-based centroids to generate smooth vessel centerlines (as opposed to the catheter centroid, which is located at the image center), and then feeds the 3D pixel coordinates

[0526] , associated pixel color data, and segmented lumen and visibility hidden layer maps

[0527] as inputs to Stage 1 of the modified physics-informed neural network. These inputs are concatenated and fed to Stage 1 multilayer perceptron(s)

[0528] , consisting of fully connected layers where activation and batch normalization occur, before max pooling

[0529] to generate a global feature set

[0530] from the image stack and segmentation map. The global feature set also extracts distinctive features that can be directly identified from the preprocessing block (i.e., features selected by an algorithm or by an experienced user prior to automated processing), such as the lumen centroid for each frame. The local features

[0528] are also fed forward and concatenated with the global feature set before the second-stage multilayer perceptron(s)

[0532] . Predetermined partial differential equations

[0524] governing tissue continuity, nonlinear tissue properties, and imaging characteristics (including spectral, frequency, and time-domain optical properties in the case of optical coherence tomography imaging systems) are combined with blood pressure measurements, spatial derivatives from the multilayer perceptron(s), and preprocessed information from the lumen and visible media

[0525] to generate a customized loss function. The latter (loss function) further provides initial and boundary conditions to improve convergence. The customized loss function can preferably backpropagate features throughout the network to constrain pixel / image-based network segmentation with knowledge of the imaging system and physical properties of the vasculature.In another embodiment, the predetermined partial differential equations (0524) may impose physical properties associated with one or more imaging systems, and it should be recognized that the embodied methods may be applied to other intravascular imaging systems without departing from the scope of the present invention. Outputs include a segmentation map (0534) containing plaque components and vascular structures in attenuated areas suitable for various voxel- or density-based 3D reconstructions, and an estimated tissue property map (0533).

[0096] The overall method is implemented on a graphical processing unit (GPU) after taking the imaging stack and associated physiological data system as input from memory. Four outputs are generated, including lumen segmentation, visible medial layer segmentation, segmentation of adventitia wall and plaque components, and estimated associated tissue properties, which are communicated from video memory back to system memory upon completion of the process. Inter-process communication between the lumen, visible medial layer and adventitia / plaque segmentation, and fitting is handled by a video cache, while the original image stack is stored in video memory for rapid processing as a single operation, the general-purpose hardware components of which are illustrated in Figure 4.

[0097] FIG. 6 outlines key features of invasive angiography and related embodiments. Referring to FIG. 6A, characteristics of an angiography C-arm machine

[0601] that may be useful for subsequent embodiments are defined, including the primary angle

[0602] and secondary angle

[0603] , the location and spatial characteristics of the detector module

[0604] , the location of the X-ray source point

[0607] , the distance from the X-ray source to the C-arm isocenter

[0606] , and the distance from the X-ray source to the detector module

[0605] . The detector module captures X-ray characteristics to generate image sequences

[0608] of vascular structures over the cardiac cycle(s), which may be illuminated by injected contrast and not visible throughout the image sequence. Machine learning or numerical approaches are used to identify vascular structures throughout the image sequence and generate binary maps

[0609] (see FIG. 7 for further details regarding these embodiments). It should be appreciated that these binary maps are used to highlight vascular structures in subsequent processing steps. Unlike previous approaches that attempt to purely identify vascular structures or remove / filter background noise, our approach further exploits this background noise in novel embodiments of subsequent examples. Referring to Figure 6D, a binary map is used to generate a temporal multigrid over the entire vascular region and its boundaries by incorporating the original image sequence

[0610] , pixel locations

[0611] , and pixel boundaries

[0612] , which are used to construct an angiographic neural radiance field with the spatial structure shown by Figure 6E. It should be emphasized that the selection of the circular domain at pixels

[0611] and pixel intersections

[0612] is not a typical single-beam ray projection

[0616] , but rather the frustum of a cone-beam

[0614] used to represent X-ray projections on a plane. The C-arm input [602-607] is used to align the angiographic multigrid representation

[0610] in three-dimensional space

[0613] .The described method may preferably take at least one angiographic plane as input, including metadata regarding gantry orientation, and generate or render a subsequent optimal 2D projection

[0618] or vasculature to assist patient assessment or decision making or further algorithm development. In another embodiment, the method may take several input planes [613 and 618] to generate an imported 3D density map. Unlike the pinhole camera model, which places the rendered 3D scene beyond the image plane (whereby rays emitted from the camera pass through the image plane and are then projected into the near and far fields of the camera view, which can then be sampled), in invasive angiography, the "scene" (patient) is placed between the source (x-rays) [607 and 617] and the imaging plane (detector) [604 and 613], and the resulting image can be thought of as a "shadow" cast by tissue or contrast agent density (hence, the x-rays first pass through the "scene" before emitting from and being captured by the detector, necessitating a redefinition of the sampling strategy and failing to capture visible radiance or color). At or near the isocenter of the C-arm system, the cone beam contains an integrated encoding of position, size, and density

[0615] in the C-arm gantry coordinate system.

[0098] FIG. 7 is a schematic diagram of a machine learning workflow for segmenting and reconstructing 3D vasculature via angiographic neural radiance fields (ANeRF). In one embodiment, this process can also be considered angiographic neural impedance fields (ANIF) because the process of generating a density map from the C-arm gantry coordinate system occurs from X-ray absorption between the source and detector plane (thus obstructing the X-ray path and creating a meaningful shadow). In one embodiment, the first stage of angiographic processing involves segmenting a temporal image stack (0701) to generate a binary mask (0712) of vascular structures. A modified U-net architecture (0702) is used with the stack input to a modified temporal DenseNet-based encoder (0703) to identify vascular structures illuminated by contrast agents. In another embodiment, the stack is input to a standard 2D DenseNet architecture for individual image processing. In yet another embodiment, individual images can be processed with numerical processing such as a Frangi vessel filter. In the temporal embodiment, the 3D decoder block

[0703] uses the same vertical dynamic layering

[0705] and 3D max-pooling

[0706] as the previous model, with similar cross-dense connections

[0708] and concatenation in the decoder block

[0704] . These extend the receptive field with large dilations, which consist of dual 3D convolutions, 3D batch normalization, and leaky rectified linear activation functions to improve segmentation of small but important and noisy hidden features. The output layer

[0711] produces a binary stack

[0712] , which is applied together with the lumen segmentation map and the original image stack used in the final stage.

[0099] If two or more angiographic image stacks are available for either the left or right vasculature, the preprocessing step first performs coregistration of the stacks to overcome imaging system artifacts and misalignments common in invasive angiographic systems. Coregistration preferably uses identifiable features, such as bifurcation regions, between image stacks to minimize a misalignment function, which aims to minimize the distance between each set of ray tracing projections between identified feature(s). The function takes these feature locations and the orientation of the C-arm gantry as inputs and introduces the magnification, the x' and y' detector misalignment distances for each view, and a global vector for C-arm source-detector misalignment. In another embodiment, the misalignment function may include a temporal morphing coefficient, which uses quadratic interpolation to bridge large temporal gaps between angiographic frames, shift features forward or backward in time, and morph identified image frames to better address ambiguities in cardiac gating. The minimization function, which can preferably be solved by a general-purpose optimization algorithm, returns a set of offset corrections to be applied to each angiographic view. The original image stack

[0717] and its multigrid representation

[0716] and the processed binary stack

[0715] are then passed as inputs to the angiographic neural radiance field, along with information about pixel scaling characteristics (i.e., detector size), primary and secondary orientations, source to isocenter and detector distances, and offset corrections (if available from preprocessing steps). Unlike traditional neural radiance field approaches and their subsequent derivatives, the embodied angiographic neural radiance field (ANeRF) does not encode color information, but only extracts three-dimensional densities

[0721] from its multilayer perceptron(s), while reversing the order of projections that address the scene location

[0719] between the source

[0718] and the detector plane

[0717] .

[0100] FIG. 8 illustrates steps to leverage features of the previously described ANeRF embodiment to improve both vascular transient analysis and ventricular reconstruction to enable virtual assessment of ventricular function, either with or without additional ventriculography. Because ventriculography occurs in only about 50% of invasive angiography procedures and requires additional radiation exposure and procedure time, the ability to extract similar features without ventriculography is important to clinicians. Referring to FIG. 8A, due to the transient nature of invasive angiography and the limited angiography view window, the acquisition will contain significant motion artifacts. To account for heart, table, and detector motion, each frame across the image stack is processed using the previously described embodiment to identify vascular structures. The processed data stack then registers the x-ray source projection

[0801] to the detector position, followed by the segmented mask(s), original image(s), and multigrid representation(s)

[0802] . Coregistration or motion adaptation may result in a rigid body transformation of the entire system

[0805] . In other cases, the detector may be shifted to view a wider region of the vasculature, necessitating a realignment of the coordinate system to the new detector-source projection region

[0804] in each frame of the acquisition. Images throughout each temporal stack [802 and 803] are then registered to various states of the cardiac cycle and various C-arm gantry coordinate systems, so that the frames are preferably "stitched" together to expand the effective view of each individual frame to the entire region covered during detector or C-arm gantry motion. This "stitching" is illustrated with reference to FIG. 8B , whereby an initial frame (e.g., frame 15)

[0806] from an acquisition is combined with a subsequent frame (e.g., frame 24) from the same acquisition, where the subsequent frame has been affected by detector or other motion of the C-arm system or patient

[0808] , to generate a new image region

[0807] . The frames are stitched together by using an inversion of the segmented vasculature, instead generating a center-weighted map of background tissue across each frame.Background tissue structures are preferably center-weighted using important features closer to the center of each image for coregistration, since features at or near the image borders may not be visible across the entire stack due to motion artifacts. These background structures, such as ribs or spinal bones, are then used to align each frame, accounting for motion artifacts related to the C-arm gantry or patient. The aforementioned embodiment of ANeRF is then preferably applied across the entire stitched image stack, generating a differentiable density field across the vasculature, something not possible with prior approaches.

[0101] Referring to FIG. 8C, a stitched representation of the vasculature allows visualization of the entire vasculature, additional regions that may not fit within a single image view, and an idealized surface map of the ventricle

[0810] is obtained from a demographically adjusted set of volumetric imaging data

[0809] . In one embodiment, the ventricular surface map may represent both the left and right ventricles. In another embodiment, the surface map may represent only the left ventricle. If left ventriculography is available or performed, the cross-sectional anatomy of the ventricle may also be used to adapt the surface mesh to better fit the imaged ventricle. In another embodiment, non-invasive imaging, such as echocardiography or computed tomography data, of the ventricular structure may be obtained from patient data or a third-party source, such as a PACS system, and used to augment the ventricular surface map. Referring to FIG. 8D, a flowchart of the process of co-locating a ventricular surface map with a 3D vasculature model is presented. From a 3D density map of the vasculature, preferably created using the stitched image frames, 3D centerline(s) of the entire vasculature are extracted by a numerical method such as volumetric thinning.

[0811] The ventricular or myocardial surface map is then registered and resized to minimize the distance between the functional surface and the centerline of the vasculature.

[0812] In one embodiment, the surface map may have weighted regions for specific epicardial vessels to assist in accurately aligning the centerline and the surface.

[0813] A distance map is then generated for equally spaced discretized regions of the centerline(s) to identify the distance between the desired location (centerline) and the closest current location of the surface.

[0814] In one embodiment, the surface may be meshed, and constraints on mesh properties, such as mesh smoothness or stiffness, may be applied. The distance map is then iteratively minimized by fitting it to the centerline(s) of the vasculature and deforming the ventricular surface map / mesh to generate an estimate of the patient's heart surface and ventricular shape.

[0815] In one embodiment, the process may be repeated for multiple phases of the cardiac cycle to generate an estimate of ventricular function over one or more heartbeats.In another embodiment, the final deformed ventricular surface / mesh may be further moved / deformed using the transient motion of the vasculature centerline to generate an estimate of ventricular function over one or more heartbeats. The evolving surface map / mesh is then preferably used along with information about vascular function from the previous embodiment to generate real-time virtual ejection fraction, wall motion index, and wall strain function(s).

[0102] FIG. 9 is a schematic diagram of multi-level segmentation, enabling the previously described embodiments, from the major epicardial arteries and myocardial perfusion regions to high-quality discretization of diseased regions for use in simulations or calculations using machine learning or continuum mechanics methods. The major epicardial arteries

[0901] and myocardium

[0903] are first generated in three dimensions using the previously described embodiments. Perfusion boundaries across the myocardium

[0902] are generated using a three-dimensional region-growing approach that generates sectors of the myocardium associated with each epicardial vessel. The major epicardial arteries are then segmented into each of the major epicardial vessels (i.e., left main, left anterior inferior, left circumflex, cardiac, and right coronary) based on their branching points

[0904] . The major epicardial arteries are then discretized into multiple sections using only a few epicardial vessel branching points (i.e., obtuse marginal, diagonal, etc.)

[0905] . The segmental anatomical structure

[0906] is then discretized to find a solution using mesh-based techniques

[0907] such as, but not limited to, finite element / finite volume based continuum mechanics.

[0103] FIG. 10 is a schematic diagram of a blood vessel, which, in the illustrated embodiment, provides a visual representation or illustration of the flow of contrast agent used to assess the microvasculature from invasive angiography images. The diagram shows the flow of contrast agent from the epicardial arteries through the microvessels / microcirculation system and into the myocardium. The administered contrast agent is first injected through the catheter and begins flowing from the nearest region of the left or right epicardial vessel

[1001] . The remainder of the vessel

[1007] and the microvessels

[1008] are devoid of contrast agent and are generally invisible except for hypercalcium deposits. Over time

[1002] , the contrast agent moves with blood velocity and fills the epicardial vessels

[1009] , while the microvessels remain invisible

[1010] . As the contrast agent fills the epicardial vessels

[1003] and begins to perfuse into the microvessels and myocardium, they may become visible

[1011] . When contrast injection is stopped, blood velocity drives the contrast agent distally, and contrast dissipation begins in the most proximal region

[1004] , while progressing gradually from the epicardium

[1012] to the microvasculature. Abnormal microvascular function results in an increase in contrast agent intensity

[1013] . As this process continues over time, epicardial vessels become depleted of contrast agent [1005 and 1014], and healthy microvasculature exhibits reduced contrast agent intensity compared to vessels with abnormal microcirculatory function

[1016] . In vessels with abnormal microcirculatory function, contrast agent may persist within the microvasculature and can be visualized as myocardial blush

[1015] . Over time, the intensity of the visible contrast agent continues to decrease [1006, 1017, and 1018]. The location, intensity, residence time of the blush, and dissipation rate of the contrast agent are tracked to identify and grade abnormal perfusion while capturing the three-dimensional nature of the vessel, such as the epicardial volume extracted using the above-described embodiments.

[0104] FIG. 11 is a flowchart of a process for quantifying microvascular function in invasive coronary angiography without an additional invasive wire, illustrating one example of its application in reinforcing boundary conditions for general measurement evaluation. Referring to FIG. 11A, the flowchart illustrates a process using the ANeRF-based process from the previous embodiment to determine functional information about the microvessels that deliver blood to the myocardium. Starting with ANeRF

[1101] , the stitched regions of the previous embodiment are classified to visually indicate which regions belong to different frames. In this step, vascular contrast weighting allows for accounting for the various time points at which the stitched images were acquired. Preferably, a numerical or manual method is used to determine the frame immediately before visible contrast is injected into the vessel

[1103] . In another embodiment, this frame can be selected by gating the frame count to the administration of contrast. Even though the vascular structures are not illuminated by contrast, the morphology and density of background structures are mapped to allow for fine contrast details to be distinguished at later time points

[1104] . In another embodiment, this background mapping can be performed using the last frame in the acquisition when C-arm gantry or detector movement results in a different field of view; preferably, this aforementioned embodiment can be leveraged to stitch together such background details with other frames across new image regions. Lumen filling is identified using knowledge of the three-dimensional epicardial anatomy

[1105] , and these regions are classified to identify regions of interest for microvascular function

[1106] . Classified regions are tracked across multiple frames to identify changes in pixel density representing contrast agent pooling / filling

[1107] or dissipation

[1108] over time to determine the contrast agent residence time and its associated intensity in the classified region, made possible by prior knowledge of the background structure or density characteristics.These metrics are used to calculate a virtual microvascular function score (vMF)

[1110] that can use temporal contrast agent characteristics for subsequent steps, including determining left or right coronary artery dominance

[1111] , mapping the microvascular function score to epicardial segments

[1112] , and weighting boundary conditions for a detailed simulation of biomechanical factors

[1113] .

[0105] Referring to FIG. 11B, the aforementioned temporal contrast agent and virtual microvascular function information can preferably be applied as weighted boundary conditions in detailed simulations

[1114] . A method for determining standardized, easily comparable epicardial metrics between patients is shown. Such a process overcomes the significant challenge of addressing the impact of large variations in simple properties such as blood pressure and blood velocity, as well as how these subsequently affect biomechanical-based metrics, which prevent a set "cutoff" value(s) from providing the necessary prognostic value. The illustrated embodiment presents information about the morphological properties of scalar metrics, such as the intraluminal helical flow

[1115] in the current example, across an extended range of boundary conditions and with respect to vascular anatomy, within a consistent domain ranging from 0 to 1 for absolute values ​​or -1 to 1 for other properties. Those skilled in the art will recognize that the current approach can also be used to evaluate other vascular metrics. Here, cross sections from intravascular imaging

[1116] show the location of the lumen and its associated three-dimensional vessel

[1119] , along with cross sections of the counter-rotating helical flow isosurface. One large cross section

[1117] and the other small cross section

[1118] represent the large and small areas associated with a specified magnitude of the helical flow metric. In a separate domain, the two counter-rotating regions exhibit similar cross-sectional areas

[1120] . Capturing a bounded ratio between the two rotation areas, or preferably the ratio of the total cross-sectional area of ​​the isosurface to the lumen area, yields bounded domains between -1 and 1 and 0 and 1, respectively

[1121] , regardless of helical flow magnitude, blood velocity, or other anatomical or physiological factors. Such approaches are termed the intraluminal flow-to-area ratio and the intraluminal flow imbalance ratio. Here, different values ​​of the isosurface magnitude for helical flow are represented, and the gradient variation [1122 and 1123] is also extracted as a continuous variable across the length of the vasculature.Extended boundary conditions

[1124] , such as those to reproduce functional cardiac output / cardiac load, can be added to multidimensional outputs

[1125] that are still constrained within the same domain, allowing for rapid assessment of the virtual functional anatomy of the vasculature and its subsequent gradients

[1126] .

[0106] Figure 12 shows a flowchart for coregistrating or augmenting multiple imaging modalities into a single spatiotemporal model for both analysis and visualization purposes. The flowchart preferably takes invasive angiography

[1201] , endovascular

[1202] , or noninvasive imaging modalities as input. Each individual modality undergoes its own preprocessing and segmentation process, separate from the others, before making a manual decision to augment the data with another set of imaging data. In the case of invasive angiography, if augmentation is not performed, standard ANeRF-based processing is performed

[1204] before proceeding to the subsequent analysis process

[1213] as outlined in the embodiment. If augmentation is selected and noninvasive imaging, such as computed tomography, is available, both processed imaging sets are passed to generate two vascular feature sets

[1209] , including branched anatomical structures, to coregistrate the two generated vascular models

[1210] by minimizing the distance between the two feature sets. Often, computed tomography is performed before invasive angiography as a decision-making step, and ANeRF is modified to leverage the co-registered feature set to constrain the 3D density field to the patient's known vascular anatomy

[1211] , improving the speed and accuracy of density map generation. An augmented visualization is then generated

[1212] , and the data is advanced to subsequent steps, whereby additional information, specifically plaque-related artifacts and structures from noninvasive imaging, are added to both the "Level 1" and "Level 2" analyses. In another embodiment, the invasive angiography is chosen to be augmented by intravascular imaging by extracting the centerline of the vasculature using a volumetric thinning operation

[1205] and aligning the segmented intravascular stack along the vasculature using the identified bifurcation regions to first minimize a distance function

[1214] .After distance minimization, an angular rotation function is minimized

[1215] to account for twisting along the intravascular catheter during acquisition, followed by adaptive axial space adjustment

[1216] to allow for mismatched axial spacing between intravascular frames between vessel segments separated by visible epicardial branches. A final angiographic branch morphing step deforms

[1217] the branch centerline and density field to match the registered intravascular data and refine the bifurcation region morphology before generating the visualization

[1218] . A multi-stage registration procedure was developed to rapidly improve processing speed across a single step encompassing the entire process. The same procedure is performed to augment noninvasive imaging with intravascular data

[1207] . Both intravascular imaging

[1206] and noninvasive imaging

[1208] perform a similar axial stacking procedure to generate a 3D vessel model if augmentation is not selected.

[0107] Figure 13 illustrates the selection of data or features for "Level 1" and "Level 2" analysis. The use of various metrics in "Level 1" and "Level 2" analysis is calculated using a balance of acquisition or computational complexity (including, but not limited to, the time or difficulty required to acquire the data, including human capital, computation time, computer hardware requirements, and / or network bandwidth), metric quality (including, but not limited to, assessing redundancy or entropy of input data, noise, or missing values), and metric importance (including, but not limited to, node purity or GINI feature importance). "Level 1" analysis

[1301] preferably targets near-real-time analysis (low computational complexity), while "Level 2" analysis

[1302] preferably targets metrics with higher computational complexity, but may also include detailed analysis of metrics included in the "Level 1" domain. The metrics provide varying importance and quality for different target predictions

[1304] , resulting in an adaptive cutoff region

[1303] between "Level 1" and "Level 2" analysis depending on the target prediction or outcome and acquisition / computational complexity, including taking into account different hardware availability.

[0108] FIG. 14 is a schematic diagram of recording multiple events or data entries over time for a single patient within the scope of the proposed embodiment. At the first visit

[1401] , patient data is processed using the previously described embodiment, and a "Level 1" (L1) analysis is performed

[1402] . Upon preferably automatic, but also user / manual, selection, a decision is made to proceed to a "Level 2" (L2) analysis

[1403] , and a detailed L2 analysis is performed

[1404] . Both the L1 and L2 analyses are transferred over the network to a storage pool

[1405] located either locally or in a cloud environment. At the second visit

[1406] , for analysis at a different time point or with a different procedure and / or imaging analysis, the acquired data is passed on for subsequent L1 analysis. Note that, regardless of the data from the prior L1 analysis, the imaging modality that led to the prior analysis is incorporated into the subsequent analysis

[1407] to improve individualization and predictive capabilities. Also, note that the continuous learning of the various levels of analysis provides an updated L1 analysis process with retrained models and / or weights by incorporating all levels of data that were available for use from the electronic database

[1405] or integrated network prior to the current time point

[1410] , allowing for recalibration of predictions and / or analysis using current patient characteristics. By the same feedforward process, both current and past L1 analysis data are passed to a subsequently selected L2 analysis

[1408] . The process can be repeated at subsequent time points, at which the patient is analyzed using one or more supported imaging systems or acquired data

[1409] , benefiting from continuous system learning based on prior visits.

[0109] FIG. 15 is a flow diagram of a machine learning decision-making approach incorporating simulated biomechanics-based metrics, machine learning-based analysis, and patient data to capture the nonlinear interactions and characteristics of a patient's complex condition(s). Multiple metrics, including continuum mechanics-inspired metrics or metrics illustrated in the embodiments described herein, including temporal variability across the cardiac cycle, may preferably be incorporated as inputs to the machine learning decision-making process. In one embodiment, these metrics may preferably include information about the vascular structure 1501. In another embodiment, these metrics may preferably include continuum mechanics-inspired metrics at or within the vessel wall, across vascular structure layers and plaque components 1502. In another embodiment, these metrics may preferably include hemodynamic characteristics across the vascular volume (i.e., non-wall-based quantities) 1503. In yet another embodiment, these metrics may include one or more of the features illustrated by the embodiments described herein and their derivative features. The metrics may then be passed to a multi-level, multivariate feature binning process

[1504] , preferably with equal feature discretization across multiple bins. The number of bins and the features across the bins may preferably be varied and subject to automatic adjustment to optimize or maximize inter- and intra-bin variation. Detailed simulation metrics may often produce highly skewed or unbalanced distributions, and small features (small in either time, space, or feature magnitude) often contain highly relevant pieces of information that may be missed or overlooked in many scenarios. Therefore, in another embodiment, the inputs may be discretized into bins using a probability distribution or other statistical distribution(s) of the metrics that accounts for the divergence of each metric

[1505] . This statistical distribution may include a multi-level distribution of specific spatiotemporal regions to highlight important and / or nonlinear feature extraction

[1506] . Multi-level, multivariate binning can be optimized using patient features, which can preferably be captured as measurements such as various blood test results (e.g., troponin levels, lipoproteins, and other multiple metrics).Such features may also include gender, age, weight, body mass index (BMI), clinical conditions (including, but not limited to, STEMI, NSTEMI, MINOCA), as well as stable or unstable patient status, medication use, and multiple other features. In one embodiment, these features may be used as weights or "levers" to shift bins to optimally capture data from one or more metrics. Such shifting may include metrics that are not binned at one or more levels.

[1508] In another embodiment, these bin layers may be fixed based on set requirements from "Level 1" or "Level 2" analysis, as shown in the previous embodiment. In another embodiment, by using these bins as input or hidden layers in a fully connected network, the bin distribution is available for some or all other bins in the feature set. In another embodiment, the measurement input may skip bin layers that are considered to be adapted using the "lever" actions of various patient features, and continuous spatiotemporal data may be fed into various layers of the fully connected network.

[1507] Various layers of a multi-level, multivariate binning process may impose multivariate weights either on the entire bin, on the data captured within the bin, or on each layer or connection of the fully connected network. Weights may also be applied to each metric before it is input into the binning or fully connected network. In another embodiment, the fully connected network may automatically prune connections to optimize feature propagation through the network. Pruning preferably generates parallel paths for feature propagation

[1509] , allowing for simultaneous acquisition of multiple parallel endpoints

[1510] , including, but not limited to, likelihood of outcome, location or statistical probability of specific features being presented, and probability or predicted success rate of one or more therapeutic interventions or treatment pathways. In one embodiment, these paths may include all available weighted metrics. In another embodiment, these paths may include one or more metrics, preferably with no agreement between the parallel outputs being evaluated

[1511] .

[0110] FIG. 16 is an exemplary overview of the visualization and user interface for intravascular imaging. Referring to FIG. 16A, the user interface includes panels for visualizing and querying data storage systems for patient data

[1601] , including accessing one or various imaging modality types from one or more time points or sites / clinics. The main, fully interactive visualization interface

[1602] adaptively changes to suit the selected imaging mode; the illustrated embodiment is an example of intravascular optical coherence tomography imaging. The user can adjust automatically segmented or classified image structures by using a mouse pointer or a touchscreen interface. An interactive panel

[1603] provides the user with additional functionality for cropping imported image stacks or selecting one or more machine learning approaches to apply to the image stacks, some of which are defined in the previous embodiments. The longitudinal vessel map presented along the bottom panel includes an interactive slider

[1605] to drag the image stack and presents information from each image throughout the pullback as a semi-3D visualization, including the cross-sectional area of ​​the vessel and, in one embodiment, the location and volume of lipidic or calcified plaque

[1606] . In another embodiment, these features can be adapted by the user to define other plaque-related measurements along the longitudinal map, such as fibrous cap thickness. In another embodiment, the longitudinal map can also include vessel branch location

[1607] and imaging catheter misalignment over the length of the vessel

[1608] , or one or more other features that can be graphed along the length of the vessel / image stack, including various shape or color features, to identify specific metrics or details in either two or three dimensions. Electronic storage media or a third party The analysis data can be exported to other applications. The user can also choose to render the vascular structure in 3D

[1609] .

[0111] Referring to FIG. 16B, in an embodiment of the present invention, a user interface may present an axial stack of intravascular pullbacks visualized with optical coherence tomography in three dimensions. The three-dimensional interactive visualization may preferably take input from a mouse cursor or a touchscreen interface to view the vessel from any orientation, including from within the vessel. In one embodiment, the three-dimensional visualization may show the lumen (blood component), while in another embodiment, lipid and calcium components may also be shown.

[1611] In another embodiment, other structural features, such as layers of the vessel wall, may be visualized and queried by the user. In another embodiment, the simulation data may be visualized as particle tracer streamlines or pathlines representing blood flow through the vessel, or as glyphs or manifolds for higher-order tensor values ​​throughout the blood, arterial wall, and plaque domains. An interactive three-dimensional slider 1610 preferably visualizes frame- or slice-based metrics in an adaptable pop-up visualization that can be moved by the user's mouse pointer or touchscreen interaction and modified by the user to suit the user's preferences. In one embodiment, this pop-up visualization may preferably show two-dimensional image frames 1613, with or without machine learning segmentation overlaid on the image, or data on various vessel characteristics 1614, and relative metrics for the currently selected or queried frame, such as the fibrous cap thickness (FCT) overlying lipid plaques or the virtual atheroma volume fraction (vPAV) calculated from the previous embodiment. In another embodiment, this pop-up visualization may preferably show a risk profile calculated from a "Level 1" analysis, such as a risk assessment for various changes typically found in coronary vessels and plaques, and their associated statistical standard deviations or variances or confidence intervals 1615. In another embodiment, the visualization may display demographic comparisons, such as ranking patient-specific metrics or analytical or predictive results against a database of similar or different patient characteristics.In another embodiment, the predictive model may be presented as a suggested treatment pathway. Such visualization may preferably be customizable by the user, who may select from the individual metrics used at various levels of analysis from the embodiment, or from an overall risk score combining these metrics.

[0112] Figure 17 is a further example of a simplified user interface, including predictive and demographic comparisons. Referring to Figure 17A, data regarding various plaque characteristics may preferably be displayed in a tabular format

[1701] and may include changing appearance or color or other visible visual markers that change as the user interacts with different three-dimensional plaque characteristics in the central visualization

[1702] . Predictive or analytical results may preferably highlight areas of the vasculature at risk for one or more outcomes using color, transparency, or other visible techniques

[1703] . The user may interact with the highlighted region(s) using a mouse cursor or touchscreen interactivity, including a visual two-dimensional view of the vessel wall that is "unwrapped" from the vessel wall, and data regarding specific metrics or markers, or predictive models, may be presented, preferably as color contours

[1704] . In another embodiment, the two-dimensional contours may preferably present plaque-specific or intravascular data, or characteristics of plaque structure, such as fibrous cap thickness

[1705] . Referring to FIG. 17B, an interactive user interface is presented that augments intravascular imaging with invasive coronary angiography in real time. Conventional two-dimensional angiography frame(s)

[1706] are visualized using C-arm orientation data, and a subsequent three-dimensional visualization of the vasculature is also shown

[1707] . In one embodiment, this three-dimensional visualization is fully interactive for the user to rotate, pan, and zoom through the three-dimensional view, and associated C-arm-specific view angles are also displayed. In another embodiment, both the two-dimensional and three-dimensional views may be transient in nature, allowing for vascular function to be shown over time. In yet another embodiment, the three-dimensional views may be color-coded to visualize regions of interest, such as the location of the intravascular imaging region, as shown. Also, the axially stacked three-dimensional pullback data from the previous embodiment may be visualized in three dimensions at various levels of detail. 1712 In another embodiment, this visualization may include the two-dimensional "unwrapped" visualization from the previous embodiment.In yet another embodiment, the user may interact with any of the three main views and move shape- or color-based markers [1708, 1709, and 1711] that register positions between various imaging types and visualize this in real time. In another embodiment, this coregistration may be automated, displaying additional catheters being inserted through the vessel(s) in real time. In another embodiment, several imaging modalities or acquired data may be manually selected to add or augment the visualization from available patient data

[1710] .

[0113] FIG. 18 is an example of an enhanced user interface that includes data from multiple modalities, analyses, and / or prediction results in a single interface. Referring to FIG. 18A, multiple angiograms are presented for interactive selection of one or more vessels or vessel sections to query available metrics

[1801] . The three-dimensional representation of the selected vessel and various analysis or simulation metrics are available for user interaction

[1802] , including rotation, zoom, and panning, and the selected vessel or segment is highlighted relative to the metrics on each image

[1803] . Various metrics from one or both of the "Level 1" and "Level 2" analyses may be available for display

[1804] . In one embodiment, the metrics may be interactive, allowing cursor or touch interaction to display or select important data points for visualization in an interactive three-dimensional view of the vessel. In another embodiment, these metrics may be queried in more detail, such as across the length of the vessel or segment of interest

[1806] . The displayed hemodynamic instability rate may automatically highlight regions of interest or prognostic significance, preferably highlighting these in an interactive three-dimensional view.

[1805] Those skilled in the art will recognize the benefits from being able to automatically query not only any individual metric, but also a combination of metrics that result in a predictive analysis, such as embodied biomechanical stress profiling from the analysis of the "Level 1" and "Level 2" metrics.

[0114] Referring to FIG. 18B, yet another interactive visualization is presented to expand on data from multiple imaging modalities and both "Level 1" and "Level 2" analyses. Multiple invasive angiograms are presented, including one each from the left and right coronary artery branches

[1807] . In another embodiment, several angiograms may be presented in this window. It should be appreciated that other imaging modalities, such as noninvasive computed tomography, may also be visualized in this section in various forms, such as axial, sagittal, and coronal views, rather than C-arm-specific coordinates. Both the left and right coronary artery branches may preferably be visualized from only two angiograms (one each for the left and right), and are fully interactive with various user inputs, such as via mouse pointer or touchscreen interaction, allowing for zooming, rotation, panning, and other three-dimensional interactive processes. The vasculature may be color-coded to present additional information to the user, such as predicted outcomes or areas of intravascular imaging pullback. 1812 In another embodiment, the three-dimensional visualization may include shape-based markers or other visually relevant methods to highlight specific areas or data points intended to be important to the user. 1811 Such markers may be interactive, displaying additional information, such as predicted graphs or data points. 1810 Such additional data may be displayed dynamically and automatically, preferably by using the most important pieces of information designated or extracted from the decision-making process of the preceding embodiment, rather than static data points, to present the most important data to the user first. In one embodiment, this may include important outcomes or biomechanical stress profiling indicators. 1808 Alternatively, in another embodiment, it may include automatically opening additional drop-down menus or similar methods of displaying otherwise hidden outcomes. 1810Other data points or metrics may also be presented and accessed through various menus 1809, including the ability to specify a specific data layout tailored to the user's needs, which may be saved as a user preference. Input data may be gated to cardiac electrophysiology 1815, allowing the user to identify changes throughout the cardiac cycle and enable a temporal display of the vasculature, identifying phases throughout the cardiac cycle, rather than just a static model from a specific point in time, which may be more interactive. Such capabilities allow for real-time visualization of additional wires, such as those for intravascular imaging or stent insertion, as they are inserted into the vasculature. Such visualization may preferably allow for user interaction, including rotating the view to improve three-dimensional visualization of inserted devices and subsequent coregistration and visualization with two-dimensional angiographic images, as presented in FIG. 18A.

[0115] Figure 19 shows an example suggesting the ability of embodiments described herein to identify plaque behavior over time. Based on the results, good correlation is demonstrated between changes estimated / calculated from embodiments described herein and changes measured in patients for an important, but non-exhaustive, list of plaque features. The image is visualized as a locally weighted logistic regression fit with 95% confidence intervals and the strength of Pearson's correlation coefficient r for fibrous cap thickness

[1901] , lipid arc

[1902] , lumen area

[1903] , and hypothetical atheroma volume fraction

[1904] determined by the illustrated embodiment.

[0116] Those skilled in the art will recognize that the aforementioned methods may only identify physiologically relevant plaques, but at a relatively weak level. While these methods may not be able to predict the degree of vasculature change, they can be achieved in the illustrated and exemplary embodiments. Additionally, while these methods may not be able to distinguish between different types of changes (e.g., the change between a thin fibrous cap of a plaque and a stenotic lumen), they can be achieved in the illustrated and exemplary embodiments.

[0117] There are several differences between the methods of the present invention and systems disclosed in the prior art: In one embodiment, the systems and methods of the present invention use invasive coronary angiography (also known as contrast angiography or biplane angiography) to generate predictive models that include treatment or testing pathway recommendations.

[0118] Another embodiment of the systems and methods of the present invention uses intravascular imaging (presented here using optical coherence tomography) to develop predictive models that recommend treatment or testing pathways.

[0119] In another embodiment of the systems and methods of the present invention, non-invasive computed tomography is used to develop predictive models that recommend treatment or testing pathways.

[0120] However, the reader should recognize that the steps of the systems and methods of the present invention may utilize further derived features of the mentioned imaging modalities without departing from the scope of the present invention, and that the systems and methods of the present invention extend these imaging modalities into a single system rather than a fragmented analysis.

[0121] The method of the present invention can also produce two levels of analysis: a real-time "Level 1" analysis and, if identified as required by the "Level 1" analysis, a more computationally intensive "Level 2" analysis to generate a predictive model, which differs from all prior approaches that take set inputs to produce a single static output.

[0122] The methods of the present invention incorporate patient input from multiple time points or previous examinations or different imaging modalities to improve analysis and further individualize predictive models that other systems and methods are unable to achieve due to their static nature.

[0123] The method of the present invention uses adaptive spatiotemporal machine learning segmentation model(s) and customizable physics-informed neural networks in a single process to automatically identify vascular components and reconstruct regions with significant attenuation artifacts that were not previously possible.

[0124] The method of the present invention can generate a three-dimensional density map of the vasculature from just a single invasive angiographic frame via angiographic neural radiance fields (ANeRF), which differs significantly from both prior approaches to segmenting angiographic images and conventional neural fields, increasing the information available to clinicians while reducing radiation exposure to the patient.

[0125] The methods of the present invention may further utilize ANeRF to generate ventricular estimates, including virtual ejection fraction (vEF), from angiography, either with or without ventriculography, thereby further reducing radiation exposure and treatment time for the patient.

[0126] The methods of the present invention may further assess physiological function from angiographic images, such as by using ANeRF to generate temporal and differentiable vascular density maps over one or more cardiac cycles to define vessel-specific virtual microvascular function (vMF) scores or virtual vascular strain (vVS), which previously required the insertion of additional invasive wires into the patient's body.

[0127] The methods of the present invention use existing measured and newly identified metrics to assess multidirectional stresses, and utilize these metrics in conjunction with patient factors to generate a multidimensional risk score or index to identify multiple probabilistic outcomes.

[0128] The methods of the present invention calculate this risk score using a set of measurements determined by combining an adaptively pruned or weighted set of metric calculations adjusted for patient factors and vasculature characteristics.

[0129] The method of the present invention preferably provides an augmented visual display that can combine and display visual information from predicted results and images from one or all available imaging systems to increase the information available in the clinic.

[0130] The following provides further clarification of terminology relating to continuum mechanics and vasculature metrics calculated and used throughout this specification and claims. Additionally, patient-specific data such as age, sex, and clinical symptoms, among others, are also considered in the predictive models.

[0131] Metrics marked with "**" indicate metrics devised by the inventors of novel embodiments of the present invention and, to the best of the inventors' knowledge, have not previously been used or calculated by such methods. While such metrics may have been calculated or obtained by pre-existing invasive or alternative methods, the current embodiments present novel, non-invasive approach(es) to determine or quantify these metrics. Other metrics are generally believed to be well known in the fields of engineering and / or cardiology.

[0132] Vascular morphological characteristics: These metrics are automatically extracted from various imaging modalities, allowing for the extraction of highlighted metrics (**) from intravascular optical coherence tomography without the need for additional imaging systems. If angiography or computed tomography is available, it may be possible to: Vessel volume, curvature, tortuosity, stenosis rate, lumen area, lesion diffusivity, lesion length, branching angle, and orifice location. If intravascular imaging is available, the aforementioned features can be supplemented with the following: Plaque and vessel morphology, including but not limited to: fiber, lipid, lipid rich, lipid arc, lipid volume, calcification, calcium volume, complex plaque, fibrous cap thickness, fibrous cap morphology, eccentricity, macrophage index, microvessels, cholesterol crystals, thrombus, intimal thickening, branching morphology, lumen area, and nonlinear material properties of different components. Internal and external elastic lamina volume (**): Cross-sectional area of ​​both the internal and external elastic lamina that is not visible with conventional intravascular imaging due to optical attenuation. Virtual Atheroma Volume Fraction (vPAV) (**): Expands on the above metric to provide the ratio of the external elastic membrane to the lumen area as a percentage and is used to identify plaque burden. Lipid Volume (**): The cross-sectional area or volume of lipids previously unavailable in intravascular optical coherence tomography due to optical attenuation, but made possible by the embodiments shown. When intravascular imaging is not available, Level 1 and Level 2 analyses use probabilistic methods to provide estimates of the aforementioned features.

[0133] Feature-based metrics: These metrics are automatically extracted from invasive angiogram images (**), eliminating the need to insert additional wires into the patient's body. Virtual Microvascular Function (vMF) (**): A measure of the resistance to blood flow from microvessels arising from epicardial coronary vessels assessed from angiographic images and subsequent differentiable density maps from ANeRF. Contrast flow velocity and perfusion time: Blood flow velocity was calculated using contrast motion and 3D density map features (including contrast dissipation rate). Thrombolysis in Myocardial Infarction (TIMI) flow grade: a risk score for future adverse cardiac events for people with unstable angina or NSTEMI symptoms. o TIMI Myocardial Perfusion (TMP): A flow grading system for myocardial perfusion. Blush residence time and intensity: A quantitative measurement of the duration and intensity of "blush" present on a coronary angiogram, related to the velocity and dissipation of contrast material, assessed using the temporal characteristics of 3D density maps. Contrast Pooling Time: A quantitative measure of contrast pooling time and severity of epicardial vessels, showing areas of slow or turbulent flow or highlighting areas of significant narrowing. Virtual Vascular Strain (vVS)(**): A metric that measures the 3D displacement of epicardial vessels over one or multiple cardiac cycles by utilizing differentiable, temporal vessel density maps. · Virtual Ejection Fraction (vEF) (**): A quantitative measure of the output from the left ventricle calculated using temporal 3D features of the vascular density map and optionally augmented with ventriculography. Virtual Pulse Wave Velocity (vPWV) (**): A measure that virtually calculates the stiffness of blood vessels using the temporal features of the vessel density map and knowledge of blood properties and its physical characteristics. Virtual Arterial Distensibility (**): A measure that extends vPWV and determines how blood vessels change over the cardiac cycle (i.e., how the heart contracts and dilates). Virtual Augmented Pressure (**): Extends metrics based on vPWV, arterial distensibility, and physical properties to measure pressure wave reflection across the vessel.

[0134] Continuum mechanics based metrics: Hemodynamics (fluid dynamics-based metrics): Wall shear stress; the frictional force between the vessel wall and the blood flow, shown as a vector, often presented together with its magnitude, calculated using the gradient of the velocity field and the fluid strain rate; and further derived features include the wall shear stress gradient, time-averaged wall shear stress, oscillatory shear index, relative residence time, transverse wall shear stress, cross-flow index, axial wall shear stress, secondary wall shear stress, wall shear stress divergence, wall shear stress exposure time, critical point location and residence time, wall shear stress fluctuations, and their subsequent normalized or temporal fluctuations over one or more cardiac cycles. Helical Flow: Helical flow is the "spiral" behavior of blood flow through an artery. There are four different metrics commonly used to quantify it: H1, H2, H3, and H4. Pressure: a measure used to assess the significance of stenosis (i.e., narrowing). Currently, it is measured using fractional flow reserve [FFR], which requires inserting a pressure wire into the artery, or quantitative flow ratio [QFR], which is calculated non-invasively using only angiographic images; both are commercially available processes. Blood velocity profile (magnitude and direction). Intraluminal flow to lumen area ratio (**): The ratio of the effective cross-sectional area of ​​the absolute intraluminal flow feature (assessed using an isosurface of either the velocity, helical geometry-based volume, or Lagrangian coherent structure intraluminal flow metric) to the cross-sectional area of ​​the arterial lumen (fluid component). The resulting metric is a geometric representation of the flow metric that is constrained to be 0-1 at all locations, making comparisons between patients more meaningful. Intraluminal flow instability ratio (**): An extension of the previous metric, this is the ratio of the effective cross-sectional area of ​​the positive intraluminal flow feature to the effective cross-sectional area of ​​the negative intraluminal flow feature, providing a geometric interpretation of flow imbalance that is constrained to be between -1 and 1 at all locations, making comparisons between patients more meaningful. Turbulent kinetic energy and its dissipation rate: Describes the average kinetic energy per unit mass of turbulent blood flow.

[0135] Structural mechanics: Cauchy stress tensor: A nine-parameter tensor that completely describes the state of volumetric stress in a deformable body, including as its derivatives the magnitude of the principal stresses, the gradient of the principal stresses, the magnitude of the axial principal stresses and their normalized misalignment (from the axial vector), the magnitude of the secondary principal stresses and their normalized misalignment (from the secondary vector), the magnitude of the radial principal stresses and their normalized misalignment (from the radial vector), the tensor divergence, and their subsequent normalized variations or temporal variations over one or more cardiac cycles. Ratio of structural stress to external elastic membrane area (**): The ratio of the effective cross-sectional area of ​​the absolute stress flow feature (assessed using an isosurface or stress tensor invariant manifold of either stress metric) to the cross-sectional area of ​​the external elastic membrane. The resulting metric is a geometric representation of structural stress that is constrained to be 0–1 at all locations, making comparisons between patients more meaningful. Intrastructural Stress Instability Ratio (**): An extension of the previous metric, this is the ratio of the effective cross-sectional area of ​​the positive intrastructural stress feature to the effective cross-sectional area of ​​the negative intrastructural stress feature, providing a geometric interpretation of stress flow and stress imbalance that is constrained to -1 to 1 at all locations, making comparisons between patients more meaningful.

[0136] Those skilled in the art will now recognize the advantages of the presented invention over the prior art. In one form, the presented invention provides methods and algorithms to assist clinician decision-making and patient treatment decisions by providing predictive models that provide individualized biomechanical stress profiling indices for a patient.

[0137] "Currently, there is a missed opportunity to analyze and predict which diseases will progress and result in hospital readmissions. The technology developed overcomes several important limitations of imaging systems and could play a role in assisting physicians in predicting which diseases will progress, allowing them to personalize treatment, keeping patients healthier and avoiding hospitalizations, thereby reducing costs to the community."

[0138] While various features of the present invention will be particularly shown and described in connection with illustrated embodiments of the invention, it should be understood that these specific configurations are merely illustrative of the invention and that the invention is not limited to the specific configurations. Accordingly, the present invention can include various modifications that are included within the spirit and scope of the invention. For purposes of this specification, the terms "comprise," "comprises," or "comprising" mean "including but not limited to."

Claims

1. 1. A computer-implemented method for generating an advanced visualization and prediction model to provide an individualized biomechanical stress profiling index for a patient, the method comprising: acquiring images, data, and characteristics related to the patient; b. constructing a vasculature model of at least a portion of the patient's artery; c) extracting or calculating physiological information from the acquired images, data, and features of the patient; d. Performing a lightweight "Level 1" artificial intelligence and / or biomechanical assessment optimized for real-time analysis from available data, including local and / or cloud computing hardware and several time points or imaging / assessment types; e. Using the "Level 1" results to suggest an optimal route or the need for "Level 2" analysis; f. Generating an enhanced visual display and / or report of said "Level 1" results to assist clinician decision making; g. Conducting "Level 2" artificial intelligence and / or biomechanical simulations tailored to complement or overcome gaps in "Level 1" analysis for comprehensive patient assessment using several metrics; h. generating a continuous multi-dimensional biomechanical stress profiling index of one or more plaques and / or arteries using the metrics, acquired images, data, and features of the patient; i. retraining and updating the "Level 1" analysis using results from a biomechanical stress profiling index; j. utilizing the biomechanical stress profiling index to highlight vulnerable plaque areas, plaque synthesis, risk of future growth and destabilization, and / or vascular changes over time; k. generating an enhanced visual display and / or report of said indicators including one or more imaging modalities to assist said clinician in making decision making to determine treatment for the patient; A computer-implemented method comprising:

2. The processing and extraction of important features from intravascular images is automated, and further Acquiring an intravascular imaging pullback / image stack and associated data at the time of acquisition, including blood pressure, heart rate, and partial differential equations based on the physics of the imaging system; b. Pre-processing the image stack to remove unwanted regions, preferably pre-filtering noise or artifacts; c) passing the pre-processed imaging stack and knowledge of the acquired data / physical properties to general-purpose computing hardware such as a suitable graphical processing unit; d. Segmenting the lumen using a spatiotemporal U-net machine learning architecture that leverages long short-term memory (LSTM) and attention mechanisms to enhance robustness and generalization capabilities for sparse and noisy real-world data; i. The machine learning model also applies dynamic vertical layering to modify model layers during processing to improve segmentation, or ii. A segmentation step in which adaptive block shrinkage is applied to further reshape the encoder or decoder architecture during processing to improve segmentation; e. Masking the pre-processed image stack with the resulting lumen segmentation map; f. feeding the masked and pre-processed image stack into a 3D U-Net based machine learning architecture to segment the middle layer of the vessel wall; g. Passing the preprocessed image stack, the lumen segmentation map, and the intermediate layer segmentation maps to a preprocessing module of a modified deep physics-informed neural network architecture; h. Processing the 3D pixel location, pixel multi-layer color data, and multiple segmentation maps with a Stage 1 multi-layer perceptron; i. extracting global features from the multi-layer perceptron to obtain further data from the pre-processing module, including smoothed 3D vessel centerlines; j) passing the local spatial information to a Stage 2 multilayer perceptron that has access to tissue continuity, nonlinear tissue properties, imaging physics-based properties, and partial differential equations governing pressure distribution; k. Imposing boundary or initial conditions on the network to aid convergence, such as by using the segmentation mask from step "d." and step "h."; l. Minimizing a network loss function to extract a segmented tissue map containing within it areas with significant imaging attenuation artifacts and information about tissue properties; m. performing the above steps within a single operation on a graphical processing unit for rapid and efficient processing; 10. The computer-implemented method of claim 1, comprising:

3. A three-dimensional density map or anatomical model of the vasculature from invasive coronary angiography is generated via angiographic neural radiance field (ANeRF), thereby minimizing radiation exposure to the patient while increasing the information available to the clinician, the method further comprising: acquiring at least one invasive angiogram of said vasculature comprising one or more images over a cardiac cycle; b. extracting C-arm orientation metadata from the acquired image data, including primary and secondary angles, detector characteristics, x-ray characteristics, and source position relative to the patient / gantry isocenter and the detector plane; c. Pre-processing the angiographic image or image stack using machine learning models or numerical methods to identify vascular structures; d. Generating a multi-grid or sub-pixel representation of the vascular structures identified in step "c." to improve rendering resolution, i. if there are multiple acquisitions from different primary or secondary angles, steps "a." through "d." are performed for each acquisition, or ii. Multiple views are aligned using the C-arm coordinate metadata extracted in step "b.", or iii. These views are aligned using an energy minimization algorithm to overcome patient motion, including that resulting from acquisition setup errors, C-arm gantry motion, respiratory / cardiac motion, and table or detector panning; e. Providing a multi-scale representation of the angiographic image(s), a binary mask, and an associated C-arm gantry orientation (after alignment by the energy minimization algorithm) as input to the angiographic neural radiance field; f. rendering the density field of the vasculature in three dimensions, i. including a 3D vessel connectivity filter to enhance vascular structures and reduce noise in generating said 3D density field; or ii. Processing the density field into voxelized or mesh-based visualization techniques; g. interactively visualizing the three-dimensional anatomical structure; 3. The computer-implemented method of claim 1, comprising:

4. and to obtain temporal information from invasive coronary angiography imaging to determine virtual vascular, microvascular, and ventricular function without the need for further testing or invasive wires. a. generating a three-dimensional density field of the vasculature according to claim 3; b. Identifying background features across the angiography frame including ribs or vertebrae; c. Applying a rigid body transformation to co-register background features across image frames to account for C-arm gantry or patient motion; i. coregistration to generate a set of augmented images that represent a larger two-dimensional space than any individual image frame, or ii. generating and applying a variable set of C-arm gantry orientations to account for motion artifacts across several image frames through the coregistration; d. Mapping forward-looking and backward-looking images from one or more angiographic frames to a static three-dimensional density field, i. using the co-registered image stack to generate a unique 3D density field for each set of frames over time, or ii. A mapping step in which the static density field encodes continuity constraints and is deformed over time to mimic the co-registered 2D image stack; e. fitting a predetermined myocardial map to the three-dimensional density field; f. Deforming the fitted myocardial map over one or more cardiac cycles to estimate ventricular function, such as ejection fraction; i. Ventriculography is available and used to optimize the predetermined myocardial map or ventricular estimate; 1. Reprocessing the density field to extract volumetric changes in density of the vascular structures over time; or ii. The Angiographic Neural Radiance Field (ANeRF) is modified with additional multi-layer perceptron and Navier-Stokes equation and continuity-based loss function(s) to encode hemodynamics into the vessel density field; 1. A step of calculating the dissipation or change in density of the blood vessel density field, g. Mapping the changes in dissipation or density to specific vessel segments or segments of the myocardium, or h. Querying non-vascular regions for density changes in two or three dimensions; i. a mapping step, wherein the identified dissipation or density changes are graded and mapped to vasculature or myocardial segments as areas of "blush" or microvascular dysfunction; 4. The computer-implemented method of claim 3, comprising:

5. provide novel intraluminal or intrastructural biomechanics-based metrics that are patient-specific, generalizable, and directly comparable across patients; and a. generating an expanded set of boundary conditions based on patient characteristics; b) performing a biomechanical simulation or machine learning implementation method to determine a continuum mechanics based tensor field in a fluid or structural domain using the augmented boundary conditions; c. Computing an isosurface of a normalized metric of interest, including traditional or novel metrics, from a number of equally spaced units within the domain -1 to 1 or 0 to 1; i. Taking one or more plane-based slices of one or all of the iso-surfaces from step "c." and determining the area contained within each plane-based iso-surface slice; or ii. Taking one or more plane-based slices of one or all of the isosurfaces from step "c." and determining the areas contained within the positive and negative isosurface-based regions; d. Determining the cross-sectional area of ​​the vessel in one or more planes used in steps "c.i." and "c.ii.", i. calculating the ratio of the isosurface / slice-based area to the lumen area, or the ratio of the area of ​​the positive isosurface slice to the area of ​​the negative isosurface slice, from one or more domain units; or ii. calculating the extended variability of the isosurface and / or lumen planar area ratios across one or more domain units over the range of the extended boundary conditions imposed from unit step "a."; e. generating a visual display or graph or report of metrics based on the expanded endoluminal biomechanics; 5. The computer-implemented method of claim 1, comprising:

6. and selecting, distributing, and using the available data to predict or identify an outcome or characteristic of the patient's vasculature. a. Obtaining various input metrics as identified in any one of claims 1 to 6; b. Determining a statistical or probabilistic spatiotemporal distribution of the continuous metric; c) performing a multilevel discretization of said statistical or probabilistic spatiotemporal distribution to emphasize or improve the weighting of important locations or outcomes that would otherwise be overlooked or downplayed; d. Binning the discretized metrics or the entire metric in a multi-level and multivariate feature binning process; e. Weighting or shifting the bins using patient characteristics to optimally capture data from one or more metrics; f. Implementing the bins as inputs or hidden layers in a fully connected network to capture non-linear features and interactions; g. automatically pruning connections to optimize propagation of said features through said network in a parallel or serial process; h. Providing the likelihood of an outcome, the location or statistical probability of a particular feature being presented, or the probability or predicted success rate of one or more therapeutic intervention(s) or therapeutic pathway(s) for multiple parallel endpoints; 6. The computer-implemented method of any one of claims 1 to 5, comprising:

7. Step "a." i. acquiring imaging information from one or more invasive catheter-based imaging systems, such as coronary optical coherence tomography; and / or ii. Acquiring imaging information and gantry orientation and gating information from one or more planes in an invasive coronary angiogram and / or ventriculography; and / or iii. The imaging information is obtained from non-invasive computed tomography imaging; and / or iv. Obtaining continuous measurements such as heart rate, blood pressure, and electrocardiogram, including associated data from wearable technology and patient characteristics; and / or v. Obtaining manual input values ​​from a trained technician or clinician; 10. The computer-implemented method of claim 1, comprising:

8. Step "b." further comprises: i. Pre-processing the 2D intravascular imaging stack on a computer medium such as a central processing unit or graphical processing unit (CPU or GPU); ii. Scaling and axially stacking the pre-processed and segmented image data in three dimensions into slices; iii. communicating the pre-processed and segmented data from the local system to a centralized cloud computer or containerized instance over a secure network; iv. Segmenting the pre-processed image stack on a CPU or GPU using machine learning such as a temporal neural network or a 3D neural network to identify vascular structures; v. Classifying and segmenting the vascular scaffold(s), if present, in the 2D frame and generating a 3D map of the scaffold on a GPU using a generative machine learning model and knowledge of the scaffold design prior to insertion; vi. Implementing a deep physics-informed neural network on a GPU using knowledge of tissue continuity, vascular structure, blood pressure, and image properties to reconstruct the medial and adventitial layers in the attenuated region; vii. Classifying and segmenting plaque components using a 3D neural network machine learning algorithm; viii. Interpolating the segmented data slices into voxels; ix. In another form, step "vii." includes feeding the segmented slice data into a neural field to generate a three-dimensional differentiable density map; x. Generating an adaptive mesh from the voxelized / density structure appropriate for the 3D user interaction and simulation process; xi. communicating the processed steps back to the local system over a secure network; 10. The computer-implemented method of claim 1, comprising:

9. Step "b." further comprises: i. pre-processing, on a CPU or GPU, one or more temporal angiogram and / or ventriculogram acquisitions or image sequences; ii. communicating the pre-processed and segmented data from the local system over a secure network to a centralized cloud computer or containerized instance; iii. Segmenting the epicardial vasculature using numerical and / or machine learning based algorithms; iv. inputting the pre-processed image sequence(s) and segmented vascular structure(s) and gantry orientation(s) metadata into an Angiographic Neural Radiance Field (ANeRF); v. generating, on a GPU, a three-dimensional density map of vascular and / or ventricular structures using the angiographic neural radiance field; vi. Creating an adaptive mesh from said 3D density map suitable for 3D user interaction and simulation processes; vii. communicating the processed steps back to the local system over a secure network; 10. The computer-implemented method of claim 1, comprising:

10. Step "b." further comprises: i. pre-processing, on a CPU or GPU, a stack or stacks of computed tomography images and / or associated metadata such as axial, coronal, and sagittal planes, and bolus administration times; ii. communicating the pre-processed and segmented data from the local system over a secure network to a centralized cloud computer or containerized instance; iii. Segmenting vascular and ventricular structures using numerical and / or machine learning based algorithms; iv. identifying and segmenting vascular and ventricular structures using numerical and / or machine learning based algorithms; v. identifying and segmenting plaque components using numerical and / or machine learning based algorithms; vi. Interpolating the segmented data stack into voxelization; vii. Creating an adaptive mesh suitable for 3D user interaction and simulation processes; viii. communicating the processed steps back to the local system over a secure network; 10. The computer-implemented method of claim 1, comprising:

11. Step "c." further comprises: i. acquiring and processing a temporal range of images rather than a single image frame; ii. analyzing the acquired or processed temporal image data using probabilistic programming and / or machine learning based algorithms; iii. Extracting relevant image features as a 5-dimensional feature set; 10. The computer-implemented method of claim 1, comprising:

12. Step "c." further comprises: i. acquiring and processing time ranges of patient data or patient features rather than static data points; ii. analyzing the acquired or processed temporal data using probabilistic programming and / or machine learning based algorithms; iii. Extracting relevant data features as a multidimensional feature set; 10. The computer-implemented method of claim 1, comprising:

13. Step "d." further comprises: i. matching the acquired or extracted data to a Feature Set or Sets; ii. generating an extended set of boundary conditions to simulate cardiac or vascular loading of a patient; iii. Analyzing the feature set and augmented boundary conditions using a physics-informed machine learning model to obtain a lightweight subset of biomechanical metrics estimated in real time; iv. analyzing said feature set(s) using computational statistical models and generative machine learning models; v. Visualizing the obtained feature set(s), computational statistical models and approaches for each data point; vi. generating a report or dataset for storage in a local or cloud-based electronic medium; 10. The computer-implemented method of claim 1, comprising:

14. Step "e." further comprises: i. analyzing the feature set(s) from step "d." of claim 1 using computational statistics, probabilistic programming, and / or generative machine learning models; ii. Presenting the feature set(s) and underlying computational model(s) to a user; iii. Incorporating manual user input from a skilled clinician / technician, including selecting or adding appropriate data and computational models appropriate for said patient; iv. predicting the general risk profile of said patient; v. Generating probabilistic scenarios of various treatment option(s) and presenting said scenario(s) in a progression from strongest to weakest option; vi. Using the general risk profile and probabilistic scenario(s), recommend or not recommend the use of detailed "Level 2" simulations; vii. generating a report or dataset for storage in a local or cloud-based electronic medium; 10. The computer-implemented method of claim 1, comprising:

15. Step "f." further comprises: i. accessing said report and / or dataset from said electronic medium in accordance with the preceding steps of claim 1; ii. Loading a user profile or incorporating manual input and formatting the visual display to suit the pre-set settings; iii. Adding the report and / or dataset(s) of steps "a." through "e." of claim 1 to the visual display; iv. automatically highlighting or presenting statistically significant or important probabilistic data points in a visually perceptible manner; v. Augmenting the display with five-dimensional (three-dimensional space, time, and other metrics) data from one or more acquired datasets; vi. interactively highlighting areas of interest throughout the vasculature to the user using color, shape markers, or other visually perceptible methods; vii. user interactions to modify or enhance the display, including opening or closing additional data displays or adding / removing data points from the five-dimensional display; 10. The computer-implemented method of claim 1, comprising:

16. Step "g." further comprises: i. Capturing a user command to proceed to a "Level 2" simulation process; ii. packaging all of the data from steps "a." through "f." of claim 1 and communicating the packaged data to a centralized cloud computer or containerized instance over a secure network; iii. generating a coarse mesh and a fine mesh of the vascular structure, including the lumen, plaque components, vessel wall, and epicardial structures; iv. Defining patient-specific boundary conditions for the mesh structure, including blood properties and blood profiles, displacement profiles, and electrophysiology profiles; v. Using the acquired and calculated patient data, performing simulations at one time point and / or one heartbeat and / or several heartbeats to determine engineering-based stress metrics for said vasculature using continuum mechanics principles such as fluid-structure interaction techniques, fluid-structure-electrophysical interaction techniques, computational fluid dynamics techniques, and solid mechanics techniques; 10. The computer-implemented method of claim 1, comprising:

17. Step "h." further comprises: i. constructing a feature set from the "Level 2" engineering-based stress criteria; ii. Applying probabilistic programming and machine learning based decision approaches to the "Level 1" and "Level 2" Feature Sets; iii. Calculating continuous and multidimensional biomechanical stress profiling indices for the coarse mesh from step "iii." of step "g" of claim 16 using step "ii."; iv. Extracting feature sets of likely patient, vessel, and plaque level(s) outcomes at various time intervals using a generative method from step "iii."; v. adding said "Level 1" and "Level 2" feature set(s) to a secure cloud-based electronic storage medium; vi. communicating the processed steps back to the local system over a secure network; 17. The computer-implemented method of claim 16, comprising:

18. Step "i." further comprises: i. Retrieving the "Level 1" and "Level 2" Feature Set(s) from the secure cloud-based electronic storage medium; ii. Using the centralized cloud computer or containerized instance, calculating variances and / or errors for the "Level 2" and "Level 1" Feature set(s); iii. If the variance / error exceeds a set threshold, capturing manual input from a skilled technician; iv. Retrieving Feature Sets for all relevant patients from the secure cloud-based electronic storage medium; v. Retraining the machine learning based approach and the "Level 1" analysis from steps "a.", "b.", "c.", and "d." of claim 1 using data retrieved from steps "i." and "iv."; vi. Retraining the model from step "b." of claim 1 using a cross-imaging modality data augmentation approach; vii. Pushing the retrained hyperparameters and / or new machine learning model to a cloud-based machine learning operations (MLOps) pipeline; viii. communicating the updated parameters to the local system via an electronic network; 10. The computer-implemented method of claim 1, comprising:

19. Step "k." further comprises: i. incorporating manual input to adapt said visualization to each user's preferences; ii. Visualizing a stack of two-dimensional image(s) from one or more imaging modalities; iii. Visualizing the three-dimensional vasculature from one or more imaging modalities; iv. identifying areas or data points of interest to the user according to shape or color or other visual markers; v. manual user interaction with the marker to display additional information, such as a graph or data of the prediction; vi. automatically selecting and displaying the most important data to said user by using the most important pieces of information designated or extracted from the decision-making process, rather than static data points; vii. Presenting the data or metrics in both a 3D visualization, such as an "unwrapped" view, or a modified 2D visualization; viii. allowing interactive input to move, rotate or zoom the 2D and 3D visualizations of the vasculature in space and time; 10. The computer-implemented method of claim 1, comprising:

Images