A system, method, and computer-accessible medium for providing automated image path alignment.

Abstract

Exemplary methods and systems for obtaining information about at least one feature of an object can be provided. For example, it is possible to obtain intravascular and extravascular datasets, automatically detect markers in each of at least two images of the extravascular dataset, automatically align features in the images, automatically generate an imaging path based on the aligned features and at least one position of the markers in at least one of the images, automatically align the intravascular dataset to the imaging path, generate aligned intravascular data, and automatically overlay a representation of the aligned intravascular data in at least one of the images.

Description

Technical Field

[0001] (Cross - Reference to Related Applications) This application claims the benefit of priority from U.S. Provisional Patent Application No. 63 / 466,551, filed May 15, 2023, the entire disclosure of which is incorporated herein by reference in its entirety.

[0002] This disclosure generally relates to techniques of electromagnetic radiation for medical imaging, and more particularly, to systems, devices, and methods for detecting and displaying an imaging path.

Background Art

[0003] Electromagnetic radiation is a useful tool for imaging biological structures. When imaging cavities within biological structures, a probe is often used to deliver electromagnetic radiation and recover internal images. It can be difficult to assess where the internal image was taken using the probe relative to the external view of the structure. Imaging using electromagnetic radiation that can traverse through an entire biological structure from an external position can help infer the relative position of the imaging probe, but still, it remains difficult to assess the actual position in the presence of motion from either the structure or the probe.

[0004] Therefore, there is a need to provide an apparatus, system, and / or method for improving the localization of an internal imaging probe, presenting information in the presence of motion, and addressing at least some of the deficiencies associated with conventional systems and methods.

Summary of the Invention

Means for Solving the Problems

[0005] Summary of Exemplary Embodiments Such problems and / or defects can be addressed and / or overcome by providing exemplary embodiments of this disclosure, which provide systems, apparatus, and methods for automatically locating and aligning internal probe locations within a set of external structural images.

[0006] Exemplary embodiments of the present disclosure can provide exemplary apparatus, systems, and methods for automatically locating (e.g., detecting) and aligning an internal probe location (e.g., distal tip of probe, e.g., intracatheter, intracardiac catheter) and corresponding images (e.g., images of luminal lumen, e.g., images of guidewire, e.g., images of arterial walls) and automatically detected structures (e.g., biomarkers, e.g., arterial plaques, e.g., side branches, e.g., intravascular devices) and the probe's motion path (e.g., during probe placement, e.g., during probe imaging pullback) within a set of external structural images (e.g., angiographic images, fluoroscopic images, X-ray images, e.g., ultrasound images, e.g., magnetic resonance images).

[0007] During cardiac catheterization, radiographic imaging may be performed to diagnose (e.g., diagnostic angiography) and to plan and guide treatment (e.g., stent placement, ballooning, plaque correction). In some exemplary cases, intravascular sensing (e.g., imaging, e.g., physiological function) can improve procedural outcomes by providing local diagnostic and prognostic information that is not readily available on radiographic angiography, such as lumen size (e.g., at high resolution), location and diameter of the external elastic lamina (EEL), and composition and location of plaque (e.g., lipid plaque or calcified plaque). Intravascular imaging devices often image in multiple axes, perform rotation (e.g., around the circumference of the vessel) and translation (e.g., pullback of the imaging core along the longitudinal direction of the vessel), and provide a two-dimensional (2D) or three-dimensional (3D) map of the intraluminal structure (e.g., if the modality provides, e.g., a depth axis).

[0008] When intravascular imaging is performed, understanding where internal information is generated in relation to the X-ray angiography can be complex due to both the movement of the intravascular imaging probe and patient movement (e.g., heart rate, respiration, body movement) during imaging. The challenge of localizing the information is further complicated by the fact that modern intravascular imaging systems can image much larger areas (e.g., 360° rotation and translation exceeding approximately 100 mm) at much higher speeds than conventional concepts (e.g., 50 mm / sec), while X-ray systems offer relatively lower imaging speeds (e.g., 15 fps). Furthermore, the cardiac catheterization laboratory is a fast-paced, life-or-death environment demanding efficacy and efficiency, offering little opportunity for physicians to leave the sterile area and provide manual input to the device for localization.

[0009] Conventional methods for aligning the imaging path of an intravascular imaging device onto extravascular images (e.g., angiography) require user input, are performed on a single extravascular image, and eliminate the complexity of motion, but this minimizes the value of a temporal extravascular imaging sequence that shows multiple views of the heart as it pulsates. Furthermore, conventional methods can only perform alignment on an angiography of the coronary artery injected with contrast agent. The apparatus, systems, and methods according to the exemplary embodiments of this disclosure can overcome these shortcomings, maximize the potential of combined intravascular and extravascular imaging approaches to the procedure, and improve patient outcomes.

[0010] In exemplary embodiments of the present disclosure, a method and intravascular imaging device can be provided which can be used to receive an intravascular image set and an extravascular image set, to automatically detect the start and end of an intravascular imaging shift within the extravascular image set, and to overlay the imaging path and / or other intravascular information of the intravascular imaging shift onto the images in the extravascular image set or onto several images. The exemplary system and method can be used to automatically align the imaging path without any user input, to overlay the intravascular imaging path onto multiple extravascular images, to overcome the movement of both the imaging probe and the patient, and to perform exemplary alignment, for example, without the presence of a contrast agent filling the coronary vessels.

[0011] According to another exemplary embodiment of the present disclosure, a method and system for obtaining information about at least one feature of an object can be provided. For example, it is possible to obtain intravascular and extravascular datasets, automatically receive information about features in the extravascular dataset, automatically generate an imaging path based on the received information about the features, automatically align the intravascular dataset to the imaging path, generate aligned intravascular data, and automatically overlay a representation of the aligned intravascular data into at least one image of the extravascular dataset.

[0012] For example, features may include radiopaque markers on an intravascular probe. A registered intravascular dataset may include at least one intravascular sensing dataset. An intravascular sensing dataset may include data relating to at least one of pressure, electric field, or fluidity. In addition, or alternatively, a registered intravascular dataset may include at least one intravascular imaging dataset, which is at least one of optical coherence tomography (OCT) data, intravascular ultrasound (IVUS) data, near-infrared spectroscopy (NIRS) data, reflectance data, and / or fluorescence lifetime imaging (FLIM) data. An extravascular dataset may include data relating to coronary angiography images.

[0013] In another exemplary embodiment of the present disclosure, the overlaid representation may indicate the location of a stenosis. The overlaid representation may further indicate the type of plaque in the stenosis. The overlaid representation may indicate measurement outputs at specific locations along the coronary artery. Measurement outputs may include plaque load, lipid load, and / or calcium load. The step of automatically detecting features may include performing a machine learning-based procedure for object segmentation (e.g., U-Net, Res-Net, etc.). The step of automatically detecting features may include performing a machine learning-based procedure for object detection. The step of automatically generating image paths may include automatically interpolating data points.

[0014] According to further exemplary embodiments of the present disclosure, it is possible to align image paths between extravascular images in an extravascular dataset. The step of automatically aligning image paths may include performing a machine learning-prepared non-rigid transformation procedure between images in an extravascular dataset. In yet another exemplary embodiment of the present disclosure, it is possible to automatically segment the coronary vascular system in an extravascular dataset. In addition, or alternatively, it is possible to segment data associated with guidewires in an extravascular dataset. The intravascular dataset may be a multimodal intravascular imaging and spectroscopy dataset.

[0015] According to further exemplary embodiments of the present disclosure, methods and systems for obtaining information about at least one feature of an object can be provided. For example, it is possible to obtain intravascular and extravascular datasets, automatically detect markers in each of at least two images of the extravascular dataset, automatically align features in the images, automatically generate an imaging path based on the aligned features and at least one position of the markers in at least one of the images, automatically align the intravascular dataset to the imaging path, generate aligned intravascular data, and automatically overlay a representation of the aligned intravascular data in at least one of the images.

[0016] In an exemplary embodiment of the present disclosure, the feature may include at least a portion of a guidewire, imaging catheter, guide catheter, and / or intravascular device. Alternatively, or in addition, the feature may include at least a portion of a coronary vessel, a coronary artery at least partially filled with contrast agent, and / or coronary vessel features in the absence of contrast agent. The step of automatically aligning the intravascular dataset to the imaging path may include a step of automatically addressing vascular shortening in the extravascular image. The automatic addressing step may use constrained interpolation. The acquisition of the intravascular and extravascular datasets may be synchronized with each other. In addition, or alternatively, the step of automatically aligning the intravascular dataset to the imaging path may include a step of automatically addressing at least one temporal offset in the acquisition of the extravascular and intravascular datasets. The temporal offset may be characterized prior to the acquisition of the intravascular data.

[0017] According to additional exemplary embodiments of this disclosure, a representation of intravascular data can be aligned and displayed on top of one of the images from an extravascular dataset and / or on any image from an extravascular dataset, thereby achieving a threshold derived from imaging. The threshold derived from imaging may include a alignment reliability threshold and / or an image quality threshold. In addition, or alternatively, a representation of intravascular data can be aligned and displayed on an automatically selected image from an extravascular dataset and / or an enhanced image from an extravascular dataset. Alignment of the intravascular dataset may include the step of automatically colocalizing automatically detected side branches in the intravascular and extravascular datasets.

[0018] In further exemplary embodiments of this disclosure, the step of automatically generating an image path includes the step of interpolating data points in a aligned space provided by at least one aligned feature. Alignment of an intravascular dataset to an imaging path may include the step of automatically colocalizing automatically segmented lumen profiles in the intravascular and extravascular datasets. In addition, or alternatively, the alignment of an intravascular dataset to an imaging path may include the step of automatically colocalizing automatically detected stenosis locations in the intravascular and extravascular datasets. Automatic feature detection may include the step of performing a machine learning-prepared procedure relating to at least one of object segmentation or object detection. Automatic alignment of at least one feature may include the step of performing a machine learning-prepared non-rigid transformation procedure.

[0019] The overlaid representation can show automatically detected and / or measured vascular characteristics from an intravascular dataset at a specific location along the coronary vessel. Markers can indicate the sensing location of the intravascular probe. The intravascular dataset may be an intravascular sensing dataset and / or an intravascular imaging dataset. The extravascular dataset may be a set of coronary angiography images. The overlaid representation can show the location of an automatically detected stenosis, the type of plaque in the stenosis, and / or lesions with an ischemic index exceeding a predetermined threshold. The ischemic index may be an index derived at least partially from the images. Markers may be radiopaque markers.

[0020] These and other purposes, features, and advantages of the exemplary embodiments of this disclosure will become apparent upon careful reading of the following detailed description of the exemplary embodiments of this disclosure, when considered in conjunction with the appended claims. [Brief explanation of the drawing]

[0021] Further objectives, features, and advantages of this disclosure will become apparent from the following detailed description, in conjunction with the accompanying diagrams illustrating illustrative embodiments of this disclosure.

[0022] [Figure 1] Figure 1 is a flowchart of a method / process for automatically aligning intravascular information onto an extravascular image set, according to an exemplary embodiment of the present disclosure.

[0023] [Figure 2] Figure 2 is a set of exemplary graphs illustrating exemplary methods / processes, systems, and apparatus for aligning intravascular information onto an extravascular image set according to exemplary embodiments of the present disclosure.

[0024] [Figure 3]FIG. 3 is a set of illustrations using extravascular image segmentation. In some embodiments, a body object (e.g., a vascular system filled with a contrast agent), an intravascular insert (e.g., a guide catheter, e.g., a guide wire), or both are detected and / or segmented to induce registration of a plurality of extravascular images relative to each other or to induce registration of an extravascular image relative to an intravascular image.

[0025] [Figure 4] FIG. 4 is a set of illustrations regarding extravascular object detection according to an exemplary embodiment of the present disclosure.

[0026] [Figure 5] FIG. 5 is a set of illustrations regarding creation of a non-rigid registration / image deformation map to overcome motion during an imaging session and create an accurate imaging path according to an exemplary embodiment of the present disclosure.

[0027] [Figure 6] FIG. 6 is a set of illustrations providing an example of intravascular-extravascular registration refinement using detected collateral positions within an extravascular image and an intravascular image according to an exemplary embodiment of the present disclosure.

[0028] [Figure 7] FIG. 7 is a set of illustrations providing an example of a measurement profile from an extravascular image overlaid on a plurality of frames from an extravascular image data set according to an exemplary embodiment of the present disclosure.

[0029] [Figure 8] FIG. 8 is a set of illustrations providing exemplary non-contrast agent object detection for image path reconstruction and registration according to an exemplary embodiment of the present disclosure.

[0030] [Figure 9]Figure 9 is a set of illustrations providing exemplary contrast-free object segmentation for image path reconstruction and alignment according to another exemplary embodiment of the present disclosure.

[0031] [Figure 10] Figure 10 is an illustrated set that provides examples of intravascular image features (e.g., automatically detected features) overlaid on extravascular images with and without contrast agents, according to exemplary embodiments of the present disclosure.

[0032] [Figure 11] Figure 11 is a system diagram of an intravascular imaging system according to an exemplary embodiment of the present disclosure.

[0033] [Figure 12] Figure 12 is an exemplary representation of co-located intravascular and extravascular imaging on an intravascular image using a user interface (UI) that utilizes the exemplary system of Figure 11, according to an exemplary embodiment of the present disclosure.

[0034] [Figure 13] Figure 13 is an exemplary representation of co-located intravascular and extravascular imaging on an intravascular image using a user interface (UI) that utilizes the exemplary system of Figure 11, according to another exemplary embodiment of the present disclosure.

[0035] [Figure 14] Figure 14 shows an exemplary computer system according to an exemplary embodiment of the present disclosure.

[0036] Throughout the drawings, unless otherwise noted, the same reference numerals and letters are used to represent similar features, elements, components, or parts of the illustrated embodiments. Furthermore, this disclosure will be described in detail herein with reference to the drawings, in relation to illustrative embodiments and not limited to the specific embodiments illustrated in the drawings and the accompanying claims. [Modes for carrying out the invention]

[0037] Detailed description of exemplary embodiments The following description of exemplary embodiments provides non-limiting representative examples with reference to figures, and in particular, describes the features and teachings of different aspects of the disclosure. The embodiments described should be recognized as implementable separately or in combination with other exemplary embodiments from the description of exemplary embodiments. A person skilled in the art who examines the description of exemplary embodiments should be able to learn and understand the different aspects of the disclosure described. The description of exemplary embodiments is not specifically exhaustive, but should facilitate an understanding of the invention to the extent that other implementations in the knowledge of a person skilled in the art who has carefully read the description of embodiments will be understood to be consistent with the application of the disclosure. In some exemplary embodiments of the disclosure, external structural images can be provided by exemplary systems (e.g., X-ray systems, angiography systems, fluoroscopy systems) according to exemplary embodiments of the disclosure. The exemplary systems can be used, for example, to image a patient or a region of a patient. In some exemplary embodiments of the disclosure, the exemplary systems can be used to perform coronary angiography. In some embodiments, the exemplary systems can image the patient's chest during cardiac catheterization. Exemplary images (e.g., X-ray images) provided by the exemplary system can be used to guide cardiac catheterization procedures.

[0038] In some embodiments, a radiopaque agent may be injected into the coronary arteries to visualize exemplary imaging structures of the patient's chest. In some embodiments, the exemplary system may perform coronary angiography (e.g., while a probe or cardiac catheter is inserted into the coronary vessels). In some exemplary embodiments of the present disclosure, the exemplary system may have a specific frame rate (e.g., 5fps, 7fps, 10fps, 15fps, 30fps, 60fps, etc.), and an image sequence (e.g., video) may be acquired at the specified frame rate. While the image sequence is being acquired, motions that may be caused by the biological sample (e.g., respiration, heartbeat, body movement) or motions caused by displacement of the exemplary system may be present. In some exemplary embodiments of the present disclosure, the image sequence may be acquired while the imaging probe is positioned within the coronary vessels and its position will move over time due to any source of motion. According to a further exemplary embodiment of the present disclosure, the image sequence may be acquired while the probe is being translated.

[0039] According to an exemplary embodiment of the present disclosure, the imaging probe may be partially radiopaque (e.g., completely radiopaque) at certain locations (e.g., the entire probe, e.g., the distal tip) to facilitate the localization of the probe's position in an externally obtained exemplary (e.g., X-ray) image. For example, a radiopaque marker may be positioned on the probe, for example, near the distal tip where imaging may be performed. In an exemplary embodiment of the present disclosure, the probe may be housed in a protective sheath (e.g., a plastic sheath) that is partially transparent (e.g., completely transparent) to certain forms of electromagnetic radiation (e.g., visible wavelength light, infrared wavelength light and all in between). The protective sheath may have locations with controlled radiopaqueness (e.g., radiopaqueness) (e.g., at specific locations along its length, e.g., at the distal tip of the sheath) to facilitate the localization of the sheath's position in an externally obtained (e.g., a set of X-ray images) X-ray image. For example, the probe and its sheath may be configured or housed within a catheter (e.g., a cardiac catheter). A catheter can be deployed, for example, to image intraluminal objects such as the inside of an arterial wall.

[0040] According to additional exemplary embodiments of the Disclosure, a catheter may be connected to a catheter interface unit (CIU), and the CIU may be connected to a console device. Such interconnection configurations may constitute exemplary intravascular imaging systems. For example, an intravascular imaging system may be used to image the interior of an arterial wall using an optical modality (e.g., optical coherence tomography, near-infrared spectroscopy (NIRS), diffuse reflectance, Raman spectroscopy, etc.). In some exemplary embodiments of the Disclosure, an intravascular imaging system may be used to image the interior of an arterial wall using an ultrasonic modality (e.g., a combined optical and ultrasonic modality, e.g., photoacoustic modality). In some exemplary embodiments of the Disclosure, an intravascular imaging system may be a multimodal intravascular imaging system, such as an OCT-NIRS imaging system, configured to provide both high-resolution intravascular imaging and measurement (e.g., using OCT features) and highly specific and accurate plaque characterization (e.g., using NIRS features).

[0041] According to an exemplary embodiment of the present disclosure, an intravascular imaging system can be provided, which may include an imaging probe (e.g., a multimodal imaging probe, an intravascular imaging probe, etc.). For example, the intravascular imaging probe may be designed to be observable in an X-ray image (e.g., using a radiopaque material). The intravascular imaging probe may have a protective sheath and an imaging core, while the imaging core may be translated through the protective sheath to create an image. The intravascular imaging probe may also include at least one radiopaque marker. For example, a radiopaque imaging marker (e.g., an imaging marker) may be positioned on the imaging core of the imaging probe so that any translation of the imaging portion can be monitored. The radiopaque marker may also be permanently located on the protective sheath portion of the imaging probe to mark a specific location, e.g., the distal tip of a catheter (e.g., to allow a clinician to guide the catheter to the correct position in the coronary vascular system). In some exemplary embodiments of this disclosure, the radiopaqueness of any component of the imaging probe can be adjusted (e.g., optimized) to provide a controlled amount of contrast in the X-ray image. The intravascular imaging probe may be an optical imaging probe (e.g., optical coherence tomography, near-infrared spectroscopy (NIRS), diffuse reflectance, Raman spectroscopy, etc.). In some further exemplary embodiments of this disclosure, the intravascular imaging probe may be an ultrasonic imaging probe with an ultrasonic modality (e.g., a combined optical and ultrasonic modality, e.g., a photoacoustic modality). In some further exemplary embodiments of this disclosure, the intravascular imaging probe may be a multimodal intravascular imaging probe, such as an OCT-NIRS imaging system, configured to provide both high-resolution intravascular imaging and measurement (e.g., using OCT features) and highly specific and accurate plaque characterization (e.g., using NIRS features). In some further exemplary embodiments of this disclosure, the intravascular imaging probe may be a multimodal intravascular imaging probe, such as an OCT-IVUS imaging system.

[0042] According to further exemplary embodiments of the present disclosure, an intravascular imaging system is connected to an X-ray system and can therefore facilitate the logging (e.g., saving, e.g., writing to disk) and processing of X-ray images (e.g., image sequences). For example, logging and / or processing of X-ray image data can be triggered by the initiation of an imaging probe pullback (e.g., providing synchronized intravascular and X-ray images). In some exemplary embodiments of the present disclosure, the exemplary intravascular imaging system can read X-ray images via a wired connection (e.g., video output signal) to an angiography system. In some exemplary embodiments of the present disclosure, the exemplary intravascular imaging system can read X-ray images wirelessly (e.g., via a wireless network, e.g., from a PACS server). The intravascular imaging system can also use triggered data to automatically align the locations of intravascular landmarks on X-ray images (e.g., on multiple X-ray images).

[0043] According to yet another exemplary embodiment of the present disclosure, an intravascular imaging system calibration configuration can be provided (for example, to improve the temporal alignment between the intravascular imaging system and the extravascular imaging system). System calibration may include the step of performing an imaging session of the calibration configuration on a particular X-ray system (for example, the step of obtaining both intravascular imaging system data and extravascular imaging system data, for example, simultaneously). For example, the system calibration configuration may include the step of recording or storing information about the X-ray system. Furthermore, or alternatively, the exemplary intravascular imaging system may be calibrated for multiple X-ray systems (for example, a given extravascular system may be manually selected or automatically detected after communication with the intravascular imaging system, for example, a stored calibration file for each extravascular imaging system may exist on the intravascular imaging system, for example, the calibration file may be generated after a calibration procedure on a given extravascular system).

[0044] According to one exemplary embodiment of the present disclosure, the location of radiopaque imaging markers of an intravascular probe can be automatically detected using a procedure (e.g., a machine learning procedure, e.g., a deep learning procedure, e.g., an object detection procedure, e.g., a YOLO procedure) that can facilitate the alignment of the imaging position of the intravascular probe into extravascular images (e.g., fluoroscopic images, e.g., angiographic images). For example, the location of radiopaque imaging markers from multiple extravascular images (e.g., all images provided by fluoroscopy or cine acquisition) can be automatically detected and used to automatically infer (e.g., interpolate) the imaging position of the intravascular probe between extravascular images (e.g., aligned extravascular images), and thus facilitate the alignment of the imaging path from the intravascular probe to the intravascular data. This exemplary feature may be advantageous, for example, when the frame rate or translation speed of the intravascular imaging probe is higher than the extravascular imaging frame rate. To infer intravascular imaging pathways traversing multiple extravascular images in the presence of patient movement (e.g., cardiac motion, patient motion, extravascular imaging device motion, etc.), automatically detected features from multiple extravascular images can utilize interframe alignment using alignment procedures (e.g., rigid body transformation procedures, autocorrelation-based procedures, non-rigid body transformation procedures, warping procedures, machine learning-based warping procedures, VoxelMorph, etc.).

[0045] In some embodiments of the present disclosure, automatically detected extravascular image features (e.g., automatically detected coronary artery architecture, automatically segmented guidewires, etc.) can be used to automatically estimate transformation maps (e.g., deformation maps) between extravascular images in order to perform a registration procedure (e.g., to register extravascular images, to align extravascular image features, etc.). In some exemplary embodiments of the present disclosure, automatically detected radiopaque markers from several extravascular images (e.g., several acquisition time points) can include transformation maps applied to them (e.g., when it is understood that radiopaque imaging markers have moved between acquisitions of extravascular images) to track the imaging path of radiopaque markers on a particular frame (e.g., in a registered space). According to one exemplary embodiment of the present disclosure, a neural network is trained to detect multiple features of interest in angiographic images acquired under various imaging conditions (radiation, flash delivery), system specifications (frame rate, resolution), and biological environments (e.g., RCA, LDA in the presence of multiple guidewires, stents, internal radiopaque markers, etc.). According to an exemplary embodiment of the present disclosure, a spatiotemporal neural network (e.g., a recurrent CNN, MeMOTR, etc.) can be trained to detect a feature of interest (e.g., a radiopaque imaging marker, a guide catheter, a guidewire, a coronary artery architecture, etc.) under various imaging conditions, system specifications, and / or biological environments.

[0046] Injection of radiopaque contrast agents into the coronary arteries is a routine procedure that allows clinicians to visualize the coronary vascular system, for example, to visualize the degree of stenotic plaque and diagnose coronary artery disease. However, radiopaque contrast agents can be toxic, and their toxicity varies depending on the patient's condition. For this reason, providing a method of alignment between intravascular and extravascular imaging, even in the presence of reduced vascular visibility (e.g., reduced contrast) on extravascular imaging, is extremely beneficial (for example, to indicate stent placement sites, for example, to indicate plaque correction sites).

[0047] In some exemplary embodiments of this disclosure, the imaging path can be automatically detected based on the alignment of a moving radiopaque imaging marker with an automatically detected internal implant (e.g., a guidewire, guide catheter, etc.) in the extravascular image. The imaging marker may travel a variable distance between extravascular images in the extravascular dataset based on patient motion or based on the angle of the 2D extravascular imaging system with respect to the vessel being imaged and the imaging path direction (e.g., a cause of vascular shortening). In some exemplary embodiments of this disclosure, such detection and alignment can be used to generate a virtual vascular shortening map to provide higher accuracy co-registration in the presence of reduced injected contrast agent (e.g., reduced vascular architecture visibility, e.g., during a reduced imaging session) or in the absence of any injected contrast agent (e.g., no vascular architecture visibility, during a contrast-free imaging session, etc.). In some exemplary embodiments of the present disclosure, the imaging path can be automatically detected (e.g., throughout a set of extravascular images) based on the alignment of a moving radiopaque imaging marker on an estimated path between two internal implants (e.g., the tip of a guide catheter and the tip of a guidewire), and (e.g., if the radiopaque imaging marker is cylindrical) measurements of the radiopaque imaging marker's angle can be utilized. In some exemplary embodiments of the present disclosure, the imaging marker's path can be automatically detected during a contrast-free imaging session and aligned to a previously acquired extravascular image (e.g., a diagnostic angiogram) in which a contrast agent is injected.

[0048] In some further exemplary embodiments of the present disclosure, extravascular image quality can be assessed prior to or after the registration routine. In some embodiments of the present disclosure, image quality can be configured to automatically toggle whether an overlay is provided on a given frame. For example, the image quality overlay can be based on automatically measured extravascular imaging device characteristics (e.g., frame rate, contrast, brightness) or automatically measured imaging motion. In some embodiments of the present disclosure, a registration quality metric (e.g., an index) can be configured to automatically toggle whether an overlay is provided on a given frame. The registration quality metric can be based on, for example, an automatically detected velocity of intravascular device pullback, an automatically detected curvature of the imaging path, an automatically detected consistency of the observed X-ray signal with respect to the imaging marker, an automatically detected range of detected shortening values, an automatically detected overlap between detected pullback paths and detected vessel and guide catheter locations, or a machine learning-based reliability metric associated with intermediate and final procedure outputs. In some further exemplary embodiments of the Disclosure, any automatically acquired characteristics in an intravascular or extravascular imaging dataset that reach a predetermined threshold (e.g., a reassessment reliability threshold, an image quality threshold, etc.) can be used to adjust the appearance of the overlay. In yet another exemplary embodiment of the Disclosure, the automatically acquired characteristics can then be used to utilize a different set of feature processing methods (e.g., machine learning models, multimodality image reassessment, feature tracking, etc.). In yet another exemplary embodiment of the Disclosure, quality metrics for feature and image processing results (e.g., geometric quality of the path, overlap between detected radiopaque marker paths and coronary artery architecture, estimated level of shortening, etc.) can be used to determine whether the overlay is displayed on the extravascular or intravascular image.In yet another exemplary embodiment of the present disclosure, a quality metric for the results may be used to determine whether an additional set of calculation methods (e.g., a second, third, etc.) can be applied to obtain improved results and / or to correct existing results.

[0049] Figure 1 shows a flowchart of a method / process 100 for automatically aligning intravascular information onto an extravascular image set, according to an exemplary embodiment of the present disclosure. The exemplary method / process shown in Figure 1 can be performed by an exemplary system and / or computer-accessible medium, according to an exemplary embodiment of the present disclosure, and can provide fully automated image path reconstruction across multiple synchronized images from an extravascular imaging dataset and an intravascular imaging dataset. For example, as shown in Figure 1, in step 102, the processor can receive synchronized extravascular images of the coronary vascular system (e.g., in the presence or absence of injected contrast agent) and intravascular images. In step 104, the processor can automatically detect extravascular features in the extravascular images and (optionally) intravascular features in the intravascular images. In step 106, the processor can align the detected extravascular features across a subset of the extravascular images. In step 108, the processor can estimate the imaging path from the aligned extravascular features across a subset of the extravascular images. Furthermore, in step 110, with respect to a subset of extravascular images, the processor can align a subset of intravascular images and / or intravascular features to the imaging pathway.

[0050] Figure 2 shows a set of exemplary graphs illustrating exemplary methods / processes, systems, and apparatus for aligning intravascular information onto an extravascular image set according to exemplary embodiments of the present disclosure. As shown in Figure 2, radiopaque markers 202 (Image 0), 208 (Image N) are detected at the start of pullback within coronary artery branches (e.g., contrast-filled coronary artery branches) in each extravascular image up to extravascular image 0 and extravascular image N, and can represent the end of intravascular probe translation or, similarly, the midpoint of intravascular probe translation. Radiopaque marker positions traversing multiple extravascular images can be used to create an imaging path 212.

[0051] To create an accurate imaging path, extravascular image registration can be used to interpolate the imaging path in the presence of motion (e.g., cardiac motion, respiratory motion, contrast agent variability) and to further register to each extravascular image. A registration transformation map (e.g., vector map, deformation map, etc.) can be automatically inferred for each extravascular image frame in relation to a reference frame, or it can be registered to multiple frames, or to any combination of sequential frames that can improve the efficiency or performance of the registration procedure. The original radiopaque marker position 214 is shown in Figure 2 in image N 214, along with another radiopaque marker position 210 at the time / image acquired in images 0 to N. The positions of the radiopaque marker 216 located at the distal tip of the cardiac catheter sheath (e.g., a marker not translated during intravascular imaging pullback) and the guidewire tip 218 are also shown in Figure 2, both of which can be used as markers for registration in the absence (e.g., partial absence) of the visible vascular system.

[0052] Figure 3 shows a set of exemplary illustrations using extravascular image segmentation according to exemplary embodiments of the present disclosure. For example, a body object (e.g., a contrast-filled vascular system), an implant (e.g., a guide catheter, guidewire, etc.), or both can be segmented to guide the alignment of multiple extravascular images relative to each other, or to guide the alignment of extravascular images relative to intravascular images. According to an exemplary embodiment of the present disclosure, object detection can be performed using certain procedures such as binarization, morphological calculations, Fourier region analysis, edge detection, or it can be performed using machine learning or deep learning, for example, using procedures such as U-net or Res-Net architectures. Turning to the exemplary items identified in Figure 3, item 300 is a guide catheter for coronary artery intervention / imaging, item 302 is a contrast-filled artery in an extravascular image, item 304 is an automatically segmented guide catheter using machine learning / deep learning (e.g., U-Net, ResNet, etc.), and item 306 is an automatically segmented coronary vascular system using machine learning / deep learning (e.g., U-Net, ResNet, etc.).

[0053] Figure 4 shows a set of exemplary illustrations relating to extravascular object detection according to exemplary embodiments of the present disclosure. According to one exemplary embodiment of the present disclosure, as illustrated in Figure 4, a body object (e.g., a contrast-filled vascular system), an implant (e.g., a guide catheter tip, e.g., a guidewire edge, e.g., a radiopaque marker), or both are detected (e.g., localized) to guide the alignment of multiple extravascular images relative to each other, or to guide the alignment of an extravascular image relative to an intravascular image. Exemplary object detection can be performed using manual procedures such as binarization, morphological calculations, Fourier region analysis, edge detection, and / or using machine learning or deep learning, for example, using procedures such as YOLO.

[0054] For example, exemplary object detection can be used for post-initial alignment inference or alignment refinement using semantically segmented objects. Turning to the exemplary items identified in Figure 4, item 400 is an extravascular image, item 402 is an extravascular image with an overlay, item 404 is an automatically detected side branch, item 406 is, for example, an automatically detected first radiopaque marker on an intravascular imaging catheter (e.g., a radiopaque housing / can across the distal optics to locate the imaging position, e.g., a radiopaque marker that is translated during intravascular imaging pullback), item 408 is an automatically detected first guidewire (e.g., in a branch not imaged intravascularly), item 410 is an automatically detected second radiopaque marker (e.g., the tip position of the intravascular imaging catheter, e.g., a radiopaque marker that is not translated during intravascular imaging pullback), and item 412 is an automatically detected second guidewire (e.g., in a branch being imaged intravascularly).

[0055] Figure 5 shows an exemplary set of illustrations relating to the generation of non-rigid alignment / image deformation maps for overcoming motion between imaging sessions and generating accurate imaging paths, according to an exemplary embodiment of the present disclosure. According to one exemplary embodiment of the present disclosure, extravascular image and / or feature alignment can be performed in a non-rigid format. Exemplary alignment can be used to account for cardiac motion and vascular movement, along with vascular shortening, to create accurate imaging paths at each cardiac / vascular location. For example, alignment can be performed by measuring deformation maps between vascular segmentation in sequential images. The deformation maps can then be applied to update the relative positions of detected radiopaque markers, and the accumulation of performing this operation for all images enables the creation of more accurate imaging path measurements on each extravascular image / vascular location.

[0056] In practice, for example, vascular shortening can be considered by measuring the distance between automatically detected radiopaque marker positions in sequential extravascular imaging (e.g., if marker translation is not perfectly linear in distance), however, this is more likely to be achieved after extravascular image registration. Registration can be intensity-based, segmentation-based, marker-based, and / or any combination; it can be rigid (rotation / translation), or it can be affine (e.g., rigid + scaling and / or "shearing" or "skewing"), or it can be non-rigid (e.g., deformable, affine with, e.g., "vector" fields), along with many other methods. Exemplary registration can be performed between different modalities (e.g., intravascular-extravascular imaging) or a single modality (for extravascular imaging sets).

[0057] For example, when performing alignment, a reference space (Image 0 in Figure 5) and a current image space (Image 1 in Figure 5) exist. To align Image 1 to Image 0, there exists an optimization routine based on the method described above. In some exemplary cases, machine learning models and / or deep learning models (e.g., convolutional neural network models, transformer models, VoxelMorph, SynthMorph, HyperMorph, etc.) can be used to measure and / or detect deformation maps that describe transformations from one image space to another (e.g., alignment, alignment, rigid alignment, non-rigid alignment, affine alignment, non-parametric alignment, etc.). In some exemplary cases, it is possible to align an image from one imaging modality to another using a conditionally deformable template based at least partially on a convolutional network (e.g., a coronary artery atlas template). In other exemplary cases, it is possible to align an image with another from any imaging modality that uses a learning procedure (e.g., supervised learning, unsupervised learning, etc.) based on a differential homeomorphism transformation (e.g., stochastic differential homeomorphism alignment). In yet another exemplary case, it is possible to align an image with another from any imaging modality that uses a domain-specific differential homeomorphism metric mapping.

[0058] Looking at the items identified in Figure 5, in the image labeled “Reference Feature” (or Image 0), item 502 is an automatically detected vascular feature, item 504 is a first automatically detected radiopaque marker feature (e.g., an imaging position marker), and item 506 is a second automatically detected radiopaque marker feature (e.g., a catheter tip position marker). In the image labeled “Current Feature” (or Image 1), item 508 displays an automatically detected vascular feature at a different point in time (e.g., after movement). In the image labeled “Regulated Current Feature” (or Image N), item 510 displays an automatically detected vascular feature 508 aligned to vascular feature 502. Alignment can be performed based on any of the learned or conventional alignment methods described herein. Importantly, the alignment / deformation map is inferred, for example, based on automatically detected vascular features or intravascular devices, but can be applied to other automatically detected features and / or, for example, to the raw image in which the features are detected. Such exemplary methods of applying an alignment map inferred from vascular features but applied to other features can also be utilized in an image labeled as “Aligned Current Feature” (or Image N), while items 512 and 514, representing automatically detected radiopaque markers not used to align the reference frame and the current frame, have alignment transformations applied to them, for example, to assess and consider vascular shortening, to perform more accurate imaging path estimation.

[0059] Figure 6 shows a set of illustrations 600, 620, providing an example of intravascular-extravascular registration refinement using detected side branch positions in extravascular and intravascular images according to an exemplary embodiment of the present disclosure. While the imaging path can be created using the trajectory of a radiopaque marker during pullback across the aligned extravascular image / features, Figure 6 illustrates that other image features may be used to increase the longitudinal accuracy of multimodal (intravascular-extravascular) registration using automatically detected features in both extravascular and intravascular images (e.g., side branch positions, as shown in Figure 6). In some exemplary embodiments of the present disclosure, any automatically detected objects and / or automatically measured attributes, independently from either of the two imaging modalities and / or through the use of heterogeneous modalities together, may be used to refine the registration (e.g., stents, stenosis, lumen profiles, FFR gradients, etc.).

[0060] Figure 7 shows a set of figures 700, 701, which provide an example of measurement profiles from extravascular images overlaid on multiple frames from an extravascular image dataset according to an exemplary embodiment of the present disclosure. As illustrated in Figure 7, it is possible to visualize measurements from intravascular images (e.g., lumen profiles, external elastic membrane "EEM" profiles). Turning to the items identified in Figure 7, item 702 is the detected image path centerline and EEM profile at time point 1, color-coded by pullback distance (e.g., distal to proximal endpoint of imaging pullback), and item 704 is the detected image path centerline and EEM profile at time point 2, color-coded by pullback distance (e.g., distal to proximal endpoint of imaging pullback).

[0061] Figure 8 shows a set of illustrations that provide exemplary contrast-free object detection and segmentation for image path reconstruction and alignment according to exemplary embodiments of the present disclosure. According to one exemplary embodiment of the present disclosure, as illustrated in Figure 8, an internal implant (e.g., a guide catheter, guide wire, etc.) can be detected to guide the alignment of multiple extravascular images relative to each other, or to guide the alignment of an extravascular image relative to an intravascular image. For example, to account for shortening or motion, the imaging path can be assumed to progress along a guide wire, and radiopaque marker positions (e.g., translated markers, non-translated markers, etc.) can be used to interpolate the imaging path between extravascular frames. In other exemplary embodiments of the present disclosure, a radiopaque marker path (e.g., created by being detected in multiple extravascular images) may be sufficient to create an image path for each image using dynamic features located in the image (e.g., the direction of radiopaque marker movement, its angle, and / or tilt of the radiopaque marker based on its eccentricity, etc.).

[0062] Exemplary object detection can be performed using manual procedures such as binarization, morphological calculations, Fourier region analysis, and edge detection, as well as using machine learning or deep learning, such as YOLO. Turning to the items identified in Figure 8, item 800 is an exemplary extravascular image (e.g., a coronary angiography image). Some patients cannot tolerate contrast injection, in which case an intravascular imaging dataset (e.g., pullback) can be aligned to an extravascular image with the coronary artery branches not filled with contrast. Item 802 in Figure 8 provides an exemplary object detection overlay embodiment for contrast-free or contrast-reduced alignment (e.g., to guide the alignment of multiple extravascular images relative to each other, or the alignment of an extravascular image relative to an intravascular image). Item 804 shows an automatically detected guide catheter tip (e.g., via a machine learning / deep learning procedure, e.g., YOLO). Item 806 is the automatically detected guidewire position (e.g., via a machine learning / deep learning procedure, e.g., YOLO). Item 808 is the automatically detected stent (e.g., via a machine learning / deep learning procedure, e.g., YOLO). Item 810 is the automatically detected intravascular imaging catheter radiopaque marker 1 (e.g., marking the distal optical system location, a radiopaque housing, acquired via a machine learning / deep learning procedure, e.g., YOLO). Item 812 is the automatically detected intravascular imaging catheter radiopaque marker 2 (e.g., marking the distal position of the catheter sheath, a radiopaque marker that is not translated during imaging pullback, acquired via a machine learning / deep learning procedure, e.g., YOLO). Item 814 is the automatically detected portion of the guidewire (e.g., marking the distal position of the vessel, acquired via a machine learning / deep learning procedure, e.g., YOLO).

[0063] Figure 9 shows a set of illustrations providing exemplary contrast-free object segmentation for image path reconstruction and alignment according to another exemplary embodiment of the present disclosure. According to one exemplary embodiment of the present disclosure, an internal implant (e.g., a guide catheter, guidewire, etc.) can be semantically segmented to guide the alignment of multiple extravascular images relative to each other, or to guide the alignment of an extravascular image relative to an intravascular image. For example, to account for shortening or motion, the imaging path can be assumed to progress along a guidewire, and radiopaque marker positions (e.g., translated markers, non-translated markers, etc.) can be used to interpolate the imaging path between extravascular frames. For example, a radiopaque marker path (e.g., created by being detected in multiple extravascular images) may be sufficient to create an image path for each image using dynamic features located within the image (e.g., the direction of radiopaque marker movement, its angle, and / or its eccentricity, such as the tilt of the radiopaque marker).

[0064] Exemplary object segmentation can be performed using manual procedures, such as those based on intensity, frequency, or morphological characteristics, or it can be performed using machine learning or deep learning, for example, using procedures such as U-net or Res-Net architectures. Turning to the items identified in Figure 9, item 900 is an exemplary extravascular image (e.g., coronary angiography image). Some patients cannot tolerate contrast injection, and in these cases, an intravascular imaging dataset (e.g., pullback) can be aligned to the extravascular image with the coronary artery branches not filled with contrast. Item 902 points to an embodiment of object detection overlay for contrast-free or contrast-reduced alignment (e.g., to guide the alignment of multiple extravascular images relative to each other, or the alignment of an extravascular image relative to an intravascular image). Item 904 is an automatically segmented guide catheter (e.g., via machine learning / deep learning procedures, e.g., U-Net, Res-Net). Item 906 is an automatically segmented guidewire (e.g., via a machine learning / deep learning procedure, e.g., U-Net, Res-Net). Item 908 is an automatically segmented radiopaque marker (e.g., via a machine learning / deep learning procedure, e.g., U-Net, Res-Net). Item 910 is the automatically measured trajectory angle of the radiopaque marker.

[0065] Figure 10 shows a set of illustrations that provide examples of intravascular image features (e.g., automatically detected features) overlaid on extravascular images with and without contrast, according to exemplary embodiments of the present disclosure. Turning to the items identified in Figure 10, item 1000 is an extravascular image without contrast injection, overlaid with automated registration or automated intravascular features. Item 1002 is an extravascular image with contrast injection, overlaid with automated registration or automated intravascular features. Item 1004 is an overlaid lipid load (e.g., detected using NIRS or intravascular OCT, e.g., from an automatically detected lipid load map). Item 1006 illustrates an overlaid calcium load (e.g., detected using NIRS or intravascular OCT, e.g., from an automatically detected calcium load map). Item 1008 indicates the overlaid side branch location (e.g., detected via intravascular imaging, e.g., NIRS or intravascular OCT, or intravascular IVUS). Item 1010 indicates the overlaid stent descent flag location (e.g., for percutaneous coronary intervention (PCI) pre-leads, regarding where the optimal descent area for the stent would exist). For example, these flags can be automatically generated but may be editable based on user input (e.g., recommended based on intravascular imaging, NIRS or intravascular OCT, or intravascular IVUS, etc.). Item 1012 provides the overlaid lipid load (e.g., detected using NIRS or intravascular OCT, e.g., from an automatically detected lipid load map). Item 1014 provides the overlaid calcium load (e.g., detected using NIRS or intravascular OCT, or intravascular IVUS, e.g., from an automatically detected calcium load map). Item 1016 provides an overlaid side branch position (detected, for example, via extravascular imaging, intravascular imaging, or both).Item 1018 provides overlaid stent descent flag locations (for example, for pre-PCI leads regarding where the optimal descent zone for the stent would exist). For example, these flags can be automatically generated, but may be editable based on user input (for example, recommended based on intravascular imaging, e.g., NIRS or intravascular OCT, or intravascular IVUS).

[0066] In some exemplary embodiments of this disclosure, automatic alignment between intravascular and / or extravascular data as described herein can be used as a preprocessing step to improve the determination of physiological indices (e.g., ischemia index, FFR index, IMR index, CFR index, iFR index, etc.) based on physiological procedures derived from images (e.g., physiological procedures derived from images, optimized physiological workflows derived from images, etc.). In some exemplary embodiments of this disclosure, physiological function measurements derived from images can be aligned and overlaid within extravascular images, intravascular images, or longitudinal representations displayed within a user interface (UI).

[0067] According to further exemplary embodiments of the present disclosure, physiological function measurements derived from images from at least one of various imaging modalities (e.g., physiological function measurements derived from angiography, physiological function measurements derived from OCT, etc.) can be used to improve (e.g., refine alignment). For example, the location of the lowest FFR derived from OCT along the OCT imaging pullback may be a point on the angiographic image with the smallest lumen diameter, and this information can improve the alignment between the OCT modality and the angiography modality. Similarly, for example, the location of the lowest FFR derived from OCT along the OCT imaging pullback may be the location of the lowest FFR derived from angiography, or may include it, and this information can improve the alignment (e.g., automatic alignment) between the OCT modality and the angiography modality.

[0068] According to exemplary embodiments of this disclosure, automated image path registration can be performed during or after a PCI procedure. During a post-PCI procedure, one or more intravascular imaging datasets and one or more extravascular datasets (e.g., pre-PCI and post-PCI datasets) can be further automatically registered using strategies described herein to compare pre- and post-treatment measurements (e.g., pre- and post-treatment lumen profiles). According to some exemplary embodiments of this disclosure, pre-PCI intravascular imaging data can be registered to both pre-PCI extravascular imaging datasets and PCI extravascular imaging datasets. According to some exemplary embodiments of this disclosure, pre-PCI intravascular imaging data can be registered to post-PCI intravascular imaging datasets (for example, extravascular imaging datasets can also be optionally used to assist in registration). The information derived from such exemplary embodiments may include, among other important outcome indices (e.g., functional outcome indices), treatment reliability assessment, risk assessment (patient-level risk or plaque-level risk), malabposition, and treatment accuracy (e.g., stent placement accuracy).

[0069] In some exemplary embodiments of this disclosure, an automated detection algorithm can infer coronary artery segments (e.g., from an accepted coronary artery atlas or description, coronary malformation "CAA" defined segments, etc.) and / or coronary vessels (e.g., LAD, RCA, LCx, left main trunk, etc.) corresponding to another automatically detected feature or location in the image. In some exemplary embodiments, automatically detected (e.g., segmented) coronary artery features can first be aligned to a coronary artery atlas (e.g., a general coronary artery atlas, a personalized coronary artery atlas, a 3D coronary artery atlas, a 2D coronary artery atlas, etc.). In some exemplary embodiments of this disclosure, automatically detected (e.g., segmented) coronary artery features can first be aligned to a 3D dataset, such as one derived from a coronary computed tomography angiography (CTA / CCTA) dataset. In some exemplary embodiments of this disclosure, a previously obtained CTA dataset can assist in the automated alignment of synchronized intravascular and / or extravascular imaging data. Figure 11 is a system diagram of an intravascular imaging system 1101 according to an exemplary embodiment of the present disclosure. For example, the imaging system 1101 may include a display configured to acquire intravascular data and to align the intravascular data with extravascular data. In some exemplary cases, the intravascular data may include one or more intravascular images of a blood vessel. In some exemplary cases, the extravascular data may include one or more extravascular images (e.g., radiographic angiography, MRI, etc.) of vascular shape, physiological function, anatomical structure, or any combination thereof. In some exemplary cases, the imaging system may include a computer system (1106, 1410) for processing intravascular, extravascular, user interaction, or any combination thereof.

[0070] User interaction data may include user input data to the imaging system 1101, which may include patient information, landmark designation, system operating mode selection, image processing functions, or any combination thereof. In one exemplary case, the user may input data to the imaging system 1101 using a mouse and / or keyboard electrically coupled to a computer system (1106, 1410). The user may visualize a view (i.e., a user interface) configured for inputting data to the system via a first monitor 1102 and / or a second monitor 1104. In one exemplary case, the first monitor 1102 and / or the second monitor 1104 may include a touchscreen interface and keyboard for actions performed on intravascular and / or extravascular data, such as interaction, acquisition, or any combination thereof. In one exemplary case, user interaction data may include data resulting from the user interacting with extravascular and / or intravascular data (e.g., rotation, zoom in, contrast adjustment, brightness adjustment, distance measurement, etc.). A computer system (1106, 1410) may include, or communicate with, an electronic display 1414 (e.g., a first monitor 1102 and / or a second monitor 1144), which may include one or more view configurations (i.e., user interfaces (UIs)) 1416, as described elsewhere in this specification and also shown in Figure 12, for viewing intravascular data, extravascular data, aligned intravascular and extravascular data and / or any combination thereof.

[0071] In an exemplary case, a computer system (1106, 1410) may include an input interface 1105, which may include one or more input points and / or ports electrically coupled to the computer system (1106, 1410). The input interface 1105 may receive one or more data and / or streams of data from one or more imaging systems. For example, the input interface 1105 may receive X-ray angiography data, which the computer system (1106, 1410) can then align with intravascular data. In an exemplary case, the input interface 1105 may receive data from one or more medical devices derived from angiography, such as physiological function, MRI, computed tomography, spatial position, intravascular sensor data (e.g., intravascular physiological function), or any combination thereof, so as to be displayed and / or aligned with intravascular data. In some exemplary cases, the input interface 105 may receive wirelessly, through a wireless communication platform such as ad-hoc Wi-Fi, Bluetooth®, radio frequency, or any combination thereof, data to be aligned to extravascular data, as described elsewhere in this specification.

[0072] In some exemplary cases, a computer system (1106, 1410) may process data using one or more processors 1404, as described elsewhere in this specification. In some exemplary cases, one or more processors may comprise one or more graphics processing units, integrated circuit processors, or any combination thereof. Graphics processing units, due to their highly parallelized processor architecture, provide a configuration for processing complex, large datasets. For example, processing data using one or more graphics processing units can provide the exemplary system with the ability to align intravascular data with a real-time stream of extravascular data.

[0073] In one exemplary case, the computer system (1106, 1410) may be configured to process intravascular and extravascular data and / or images. The computer system (1400, 1106) as shown in Figure 14 may comprise a central processing unit and / or graphics processing unit (CPU and / or GPU, also herein referred to as “processor” and “computer processor”) 1404, which may be a single-core or multi-core processor or multiple processors for parallel processing. The computer system (1106, 1410) may further comprise memory or memory locations 1406 (e.g., random-access memory, read-only memory, flash memory), an electronic storage unit 1402 (e.g., a hard disk), a communication interface 1408 for communicating with one or more other devices (e.g., a network adapter), and peripheral devices 1410 such as cache, other memory, data storage devices, and / or electronic display adapters. The memory 1406, storage unit 1402, communication interface 1408, and peripheral devices (e.g., mouse, keyboard, etc.) 1410 can communicate with the CPU and / or GPU 1404 through a communication bus (solid wire) such as a motherboard. The storage unit 1402 may be a data storage unit (or data repository) for storing data. The computer system (1106, 1410) can be operationally coupled to a computer network ("Network") 1412 with the help of the communication interface 1408. Network 1412 may be the Internet, the Internet and / or an extranet, or an intranet and / or extranet communicating with the Internet. In some exemplary cases, Network 1412 may be a telecommunications and / or data network. Network 1412 may include one or more computer servers that can facilitate distributed computing such as cloud computing.Network 1412 can implement a peer-to-peer network, in an exemplary case, which may facilitate the behavior of devices connected to computer systems (1106, 1410) as clients or servers, with the help of computer systems (1106, 1410).

[0074] The CPU and / or GPU 1404 can execute a sequence of machine-readable instructions, which may be embodied in a program or software. Instructions can be directed to the CPU and / or GPU 1404, which can then be programmed or otherwise configured to acquire and / or process data produced by an imaging system, as described elsewhere herein.

[0075] In some exemplary embodiments of this disclosure, a central processing unit and / or graphics processing unit 1404 of a computer system (1106, 1410) can execute machine-executable or machine-readable code, which may be provided in software form, to transfer data generated by an imaging system to a network and / or cloud 1412 for further processing, classification, data clustering, or any combination thereof. In some exemplary cases, the data may comprise intravascular and / or extravascular data, as described elsewhere in this specification. In some exemplary cases, the data may comprise image pixel data from optical coherence tomography, radiographic angiography, computed tomography, intravascular ultrasound, spectroscopy, MRI, or any combination thereof.

[0076] In some embodiments, the CPU and / or GPU 1104 may be part of a circuit such as an integrated circuit. One or more other components of the system 1410 may be included in the circuit. In one exemplary case, the circuit may comprise an application-specific integrated circuit (ASIC).

[0077] The storage unit 1402 can store files such as drivers, libraries, and saved programs. The storage unit 1402 can store acquired data and / or images from radiographic angiography, optical coherence tomography, intravascular ultrasound, near-infrared spectroscopy, photoacoustics, or any combination thereof. In an exemplary case, intravascular and / or extravascular data and / or images may be stored in a cloud, a medical system electronic medical record (e.g., EPIC), or any combination thereof. In an exemplary case, the computer systems (1106, 1410) may include one or more additional data storage units located outside the computer systems (1106, 1410), such as on a remote server that communicates with the computer systems (1106, 1110) via an intranet or the internet 1412.

[0078] In one exemplary case, the imaging system 1101 may communicate electrically and / or optically with the imaging probe actuator 1110, and an imaging probe 1112, as seen in the imaging system 1101, may communicate electrically and / or optically with the imaging probe actuator 1110 through one or more electrical and / or optical communication wires 1108. In one exemplary case, the imaging probe 1112 may be removably coupled to the imaging probe actuator 110 such that a first imaging probe can be removed from the imaging probe actuator and replaced with a second imaging probe.

[0079] In some exemplary cases, the imaging probe may comprise an intravascular imaging probe. The intravascular imaging probe may comprise an imaging probe for optical coherence tomography, intravascular ultrasound, reflectivity, photoacoustics, near-infrared spectroscopy, fluorescence, or any combination thereof. In some exemplary cases, the imaging probe may acquire, collect, and / or detect intravascular data from the inner lumen and / or body of the vessel. In some exemplary cases, the intravascular data may comprise two-dimensional (e.g., circular section data) and / or volumetric intravascular data (i.e., one or more two-dimensional circular section data as a function of the optical axis length of the imaging probe). In some exemplary cases, the imaging probe may comprise one or more radiopaque markers and / or marks that can be visualized on an extravascular imaging modality, e.g., radiographic angiography, computed tomography, MRI, or any combination thereof.

[0080] In some exemplary cases, the imaging probe actuator 1110 can rotate and / or translate the imaging probe 1112 to acquire two- and / or three-dimensional intravascular datasets. In one exemplary case, the probe can be rotated by a stepper motor, a DC brushless motor, or any combination thereof, coupled to an optical rotary joint. In another exemplary case, the imaging probe actuator 1110 can translate the imaging probe 1112 using a stage, which may comprise a linear and / or planar translation stage. The stage translation and rotation of the imaging probe actuator 1110 can be set and / or adjusted by the user via one or more interfaces of the imaging system 1101, as described elsewhere in this specification. In some exemplary cases, the stage translation and rotation of the imaging probe actuator 1110 can be determined and / or set by the system based on pre-set standard values ​​relating to a particular type of imaging procedure or frequently used settings.

[0081] Exemplary aspects of the systems and methods provided herein, such as computer systems (1106, 1410), can be embodied in programming. Various aspects of the technology can typically be considered “products” or “manufactured goods” in the form of machine (or processor) executable code and / or associated data, carried on or embodied on a certain type of machine-readable medium. Machine-executable code can be stored on electronic storage units such as memory (e.g., read-only memory, random-access memory, flash memory) or hard disks. The “storage” type medium can include any tangible memory of a computer, processor, equivalent, or associated module thereof, such as various semiconductor memories, tape drives, disk drives, and equivalents, which can provide non-transient storage for software programs from time to time. All or part of the software can be communicated from time to time through the Internet or various other telecommunication networks. Such communication can facilitate, for example, the loading of software from one computer or processor to another, such as from a management server or host computer to an application server computer platform. Therefore, other types of media that can carry software elements include optical, electrical, and electromagnetic waves, such as those used across physical interfaces between local devices, through wired and optical terrestrial networks, and via various wireless links. Physical elements that carry such waves, such as wired or wireless links, optical links, or equivalents, can also be considered media that carry software. Unless limited to non-transient tangible “storage” media as used herein, the terms computer or machine “readable medium,” etc., refer to any medium involved in providing instructions to a processor for execution.

[0082] Therefore, machine-readable media such as computer executable code can take many forms, but are not limited to tangible storage media, carrier media, or physical transmission media. Non-volatile storage media can include optical or magnetic disks, such as any storage device in any computer or equivalent, which may be used to implement databases, etc. Volatile storage media can include dynamic memory, such as the main memory of such a computer platform. Tangible transmission media include coaxial cables, copper wires, and optical fibers, including wires that form buses within a computer system. Carrier media can take the form of electrical or electromagnetic signals or acoustic or optical waves, such as those generated during radio frequency (RF) and infrared (IR) data communications. Common forms of computer-readable media include, for example, floppy disks, flexible disks, hard disks, magnetic tapes, any other magnetic media, CD-ROMs, DVDs or DVD-ROMs, any other optical media, punched card paper tapes, any other physical storage media with perforation patterns, RAM, ROMs, PROMs and EPROMs, flash EPROMs, any other memory chips or cartridges, carriers for transporting data or instructions, cables or links for transporting such carriers, or any other media from which a computer can read programming code and / or data. Many of these forms of computer-readable media can be involved in transporting one or more sequences of one or more instructions to a processor for execution.

[0083] Computer systems and machine learning models In some embodiments, the system 100 disclosed herein may include a computer system (1106, 1410) suitable for implementing machine learning procedures and / or predictive models configured to analyze, process, segment, and / or label extravascular and / or intravascular data collected by an imaging system 1101, an imaging probe 1112, and an imaging probe actuator 1110, as described elsewhere herein. In some exemplary cases, one or more intravascular and / or extravascular images may be generated from the intravascular and / or extravascular data. In some exemplary cases, a predictive model, e.g., a machine learning model and / or machine learning procedure may perform operations on the intravascular and / or extravascular data such as analysis, extraction, summarization, reduction, prediction, processing, classification, segmentation, or any combination thereof. In some embodiments, the system disclosed herein may implement one or more machine learning procedures and / or models to identify, classify, process, and / or segment regions of interest in the intravascular and / or extravascular data.

[0084] In some exemplary cases, one or more categories and / or features of the data may be provided to one or more treatment parameter machine learning models and / or procedures to determine a proposed treatment and / or treatment parameters (e.g., the type of stent to be implanted and the spatially best location for implantation of the stent in order to achieve the clinical efficacy of the treatment). One or more treatment parameter machine learning models may be trained with prior features and corresponding treatment efficacy (i.e., whether any complications persisted after a clinical intervention using the system) to generate one or more trained treatment parameter machine learning models in order to predict effective treatments. The spatial orientation of labeled features and their relationships to each other may be other features that are determined and considered by the treatment parameter machine learning models.

[0085] In some exemplary cases, one or more categories and / or features of data relating to extravascular data may comprise background data, healthy vascular morphology, narrowed vascular morphology, or occluded vessels. In some exemplary cases, one or more categories of data relating to intravascular data may comprise epithelial vascular tissue, intimal vascular tissue, adventitia vascular tissue, plaque within vascular tissue, fragile plaque within vascular tissue, or any combination thereof. In some exemplary cases, one or more categories and / or features of intravascular data may comprise spectroscopic (e.g., near-infrared) signatures of intravascular vascular tissue. For example, one or more categories and / or features may classify the composition of vascular plaque based on their spectroscopic signature. In some exemplary cases, one or more categories may comprise spectroscopic signatures of calcium or lipids. In some exemplary cases, machine learning models and / or procedures may be used to preprocess intravascular and / or extravascular data prior to classifying the features of the data. In some exemplary cases, preprocessing intravascular and / or extravascular data may involve mathematical manipulation of the data, such as denoising, smoothing, averaging, sharpening, brightness and / or contrast adjustment, or any combination thereof. In some exemplary cases, features and / or categories of intravascular and / or extravascular data may be extracted without any preprocessing steps.

[0086] In some exemplary cases, machine learning procedures may need to extract and derive relationships between features because conventional statistical techniques may be insufficient. In some exemplary cases, machine learning procedures can be used in conjunction with conventional statistical techniques. In some exemplary cases, conventional statistical techniques can provide pre-processed features to the machine learning procedure.

[0087] In some exemplary embodiments of this disclosure, any number of features can be classified by a machine learning procedure. The machine learning procedure can classify at least one feature. In some exemplary cases, a set of features may include about 1 to 5 features. In some exemplary cases, a set of features may include about 5 to 10 features. In some exemplary cases, a set of features may include about 10 to 50 features.

[0088] In some exemplary embodiments of this disclosure, the machine learning procedure may be, for example, an unsupervised learning procedure, a supervised learning procedure, or a combination thereof. Unsupervised learning procedures may include, for example, clustering, hierarchical clustering, k-means clustering, mixed models, DBSCAN, OPTICS procedure, VoxelMorph procedure, anomaly detection, local outlier factorization, neural networks, autoencoders, deep belief networks, Hebb learning, generative adversarial networks, self-organized maps, expectation maximization (EM) procedures, methods of moments, blind source separation techniques, principal component analysis, independent component analysis, non-negative matrix factorization, singular value decomposition, or a combination thereof. Supervised learning procedures may include, for example, support vector machines, linear regression, logistic regression, linear discriminant analysis, decision trees, k-nearest neighbor procedures, neural networks, similarity learning, or a combination thereof. In some embodiments, the machine learning procedure may comprise a deep neural network (DNN). The deep neural network may comprise a convolutional neural network (CNN). CNNs can be, for example, U-Net, ImageNet, LeNet-5, AlexNet, ZFNet, GoogleNet, VGGNet, ResNet18, or ResNet. Other neural networks can be, for example, deep feedforward neural networks, recurrent neural networks, LSTM (Long Short-Term Memory), GRU (Gated Recurrent Unit), autoencoders, variational autoencoders, adversarial autoencoders, denoising autoencoders, sparse autoencoders, Boltzmann machines, RBMs (Restricted Beam Networks), deep belief networks, generative adversarial networks (GANs), deep residual networks, capsule networks, or attention / transformer networks.

[0089] In some illustrative examples, machine learning models may include clustering, scalar vector machines, kernel SVMs, linear discriminant analysis, quadratic discriminant analysis, neighboring component analysis, manifold learning, convolutional neural networks, reinforcement learning, random forests, naive Bayes, Gaussian mixtures, hidden Markov models, Monte Carlo, restricted Boltzmann machines, linear regression, or any combination thereof.

[0090] In some exemplary cases, machine learning procedures can include ensemble learning procedures such as bagging, boosting, and stacking. These machine learning procedures can be applied individually to multiple features being extracted.

[0091] In some exemplary embodiments, the system can apply one or more machine learning procedures and / or ensembles of machine learning procedures.

[0092] In some embodiments, the machine learning procedure may have various parameters. These parameters may include, for example, the learning rate, small batch size, the number of reference time points to be trained on, momentum, learning weight decay rate, or the number of neural network layers.

[0093] In some exemplary embodiments, the learning rate may be approximately 0.00001 to 0.1.

[0094] In some embodiments, the small batch size can be approximately 16 to 128.

[0095] In some exemplary embodiments, the neural network may comprise neural network layers. The neural network may have at least about 2 to 1,000 or more neural network layers.

[0096] In some exemplary embodiments, the number of reference time points to be trained may be at least about 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 25, 30, 35, 40, 45, 50, 55, 60, 65, 70, 75, 80, 85, 90, 95, 100, 150, 200, 250, 500, 1,000, 10,000, or more.

[0097] In some exemplary embodiments, the momentum may be at least about 0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, or greater. In some exemplary embodiments, the momentum may be at most about 0.9, 0.8, 0.7, 0.6, 0.5, 0.4, 0.3, 0.2, 0.1, or less.

[0098] In some exemplary embodiments, the learning weight decay rate may be at least about 0.00001, 0.0001, 0.001, 0.002, 0.003, 0.004, 0.005, 0.006, 0.007, 0.008, 0.009, 0.01, 0.02, 0.03, 0.04, 0.05, 0.06, 0.07, 0.08, 0.09, 0.1, or greater. In some exemplary embodiments, the learning weight decay rate may be up to approximately 0.1, 0.09, 0.08, 0.07, 0.06, 0.05, 0.04, 0.03, 0.02, 0.01, 0.009, 0.008, 0.007, 0.006, 0.005, 0.004, 0.003, 0.002, 0.001, 0.0001, 0.00001, or less.

[0099] In some exemplary embodiments, the machine learning procedure may use a loss function. The loss function may be, for example, regression loss, mean absolute error, mean bias error, hinge loss, Adam optimizer, and / or cross-entropy.

[0100] In some embodiments, the parameters of the machine learning procedure can be adjusted with the help of humans and / or computer systems.

[0101] In some embodiments, the treatment parameter machine learning model and / or procedure may prioritize certain features. The treatment parameter machine learning model and / or procedure may prioritize features that may be relevant to the determination of anatomical and / or physiological features for characterizing variations in vascular geometry and composition. In some exemplary cases, vascular geometry and composition may classify parts of the vessel as affected areas (e.g., thin-capsular fibrous plaques, fragile plaques, fixed plaques, etc.). In some exemplary cases, features may be prioritized using a weighted system. In some exemplary cases, features may be prioritized based on probability statistics based on the frequency and / or amount of occurrence of the features. The machine learning procedure may prioritize features with the help of humans and / or computer systems.

[0102] In some exemplary embodiments, one or more of the features may be used in conjunction with machine learning or conventional statistical techniques to determine whether segments of intravascular and / or extravascular data are likely to contain artifacts. Identified artifacts may result from optical mismatch, object movement during intravascular and / or extravascular data acquisition, laser power instability, laser pulse frequency jitter, object movement via respiration or ambient tremors, or any combination thereof. In some exemplary cases, motion sensors or other sensors may be used as additional inputs to artifact reduction machine learning models and / or procedures. In some exemplary cases, identified artifacts may be excluded so as not to be used in vascular anatomical structures and / or disease classifications.

[0103] In some exemplary cases, a machine learning procedure may prioritize certain features to reduce computational cost, conserve processing power, save processing time, increase reliability, or reduce the use of random access memory.

[0104] The methods described herein can be implemented as machine-executable code (e.g., computer processor) stored on a non-transient electronic storage medium of the computer system 1410, such as memory 1406 or electronic storage unit 1402. The machine-executable or machine-readable code can be provided in software form. During use, the code can be executed by a processor (i.e., CPU and / or GPU) 1404. In some exemplary cases, the code can be read from storage unit 1402 and stored on memory 1406 for immediate access by processor 1404. In some exemplary cases, electronic storage unit 1402 can be omitted, and the machine-executable instructions are stored on memory 1406.

[0105] The code can be pre-compiled and configured for use with a machine having a processor adapted to run the code, or it can be compiled during runtime. The code can be supplied in a programming language, which may be chosen to enable the code to be executed in a pre-compiled or as-compile manner.

[0106] Figure 12 shows an exemplary display of co-located intravascular and extravascular imaging on an intravascular image using a user interface (UI) that utilizes the exemplary system of Figure 11, according to an exemplary embodiment of the present disclosure. As shown in Figure 12, the UI has three main viewports for viewing multimodal data: a first extravascular viewport for displaying angiography data, a second intravascular viewport for displaying intravascular OCT-NIRS data, and a third longitudinal viewport for displaying longitudinal images, measurements, and representations. In this embodiment, the extravascular viewport, which displays a single frame from an extravascular angiography dataset, includes an overlay of automatically detected intravascular features (such as calcium load, lipid load, and side branch location) that are automatically co-located to the vascular architecture in the extravascular image. In this embodiment, the extravascular viewport also includes an overlay for automatically detected extravascular features such as side branch locations, radiopaque marker locations (e.g., radiopaque cans specifying intravascular imaging locations in time), and descent flags for treatment guidance (e.g., stent descent flags, e.g., automatically generated stent descent flags, e.g., manually selected stent descent flags). The intravascular viewport displays a single frame from an intravascular dataset (e.g., multimodal OCT-NIRS data) with an overlay for automatically detected intravascular features such as plaque composition. Plaque composition (e.g., presence of lipids, e.g., lipid concentration) can be automatically detected by a spectroscopic modality (e.g., NIRS or fluorescence) or an imaging modality (e.g., OCT or IVUS). The longitudinal viewport illustrates a longitudinal representation of 1D, 2D, and 3D information from either intravascular or extravascular data, or any automatically detected and / or measured features from either data source (e.g., lumen diameter / area, e.g., plaque size and composition, e.g., plaque size and composition exceeding a certain clinical threshold). One of the viewports can also be configured to display priori data or patient information.One of the viewports may also be configured to further include a representation of physiological function measurement data (e.g., physiological functions derived from images, e.g., physiological functions derived from OCT, e.g., physiological functions derived from angiography).

[0107] Figure 13 illustrates an exemplary display 1300 of co-aligned intravascular and extravascular imaging on an intravascular image using a user interface (UI) that utilizes the exemplary system of Figure 11, according to yet another exemplary embodiment of the present disclosure. The angiography overlay may include an under-expansion marker and a juxtaposition marker, as well as a flag position that defines the stent position after stent placement. All overlays can be automatically generated from the intravascular imaging.

[0108] In particular, Figure 14 illustrates a computer system 1410 according to an exemplary embodiment of the present disclosure. In summary, item 1402 represents a storage unit, item 1404 represents a processor (i.e., a CPU and / or GPU), item 1406 represents random access memory (RAM), item 1408 represents a communication interface (e.g., a network adapter), item 1412 represents a network, item 1414 represents an electronic display, and item 1416 represents a user interface (UI).

[0109] The foregoing merely illustrates the principles of the present disclosure. Various modifications and alterations of the embodiments described will be obvious to those skilled in the art, taking into account the teachings herein. Therefore, it should be understood that it will be possible for those skilled in the art to devise numerous systems, arrangements, and procedures, not expressly shown or described herein, that embody the principles of the present disclosure and thus may be in the spirit and scope of the present disclosure. Various different exemplary embodiments may be used together and interchangeably with each other, so as should be understood by those skilled in the art. In addition, certain terms used in the present disclosure, including, for example, data and information, in its specification, drawings, and claims, may be used synonymously in certain cases. It should be understood that there may be cases where these words and / or other words that may be synonymous with each other may be used synonymously herein, but where such words may not be intended to be used synonymously. Furthermore, to the extent that prior art knowledge is not expressly incorporated herein by reference above, this is expressly incorporated herein as a whole. All referenced publications are incorporated herein as a whole by reference.

Claims

1. A method for obtaining information about at least one feature of an object, Obtaining intravascular and extravascular datasets, Automatically receiving information about at least one feature in the extravascular dataset, Based on the received information relating to at least one of the features, an imaging path is automatically generated. The intravascular dataset is automatically aligned to the imaging path, and aligned intravascular data is generated. Automatically overlay the representation of the aligned intravascular data into at least one image of the extravascular dataset. Methods that include...

2. The method according to claim 1, wherein at least one of the features includes a radiopaque marker on an intravascular probe.

3. The method according to claim 2, wherein the aligned intravascular dataset includes at least one intravascular sensing dataset.

4. The method according to claim 3, wherein the at least one intravascular sensing dataset includes data relating to at least one of pressure, electric field, or fluid.

5. The method according to claim 2, wherein the aligned intravascular dataset includes at least one intravascular imaging dataset.

6. The method according to claim 5, wherein the at least one intravascular imaging dataset includes at least one of optical coherence tomography (OCT) data, intravascular ultrasound (IVUS) data, near-infrared spectroscopy (NIRS) data, reflectance data, or fluorescence lifetime imaging (FLIM) data.

7. The method according to claim 1, wherein the extravascular dataset includes data relating to coronary angiography images.

8. The method according to claim 1, wherein the overlaid representation indicates the location of the constriction.

9. The method according to claim 8, wherein the overlaid representation further indicates the type of plaque in the stenosis.

10. The method according to claim 1, wherein the overlaid representation indicates the measurement output at a specific location along the coronary artery.

11. The method according to claim 10, wherein the measurement output comprises at least one of plaque load, lipid load, or calcium load.

12. The method according to claim 1, wherein automatically detecting the aforementioned features includes performing a machine learning-based procedure relating to object segmentation.

13. The method according to claim 12, wherein the machine learning procedure comprises at least one of U-Net or Res-Net.

14. The method according to claim 1, wherein automatically detecting the aforementioned features includes performing a machine learning-based procedure relating to object detection.

15. The method according to claim 1, wherein the automatic generation of the image path includes automatically interpolating data points.

16. The method according to claim 1, further comprising aligning the image path between extravascular images in the extravascular dataset.

17. The method according to claim 16, wherein the automatic alignment of the image path includes performing a machine learning-based non-rigid transformation procedure between the images in the extravascular dataset.

18. The method according to claim 1, further comprising automatically segmenting the coronary vascular system within the extravascular dataset.

19. The method according to claim 1, further comprising segmenting the data associated with guidewires within the extravascular dataset.

20. The method according to claim 1, wherein the intravascular dataset is a multimodal intravascular imaging and spectroscopy dataset.

21. A system for obtaining information about at least one feature of an object, Obtaining intravascular and extravascular datasets, Automatically receiving information about at least one feature in the extravascular dataset, Based on the at least one received feature, the imaging path is automatically generated. The intravascular dataset is automatically aligned to the imaging path, and aligned intravascular data is generated. The representation of the aligned intravascular data is automatically overlaid onto the image of the extravascular dataset. A system comprising a computer processing configuration configured or programmed to perform a certain task.

22. The system according to claim 21, wherein at least one of the features includes a radiopaque marker on an intravascular probe.

23. The system according to claim 21, wherein the aligned intravascular dataset includes at least one intravascular sensing dataset.

24. The system according to claim 23, wherein the at least one intravascular sensing dataset includes data relating to at least one of pressure, electric field, or fluid flow.

25. The system according to claim 21, wherein the aligned intravascular dataset includes at least one intravascular imaging dataset.

26. The system according to claim 25, wherein the at least one intravascular imaging dataset includes at least one of optical coherence tomography (OCT) data, intravascular ultrasound (IVUS) data, near-infrared spectroscopy (NIRS) data, reflectance data, or fluorescence lifetime imaging (FLIM) data.

27. The system according to claim 21, wherein the extravascular dataset includes data relating to coronary angiography images.

28. The system according to claim 21, wherein the overlaid representation indicates the location of the constriction.

29. The system according to claim 28, wherein the overlaid representation further indicates the type of plaque in the stenosis.

30. The system according to claim 21, wherein the overlaid representation indicates the measurement output at a specific location along the coronary artery.

31. The system according to claim 30, wherein the measurement output comprises at least one of plaque load, lipid load, or calcium load.

32. The system according to claim 21, wherein the aforementioned features are automatically detected by performing a machine learning-based procedure for object segmentation (e.g., U-Net, Res-Net, etc.).

33. The system according to claim 32, wherein the machine learning procedure comprises at least one of U-Net or Res-Net.

34. The system according to claim 21, wherein the aforementioned features are automatically detected by performing a machine learning-based procedure for object detection.

35. The system according to claim 21, wherein the automatic generation of the image path includes automatically interpolating data points.

36. The system according to claim 21, wherein the computer processing configuration is further configured to align the image path between extravascular images in the extravascular dataset.

37. The system according to claim 36, wherein the image path is automatically aligned by performing a machine learning-prepared non-rigid transformation procedure between the images in the extravascular dataset.

38. The system according to claim 21, wherein the computer processing configuration is further configured to automatically segment the coronary vascular system within the extravascular dataset.

39. The system according to claim 21, wherein the computer processing configuration is further configured to segment the data associated with guidewires in the extravascular dataset.

40. The system according to claim 21, wherein the intravascular dataset is a multimodal intravascular imaging and spectroscopic dataset.

41. A method for obtaining information about at least one feature of an object, Obtaining intravascular and extravascular datasets, Automatically detect markers in each of at least two images of the extravascular dataset, Automatically aligning at least one feature within the at least two images, Based on the aligned features, an imaging path is automatically generated based on the position of at least one of the markers in at least one of the at least two images. The intravascular dataset is automatically aligned to the imaging path, and aligned intravascular data is generated. Automatically overlay the aligned intravascular data representation into at least one of the two images mentioned above. Methods that include...

42. The method according to claim 41, wherein the at least one feature includes at least a portion of at least one of a guidewire, an imaging catheter, a guide catheter, or an intravascular device.

43. The method according to claim 41, wherein the at least one feature includes at least a portion of a coronary artery.

44. The method according to claim 43, wherein at least one of the features is a coronary artery that is at least partially filled with contrast agent.

45. The method according to claim 41, wherein the at least one feature is a coronary vascular feature in the absence of contrast agent.

46. The method according to claim 41, wherein automatically aligning the intravascular dataset to the imaging path includes automatically addressing vascular shortening in the extravascular image.

47. The method according to claim 46, wherein the aforementioned automatic handling is performed using constrained interpolation.

48. The method according to claim 41, wherein the acquisition of the intravascular dataset and the extravascular dataset are synchronized with each other.

49. The method according to claim 41, wherein automatically aligning the intravascular dataset to the imaging path includes automatically addressing at least one temporal offset in the acquisition of the extravascular dataset and the intravascular dataset.

50. The method according to claim 49, wherein the at least one temporal offset is characterized prior to the acquisition of the intravascular data.

51. The method according to claim 41, wherein the representation of the intravascular data is aligned and displayed on top of at least one of the two images from the extravascular data set.

52. The method according to claim 41, wherein the representation of the intravascular data is aligned and displayed on any image from the extravascular data set, thereby achieving a threshold derived from imaging.

53. The method according to claim 41, wherein the threshold derived from the imaging includes at least one of a registration reliability threshold or an image quality threshold.

54. The method according to claim 41, wherein the representation of the intravascular data is aligned and displayed on an image automatically selected from the extravascular dataset.

55. The method according to claim 41, wherein the representation of the intravascular data is aligned and displayed on an improved image from the extravascular data set.

56. The method according to claim 41, wherein the alignment of the intravascular dataset includes automatically colocalizing automatically detected side branches in the intravascular dataset and the extravascular dataset.

57. The method according to claim 41, wherein the automatic generation of the image path includes interpolating data points in a aligned space provided by the at least one aligned feature.

58. The method according to claim 41, wherein the alignment of the intravascular dataset to the imaging path includes automatically colocalizing the automatically segmented lumen profiles within the intravascular dataset and the extravascular dataset.

59. The method according to claim 41, wherein the alignment of the intravascular dataset to the imaging path includes automatically colocalizing the automatically detected stenosis locations in the intravascular dataset and the extravascular dataset.

60. The method according to claim 41, wherein the automatic detection of the at least one feature comprises performing a machine learning-based procedure relating to at least one of object segmentation or object detection.

61. The method according to claim 41, wherein the automatic alignment of the at least one feature comprises performing a machine learning-prepared non-rigid transformation procedure.

62. The method according to claim 41, wherein the overlaid representation shows at least one of automatically detected vascular characteristics or measured vascular characteristics from the intravascular dataset at a specific location along the coronary artery.

63. The method according to claim 41, wherein the marker indicates the sensing position of the intravascular probe.

64. The method according to claim 41, wherein the intravascular dataset is an intravascular sensing dataset.

65. The method according to claim 41, wherein the intravascular dataset is an intravascular imaging dataset.

66. The method according to claim 41, wherein the extravascular dataset is a set of coronary angiography images.

67. The method according to claim 41, wherein the overlaid representation indicates the location of the automatically detected narrowing.

68. The method according to claim 67, wherein the overlaid representation further indicates the type of plaque in the stenosis.

69. The method according to claim 67, wherein the overlaid representation indicates a lesion with an ischemic index exceeding a predetermined threshold.

70. The method according to claim 69, wherein the ischemia index is an index derived at least partially from an image.

71. The method according to claim 41, wherein the marker is a radiopaque marker.

72. An intravascular imaging system for acquiring information about at least one feature of an object, Receiving intravascular and extravascular datasets, Automatically control the detection of markers in each of at least two images of the extravascular dataset, Automatically aligning at least one feature within the at least two images, Based on the aligned features, an imaging path is automatically generated based on the position of at least one of the markers in at least one of the at least two images. The intravascular dataset is automatically aligned to the imaging path, and aligned intravascular data is generated. Automatically overlay the aligned intravascular data representation into at least one of the two images mentioned above. An intravascular imaging system comprising a computer processing configuration configured or programmed to perform the following actions.

73. The imaging system according to claim 72, wherein the at least one feature includes at least a portion of at least one of a guidewire, an imaging catheter, a guide catheter, or an intravascular device.

74. The imaging system according to claim 72, wherein the at least one of the features includes at least a portion of a coronary artery.

75. The imaging system according to claim 72, wherein the at least one feature is a coronary artery at least partially filled with contrast agent.

76. The imaging system according to claim 72, wherein the at least one feature is a coronary vascular feature in the absence of a contrast agent.

77. The imaging system according to claim 72, wherein automatically aligning the intravascular dataset to the imaging path includes automatically addressing vascular shortening in the extravascular image.

78. The imaging system according to claim 77, wherein the aforementioned automatic handling uses constrained interpolation.

79. The imaging system according to claim 72, wherein the acquisition of the intravascular dataset and the extravascular dataset are synchronized with each other.

80. The imaging system according to claim 72, wherein the automatic alignment of the intravascular dataset to the imaging path includes automatically addressing at least one temporal offset in the acquisition of the extravascular dataset and the intravascular dataset.

81. The imaging system according to claim 80, wherein the at least one temporal offset is characterized prior to the acquisition of the intravascular data.

82. The imaging system according to claim 72, wherein the representation of the intravascular data is aligned and displayed on top of at least one of the two images from the extravascular dataset.

83. The imaging system according to claim 72, wherein the representation of the intravascular data is aligned and displayed on any image from the extravascular dataset, thereby achieving a threshold derived from imaging.

84. The imaging system according to claim 72, wherein the threshold derived from the imaging includes at least one of a registration reliability threshold or an image quality threshold.

85. The imaging system according to claim 72, wherein the representation of the intravascular data is aligned and displayed on an image automatically selected from the extravascular dataset.

86. The imaging system according to claim 72, wherein the representation of the intravascular data is aligned and displayed on an improved image from the extravascular data set.

87. The imaging system according to claim 72, wherein the alignment of the intravascular dataset includes automatically colocalizing automatically detected side branches within the intravascular dataset and the extravascular dataset.

88. The imaging system according to claim 72, wherein the automatic generation of the image path includes interpolating data points in a aligned space provided by the at least one aligned feature.

89. The imaging system according to claim 72, wherein the alignment of the intravascular dataset to the imaging path includes automatically colocalizing the automatically segmented lumen profiles within the intravascular dataset and the extravascular dataset.

90. The imaging system according to claim 72, wherein the alignment of the intravascular dataset to the imaging path includes automatically colocalizing the automatically detected stenosis locations in the intravascular dataset and the extravascular dataset.

91. The imaging system according to claim 72, wherein the automatic detection of the at least one feature comprises performing a machine learning-based procedure relating to at least one of object segmentation or object detection.

92. The imaging system according to claim 72, wherein the automatic alignment of the at least one feature includes performing a machine learning-prepared non-rigid body transformation procedure.

93. The imaging system according to claim 72, wherein the overlaid representation shows at least one of automatically detected vascular characteristics or measured vascular characteristics from the intravascular dataset at a specific location along the coronary artery.

94. The imaging system according to claim 72, wherein the marker indicates the sensing position of the intravascular probe.

95. The imaging system according to claim 72, wherein the intravascular dataset is an intravascular sensing dataset.

96. The imaging system according to claim 72, wherein the intravascular dataset is an intravascular imaging dataset.

97. The imaging system according to claim 72, wherein the extravascular dataset is a set of coronary angiography images.

98. The imaging system according to claim 72, wherein the overlaid representation indicates the location of the automatically detected stenosis.

99. The imaging system according to claim 98, wherein the overlaid representation further indicates the type of plaque in the stenosis.

100. The imaging system according to claim 98, wherein the overlaid representation indicates a lesion with an ischemic index exceeding a predetermined threshold.

101. The imaging system according to claim 100, wherein the ischemia index is an index derived at least partially from the image.

102. The imaging system according to claim 72, wherein the marker is a radiopaque marker.

Images