Live Angiogram Registration

Abstract

This disclosure relates to a method for combining reference information with co-registered angiography frames to provide live x-ray images, such as fluoroscopic images, that are motion corrected in real time. In part, this disclosure describes an improvement over existing workflows where the user must remember where vascular features from OCT are and use the reference frames to estimate the location of the desired stent placement. Disclosed herein is a method for correlating the live imaging feed with a reference image such that the user can dynamically zoom and pan the live imaging feed and the reference image.

Description

[Technical field]

[0001] The present disclosure relates generally to the field of intravascular imaging and data collection systems and methods.

[0002] [CROSS REFERENCE TO RELATED APPLICATIONS] This application claims the benefit of the filing dates of U.S. Provisional Patent Application No. 63 / 323,335, filed March 24, 2022, entitled "Live Angioplasty," and U.S. Provisional Patent Application No. 63 / 477,877, filed December 30, 2022, the disclosures of which are incorporated herein by reference. [Background technology]

[0003] Coronary artery disease is one of the leading causes of death worldwide. The ability to better diagnose, monitor, and treat coronary artery disease can be of life-saving importance. During some diagnostic procedures, various medical and technical personnel view multiple sources of information, including angiographic and fluoroscopic images as part of stent planning or delivery. Switching between these two sources of data can be tiring and often requires personnel to keep in mind the association of images and other data when switching from changing fluoroscopic to angiographic views as part of stent planning and delivery. Furthermore, manipulating the live angiographic feed and the reference images can be cumbersome and can lead to inaccurate association between the two images. Summary of the Invention

[0004] This disclosure describes a method to register angiographic images from different time points, with and without contrast, so that features from the contrast image can be accurately mapped onto the non-contrast image, allowing the user to directly view contrast and pullback features in the non-contrast image, reducing errors due to using only a reference frame.

[0005] The present disclosure relates to a method that includes using one or more processors to access a live imaging feed of a blood vessel, receiving a reference image of the blood vessel, aligning the live imaging feed and the reference image such that objects depicted in a field of view of the reference image correspond to objects depicted in a field of view of the live imaging feed, receiving user input associated with the reference image, adjusting a field of view of the reference image based on the user input, and automatically adjusting a field of view of the live imaging feed based on the user input to the reference image. According to an aspect of the present disclosure, the user input can be a selected zoom level.

[0006] According to aspects of the disclosure, the method may further include aligning the live imaging feed and the reference image using one or more processors to identify an area of ​​interest on the reference image, calculate a scale factor based on a size of an object represented in a field of view of the reference image and a size of an object represented in a field of view of the live imaging feed, the field of view being associated with a grid, identify an origin of the grid associated with the live imaging feed, transpose the origin onto the reference image, calculate a distance coordinate based on a distance between the area of ​​interest and the transposed origin, and transform coordinates of the area of ​​interest to match coordinates of the live imaging feed based on the scale factor and the distance coordinate. The scale factor may be based on a selected zoom level and is calculated as a ratio between the zoomed reference image and the unzoomed reference image. According to aspects of the disclosure, the method may further include simultaneously displaying the transformed area of ​​interest on the reference image based on the transformed coordinates and the live imaging feed using one or more processors.

[0007] According to aspects of the present disclosure, the object may include at least a portion of a blood vessel. The live imaging feed may include an extravascular image derived from an angiography, fluoroscopy, x-ray, nuclear magnetic resonance, or computer-aided tomography based imaging system. The reference image may include a static extraluminal image co-registered with endovascular data, which may be derived from an intravascular device that collects data using at least one of optical coherence tomography (OCT), intravascular ultrasound, near-infrared spectroscopy (NIRS), or micro-OCT.

[0008] According to aspects of the present disclosure, the reference frame may further include a surgical guide overlay. In the steps of adjusting the reference image and adjusting the live imaging feed, the user input may include panning the live imaging feed and the reference image simultaneously.

[0009] The present disclosure relates to a system including one or more processors configured to access a live imaging feed of a blood vessel, receive a reference image of the blood vessel, align the live imaging feed and the reference image such that objects depicted in a field of view of the reference image correspond to objects depicted in a field of view of the live imaging feed, receive user input associated with the reference image, adjust a field of view of the reference image based on the user input, and simultaneously adjust a field of view of the live imaging feed based on the user input to the reference image, where the user input may include a selected zoom level.

[0010] According to aspects of the present disclosure, the one or more processors may be further configured to: identify a selected area of ​​interest on the reference image; calculate a scale factor based on a size of an object represented in a field of view of the reference image and a size of an object represented in a field of view of the live imaging feed, the field of view being associated with a grid; identify an origin of the grid associated with the live imaging feed; transpose the origin onto the reference image; calculate a distance coordinate based on a distance between the area of ​​interest and the transposed origin; and transform coordinates of the selected area of ​​interest to match coordinates of the live imaging feed based on the scale factor and the distance coordinates. The scale factor may be based on a selected zoom level and is calculated as a ratio between the zoomed reference image and the unzoomed reference image. According to aspects of the present disclosure, the one or more processors may be further configured to simultaneously display the transformed area of ​​interest on the reference image based on the transformed coordinates and the live imaging feed.

[0011] This disclosure relates to a method for combining real-time motion-corrected reference information with co-registered angiography frames to provide live x-ray images, such as fluoroscopy images. In part, this disclosure describes an improvement over existing workflows where the user must remember where vascular features from OCT are and use the reference frames to estimate the location of the desired stent placement. Disclosed herein is a method whereby motion-corrected OCT information is displayed to the user directly on top of the fluoroscopy feed.

[0012] One aspect of the disclosure relates to a method of providing annotations to a live angiography feed, the method comprising: identifying one or more portions of a patient's blood vessel through an intravascular imaging procedure; matching frames in the live angiography feed with frames from the intravascular imaging procedure such that both frames are captured during corresponding portions of the patient's cardiac cycle; correcting for rigid body displacements using rigid registration; and identifying spots on an image overlay based on the matched frames and corrected respiratory motion that correspond to portions of the patient's blood vessel shown in the live angiography feed and one or more portions identified during the intravascular imaging procedure.

[0013] The method may further include the step of the frames from the intravascular imaging procedure being co-registered frames of endoluminal and extraluminal data.

[0014] The method may further include a step in which the endoluminal data is derived from an OCT probe or an IVUS probe.

[0015] The method may further include the step of the extraluminal data being derived from an angiography, fluoroscopy, x-ray, nuclear magnetic resonance, or computer-assisted tomography based imaging system.

[0016] The method may further include determining a cardiac cycle of the patient using the cardiovascular timing parameters, the pressure signal and the heart rate.

[0017] The method may further include a step in which the rigid registration includes considering rigid displacement of the endovascular data, the rigid displacement caused by patient respiration, patient movement, machine movement, or camera movement.

[0018] The method may further include a step in which matching frames in the live angiographic feed with frames from the intravascular imaging procedure is predicted by an artificial technology technique.

[0019] The method may further include placing a marker on the live angiographic feed.The method may further include correlating coordinates of the live angiographic feed with coordinates of frames from the intravascular imaging procedure to facilitate simultaneous zoom functionality.

[0020] Another aspect of the present disclosure relates to a data collection system comprising a probe configured to capture endoluminal data, an imaging device configured to capture extraluminal image data, and a computing device configured to receive the endoluminal and extraluminal image data, wherein the computing device is configured to identify one or more portions of a patient's blood vessel through an intravascular imaging procedure, match frames in the live angiography feed with frames from the intravascular imaging procedure such that both frames are captured during corresponding portions of the patient's cardiac cycle, correct for rigid body displacements using rigid body registration, and identify spots on an image overlay that correspond to portions of the patient's blood vessel shown in the live angiography feed and the one or more portions identified during the intravascular imaging procedure based on the matched frames and the corrected respiratory motion.

[0021] The system may further include, where the endoluminal data is derived from an OCT probe or an IVUS probe.

[0022] The system may further include, where the extraluminal data is derived from an angiography, fluoroscopy, x-ray, nuclear magnetic resonance, or computed assisted tomography based imaging system.

[0023] The system may further include, wherein the computing device is further configured to determine a cardiac cycle of the patient using the cardiovascular timing parameters, the pressure signal and the heart rate.

[0024] The system may further include, wherein the rigid registration includes considering rigid body displacement of the endovascular data.

[0025] The system may further include that the rigid body registration is caused by patient respiration, patient movement, machine movement, or camera movement.

[0026] The system can further include matching frames in the live angiography feed with frames from the intravascular imaging procedure predicted by an artificial engineering technique.

[0027] The system may further include the computing device being further configured to place a marker on the live angiography feed.The system may further include the computing device being configured to correlate coordinates of the live angiography feed with coordinates of frames from the intravascular imaging procedure to facilitate simultaneous zoom functionality.

[0028] Another aspect of the present disclosure relates to a non-transitory computer-readable medium storing instructions executable by one or more processors to perform a method for providing annotations to a live angiography feed, the method including identifying one or more portions of a patient's blood vessel through an intravascular imaging procedure, matching frames in the live angiography feed with frames from the intravascular imaging procedure such that both frames are captured during corresponding portions of the patient's cardiac cycle, correcting for rigid body displacements using rigid body registration, and identifying spots on an image overlay that correspond to portions of the patient's blood vessel shown in the live angiography feed and the one or more portions identified during the intravascular imaging procedure based on the matched frames and the corrected respiratory motion.

[0029] Another aspect of the disclosure relates to a method of simultaneously zooming a live angiography feed and a reference image, the method including: identifying, using one or more processors, a first origin on the live angiography feed, the origin being a center of the live angiography feed; identifying, using one or more processors, a second origin on the reference image, the second origin being different from the first origin; calculating, using one or more processors, a scale factor based on a selected zoom level; transforming, using one or more processors, selected coordinates of the reference image using the scale factor; determining, using one or more processors, a distance between the transformed selected coordinates and a center of the reference image; and aligning, using one or more processors, coordinates of the live angiography feed and coordinates of the reference image based on the distance from the transformed selected coordinates to the center. [Brief description of the drawings]

[0030] [Figure 1] 1 is an exemplary system according to an aspect of the present disclosure. [Diagram 2] FIG. 1 is a flow diagram illustrating a method for correction in image analysis according to an aspect of the present disclosure. [Diagram 3] 13 is an exemplary graph illustrating tip position data according to aspects of the present disclosure. [Figure 4] 1 is an exemplary graph illustrating predicted data in beats per minute (BPM) according to an aspect of the present disclosure. [Diagram 5] 1A and 1B are diagrams illustrating an example of automated angiogram segmentation according to an embodiment of the present disclosure. [Figure 6] FIG. 13 shows an example of visualization of phase-matched frames between live angiogram and contrast agent data. [Figure 7] FIG. 1 illustrates an example of rigid body displacement according to aspects of the present disclosure. [Figure 8] FIG. 1 illustrates an example of rigid body motion correction according to aspects of the present disclosure. [Figure 9] 1 is a flow diagram illustrating an exemplary method of stent mapping and deployment according to aspects of the present disclosure. [Figure 10] FIG. 13 shows side-by-side fluoroscopy and reference angiography frames with OCT pullback features. [Figure 11] FIG. 13 is a diagram of an example deployment screen according to an aspect of the present disclosure. [Figure 12] FIG. 13 is a diagram of an example deployment screen according to an aspect of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0031] The present disclosure provides for displaying annotations on a live angiography feed that mark either or both vascular features and locations for desired stent placement. Annotations can be identified during an intravascular imaging procedure, such as pullback of an intravascular imaging probe. Angiography images can be registered from different time points, with and without contrast, such that features from the contrast image can be accurately mapped onto the non-contrast image. A user can view contrast and pullback features directly in the non-contrast image, reducing errors from using only a reference frame. The present disclosure also provides a method for correlating a live angiography feed with a reference angiography image, such that a user can dynamically zoom and pan between the live angiography feed and the reference angiography image.

[0032] FIG. 1 illustrates a data collection system 100 for use in collecting intravascular and extravascular data. The system may include a data collection probe 104. The data collection probe 104 may be, for example, an OCT probe, an IVUS catheter, a micro-OCT probe, a near-infrared spectroscopy (NIRS) sensor, or any other device that may be used to image a blood vessel 102. In some examples, the data collection probe 104 may be a pressure wire, a flow meter, or the like. Although the examples provided herein relate to an intravascular imaging device such as an OCT probe, the use of an OCT probe is not intended to be limiting. For example, an IVUS catheter, a pressure wire, or another intravascular data collection device may be used in conjunction with or in place of an OCT probe. A guide wire, not shown, may be used to introduce the probe 104 into the blood vessel 102. The probe 104 may be introduced and pulled back along the length of the blood vessel while collecting data. The intravascular data set, or frames of image data, may be used to identify features such as narrowing of the blood vessel due to plaque, calcium deposits, or the like.

[0033] Optical coherence tomography (OCT) is a catheter-based imaging modality that uses light to peer into the walls of coronary arteries and generate images of them for study. Using coherent light, interferometry, and micro-optics, OCT can provide video-rate in-vivo tomography inside diseased blood vessels with micrometer-level resolution. Viewing subsurface structures at high resolution using a fiber-optic probe makes OCT particularly useful for minimally invasive imaging of internal tissues and organs. This level of detail possible with OCT allows users to diagnose and even monitor the progression of coronary artery disease. IVUS imaging uses high-frequency sound waves to create intravascular images.

[0034] Intravascular imaging of a patient's body parts provides a useful diagnostic tool for physicians and others. For example, intravascular imaging of a coronary artery can reveal the location of a narrowing or stenosis. This information helps a cardiologist choose between invasive coronary artery bypass surgery and less invasive catheter-based procedures such as angioplasty or stent delivery.

[0035] The probe 104 may be connected to a subsystem 108 via an optical fiber 106. The subsystem 108 may include a light source, such as a laser, an interferometer having a sample arm and a reference arm, various optical paths, a clock generator, photodiodes, and / or other OCT components, IVUS components, micro-OCT components, NIRS components, and pressure wire components.

[0036] The probe 104 can be connected to a receiver 110. According to some examples, the receiver 110 can be a balanced photodiode-based system. The receiver 110 can be configured to receive light collected by the probe 104. The probe 104 can be coupled to the receiver 110 via a wired or wireless connection.

[0037] The system 100 may further include or be configured to receive data from an external imaging device 120. The external imaging device may be, for example, an imaging system based on angiography, fluoroscopy, x-ray, nuclear magnetic resonance, computer-aided tomography, etc. The external imaging device 120 may be configured to non-invasively image the blood vessel 102. According to some examples, the external imaging device 120 may acquire one or more images before, during, and / or after the pullback of the data collection probe 104.

[0038] The external imaging device 120 can communicate with the subsystem 108. According to some examples, the external imaging device 120 can be wirelessly coupled to the subsystem 108 via a communication interface, such as Wi-Fi or Bluetooth. In some examples, the external imaging device 120 can communicate with the subsystem 108 via a wire, such as an optical fiber. In yet another example, the external imaging device 120 can be indirectly communicatively coupled to the subsystem 108 or the computing device 112. For example, the external imaging device 120 can be coupled to a separate computing device (not shown) that communicates with the computing device 112. As another example, image data from the external imaging device 120 can be transferred to the computing device 112 using a computer-readable storage medium.

[0039] The subsystem 108 can include a computing device 112. One or more steps can be performed automatically or without user input, such as navigating an image, inputting information, selecting an input, interacting with an input, etc. In some examples, one or more steps can be performed based on receiving user input via a mouse click, keyboard, touch screen, verbal command, etc.

[0040] The computing device may include one or more processors 113 , memory 114 , instructions 115 , data 116 , and one or more modules 117 .

[0041] The one or more processors 113 may be any conventional processor, such as a commercially available microprocessor. Alternatively, the one or more processors may be dedicated devices, such as application specific integrated circuits (ASICs) or other hardware-based processors. Although FIG. 1 functionally depicts the processor, memory, and other elements of device 112 as being within the same block, those skilled in the art will appreciate that a processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be stored within the same physical enclosure. Similarly, the memory may be a hard drive or other storage medium located in a different enclosure than that of device 112. Thus, reference to a processor or computing device will be understood to include reference to a collection of processors or computing devices or memories that may or may not operate in parallel.

[0042] The memory 114 can store information accessible by the processor, including instructions 115 that can be executed by the processor 113, and data 116. The memory 114 can be a type of memory that operates to store information accessible by the processor 113, including a non-transitory computer-readable medium or other medium that stores data readable by an electronic device, such as a hard drive, a memory card, a read-only memory ("ROM"), a random access memory ("RAM"), an optical disk, and other writable and read-only memories. The subject matter disclosed herein may include different combinations of the above, where different portions of the instructions 115 and data 116 are stored on different types of media.

[0043] The memory 114 can be read, stored, or modified by the processor 113 according to the instructions 115. For example, although the disclosure is not limited by a particular data structure, the data 116 can be stored in a computer register, a relational database as a table with multiple different fields and records, an XML document, or a flat file. The data 116 can also be formatted in a computer readable format, such as, but not limited to, binary values, ASCII, or Unicode. Further by way of example only, the data 116 can be stored as a bitmap consisting of pixels stored compressed or uncompressed, or various image formats (e.g., JPEG), vector-based formats (e.g., SVG), or computer instructions for drawing graphics. Additionally, the data 116 can include associated information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations), or sufficient information to identify information used by a function to compute the associated data.

[0044] The instructions 115 may be any set of instructions executed by the processor 113 directly, e.g., in machine code, or indirectly, e.g., in script. In that regard, the terms "instructions," "application," "step," and "program" may be used interchangeably herein. The instructions may be stored in object code format for direct processing by the processor, or in any other computing device language, including scripts or collections of separate source code modules that are translated on demand or pre-compiled. The functions, methods, and routines of the instructions are described in more detail below.

[0045] Module 117 may include a co-registration module and a live angiography co-registration module. According to some examples, the modules may include other software-based components used to process image data or respond to patient triggers to facilitate co-registration of various types of image data by software such as pre-processing software, transformations, matrices, and other software-based components, or used to otherwise perform such co-registration. The modules may include lumen detection using scanline-based or image-based techniques, stent detection using scanline-based or image-based techniques, indicator generation, stent expansion assessment and evaluation, stent landing zone detection and indicators for deployed stents, angiography and intravascular imaging co-registration, and other modules that aid and are programmed to perform the methods disclosed herein.

[0046] The computing device 112 can be adapted to co-register the endovascular data with the luminal image. For example, the computing device 112 can access a co-registration module to co-register the endovascular data with the luminal image. The luminal image can be an extraluminal image, such as an angiogram, an x-ray, or the like. The co-registration module can co-register the endovascular data, such as an endovascular image, pressure readings, virtual flow reserve ("VFR"), fractional flow reserve ("FFR"), resting full-cycle ration ("RFR"), flow rates, or the like, with the extraluminal image. In some examples, the co-registration module can co-register the endovascular data with an endoluminal image, such as an endoluminal image captured by an OCT probe, an IVUS probe, a micro-OCT probe, or the like.

[0047] In one example, the co-registration module can co-register endoluminal data captured during the pullback with one or more extraluminal images. For example, the extraluminal image frames can be pre-processed. Various matrices, such as convolution matrices, Hessians, etc., can be applied on a pixel-by-pixel basis to modify brightness, remove, or otherwise alter a given angiographic image frame. As discussed herein, the pre-processing step can enhance, modify, and / or remove features of the extraluminal image to increase the accuracy, processing speed, success rate, and other characteristics of subsequent processing steps. Vascular centerlines can be determined and / or calculated. In some examples, the vascular centerlines can be superimposed or otherwise displayed on the pre-processed extraluminal images. According to some examples, the vascular centerlines can represent the trajectory that the collection probe 104 took through the vessel during the pullback. In some examples, the centerlines may be referred to as traces. Additionally or alternatively, marker bands or radiopaque markers can be detected in the extraluminal image frames. According to some examples, the extraluminal image frames and the data received by the collection probe 104 can be co-registered based on the determined positions of the marker bands.

[0048] The system may include a monitor 122. The monitor 122 may include, but is not limited to, an electrocardiogram monitor configured to generate data related to cardiac function and indicate various states of the subject, such as systole and diastole. Since the geometry of the heart, including the coronary arteries, is approximately the same at certain cardiac phases even across different cardiac cycles, knowing the cardiac phase may be used to assist in tracking the vascular centerline. In part, the present disclosure relates to systems and methods that use timing parameters of the cardiovascular system and / or signals, such as an electrocardiogram (ECG) and pressure signals, such as an aortic pressure signal, to identify an angiography frame that corresponds to a portion of the cardiac cycle. Other timing indicators, such as the dicrotic notch and those used to identify systole and diastole, may also be used. Additionally, the disclosed systems and methods may identify an angiography frame that corresponds to a current fluoroscopic image based on real-time correlation of either or both of such signals with the timing parameters.

[0049] 2 is a flow diagram illustrating motion correction applied to the co-registered images. In particular, the motion correction module can account for cardiac motion, respiratory motion, body motion, etc., prior to overlaying the co-registered images onto the live angiography feed. Motion correction can be applied to contrast agent image features in order to accurately overlay the contrast agent image features on the live angiography feed.

[0050] According to some examples, for each live fluoroscopy frame, a corresponding angiography frame is identified based on signal or timing parameter correlation, such as through ECG correlation. Additionally, in some embodiments, intravascular probe markers, such as radio opaque markers, are used under angiography and correlated therewith as part of the intravascular imaging session as described above. In this manner, intravascular imaging facilitates angiography correlation.

[0051] In the first phase, cardiac phase matching 201 is performed. During cardiac phase matching 201, cardiac motion is removed. Cardiac motion relates to the motion displayed on the image from the beating of the heart. In this phase, cardiac motion is removed from the angiography co-registered image. In the second phase, rigid body registration 202 is performed. During rigid body registration 202, rigid body motion is removed. Rigid body motion relates to respiratory motion, such as lung expansion and contraction during imaging, and patient motion. In addition to this, camera movement over time can be approximated as rigid body motion. In the third phase, after correcting the angiography co-registered image for both cardiac activity and respiratory or movement motion for each frame, the stent landing zone can be mapped onto the live angiography video feed along with the intravascular imaging pullback features from the contrast angiography image. Each of these phases is described below in turn.

[0052] The 2D projection of the 3D vessels in an angiogram may distort cardiac motion. To correct for cardiac motion, a contrast image that is in the same phase of the cardiac cycle as the angiogram frame being displayed at a particular instant during display of the live angiogram feed is identified.

[0053] Correcting for cardiac motion can include predicting the patient's beats per minute (BPM), which allows for finding an angiography frame from an intravascular imaging pullback that is phase-matched to the current non-contrast fluoroscopy frame.

[0054] There are at least two modalities used to estimate the BPM. In some examples, the modalities can be imaging modalities such as OCT, angiography, IVUS, NIRS, μOCT, etc. For both modalities, the BPM prediction algorithm is based on custom autocorrelation values ​​within a defined window size.

[0055] 3 and 4 show various examples of determining a patient's cycle length or BPM. In some examples, the second modality can be aortic (AO) pressure data. The AO pressure can have a defined period between heartbeats. The AO pressure can be used to determine or predict a patient's BPM in some examples. As shown in FIG. 3 and FIG. 4, the BPM can be determined based on a point that is at the same or substantially the same location in each cycle. The AO pressure data or other timing or signal data, such as an ECG, can be used to determine which portion of the cardiac cycle corresponds to the given AO pressure data. According to some examples, the AO pressure data can correspond to identifying systole and diastole in the cardiac cycle or other trackable periods to the cardiac cycle.

[0056] According to some examples, the first modality can be an angiogram. According to some examples, vascular features and / or imaging system features can be visible in both contrast angiogram 501 (FIG. 5B) and non-contrast angiogram 502 (FIG. 5A). The imaging system features can be, for example, wire tips 503 and / or markers 504 on the wire. The computing device can be configured to detect features or segment the image of the artery or other structure into various constituent tissue types or regions of interest through an artificial intelligence (AI) system, such as one or more machine learning models. In some examples, the machine learning model can be used to detect features in the non-contrast image. The machine learning model can be trained using raw angiogram images. In certain examples, the machine learning model can be trained using labeled image training data, such as angiogram images with and without contrast.

[0057] The AI ​​system receives a plurality of training images, each training image annotated with the location of one or more features of the imaging system. The AI ​​system may receive training image data labeled with the location of features of the imaging system detected in each image. As part of receiving the plurality of training images, the system may split the data into a plurality of sets, such as image frames, for training, testing, and validation.

[0058] The AI ​​system processes a plurality of training images and annotates each image. The AI ​​system uses the processed plurality of training images to train a machine learning model. In some examples, the model is configured to generate as output a segmentation map that represents one or more predicted regions of interest that include portions of the input images that are predicted to be characteristic of the imaging system.

[0059] The machine learning model may be trained according to any technique for supervised learning, and generally may be trained according to a training technique that uses a dataset in which at least a portion of some training examples are labeled. For example, the machine learning model may be one or more neural networks with model parameter values ​​that are updated as part of a training process using, for example, backpropagation with gradient descent, either for individual image frames or for batches of image frames.

[0060] In some examples, the machine learning model includes, at least in part, an encoder-decoder architecture with skip connections. As another example, the machine learning model may include one or more neural network layers as part of an autoencoder trained to learn a compact (encoded) representation of an image from unlabeled training data, such as images taken using OCT, IVUS, NIRS, OCT-NIRS, μOCT, or any other of a variety of other imaging techniques. The neural network layers may be further trained on input training images as described herein, and the machine learning model may benefit from a more extensive training set by being at least in part trained using unlabeled training images.

[0061] The training may be performed using a loss function that quantifies the difference between a location predicted by the system for the imaging system feature relative to a ground-truth location for the imaging system feature received as part of the training data for the input image. The loss function may be, for example, the distance between the predicted and ground-truth locations for the imaging system feature measured at one or more pairs of points at the predicted and ground-truth locations. In general, any loss function that compares each pixel between the training image with the predicted annotation and the training image with the ground-truth annotation may be used.

[0062] The system may run the training until it determines that one or more stopping criteria have been met, for example, the stopping criteria may be any or all of a preset number of epochs, a minimal improvement of the system between epochs as measured using a loss function, the passage of a predetermined amount of wall clock time, and / or until a computational budget (e.g., a predetermined number of processing cycles) is exhausted.

[0063] An existing model may be augmented with an existing machine learning model trained to predict non-lipid regions of interest to also predict the location of imaging system features as one of a number of different output channels. The machine learning model may be trained to output multiple channels, each corresponding to a respective region of interest, which may be represented, for example, as one or more output images and / or as segmentation maps or other data that characterize regions of the input image as corresponding to various regions of interest.

[0064] In part, the AI ​​system is designed to be co-located with or coupled to an imaging system, such as an intravascular imaging system, an ultrasound system, or an x-ray system, such as an angiography system or a fluoroscopy system. For example, the positions of given features, such as the wire tip 503, can be plotted over time. By plotting the x and y positions of these features over time, the patient's BPM over time can be estimated.

[0065] Figures 5A and 5B show examples of automated angiogram segmentation. Figure 5A shows features detected by a deep learning model in a non-contrast image 502, as described above. The detected features can be, for example, wire tip 503 and marker location 504. Figure 5B shows features detected by a deep learning model in a contrast image 501. The feature can be, for example, wire tip 503.

[0066] According to some examples, automated angiography segmentation can be applied. For example, features detected by a machine learning model can be displayed in the non-contrast image 502. Such features can be displayed, for example, using a mask. As one example, different parts of a blood vessel can be identified using a mask having different colors, where each color corresponds to a segmented portion of the blood vessel. It should be understood that other types of masks can also be applied that indicate any of many different features.

[0067] Once the AO pressure data or other timing data or signals have been mapped to a cardiac cycle or other clock or timing subsystem, the data of interest can be co-registered with pre-computed angiography data that is co-registered with the OCT / IVUS markers 504. Generally, the present systems and methods identify cardiac cycles that track or correspond to particular frames of image data or other parameters of interest.

[0068] FIG. 6 illustrates an example of phase-matched image frames 603. The phase-matched image frames 603 may include matched live angiography image frames 601 and contrast image frames 602. The live angiography image frames 601 may correspond or substantially correspond to the contrast angiography image frames 602. According to some examples, the contrast angiography image frames 602 may include endovascular data obtained during co-registration of the angiography and endovascular image frames. The cardiac cycle may include repeating patterns such as repeated pressure rises, dicrotic notches, and ECG pulses. These features of the live cardiac cycle data may be correlated or matched with the initial set of cardiac cycle data.

[0069] FIG. 7 illustrates rigid body displacement in an angiographic image, where the path of the guidewire 701 and the position or placement of the catheter 702 on the image do not match the position of the blood vessel 703. Rigid body displacement can be due to rigid body motion, for example. In particular, rigid body displacement can be caused by rigid body motion, such as respiratory motion, such as lung expansion and contraction during imaging, patient motion, camera movement over time, etc. To account for rigid body displacement, rigid transformation techniques can be used to compensate or correct for this type of motion. Features that are visible in both contrast and non-contrast angiographic images are used for the rigid registration 202 step of motion correction.

[0070] FIG. 8 illustrates angiographic images after a rigid body deformation has been applied. For example, images 802A and 804A illustrate non-contrast angiographic images with a centerline overlaid. The centerline can be determined, for example, using an angiographic image taken with contrast. As shown, the centerline in each image 802A, 804A is offset from the vessel. The system can apply a rigid body transformation technique to adjust the positioning of the vessel centerline based on the determined rigid body displacement. As shown in images 802B, 804B, after the system applies the rigid body transformation technique, the centerline can match or substantially match the vessel in the image.

[0071] In one embodiment, the co-registered angiographic and endovascular images may be combined or co-registered with the live angiographic image or video such that any or all of the endovascular markers, endovascular image data, endovascular data, and other parameters may be overlaid onto the live angiographic image or video. Stent detection, stent expansion, side branches, and other image data detected based on the collected endovascular data set may be linked to or displayed relative to the live angiographic and endovascular images as a result of the co-registration between the three image data sets.

[0072] These various sets of data, extraluminal imaging, intravascular imaging, and live angiograms, can be combined, interlaced, used, juxtaposed, or integrated in various ways to present a co-registered combination of data to the end user. Angiogram images can be registered from various time points, with and without contrast, so that features from the contrast images can be accurately mapped onto the non-contrast images. In this regard, diagnostic imaging can be performed using less contrast solution without sacrificing accuracy or image quality.

[0073] The use of angiographic images, fluoroscopic data, endovascular data, and / or other imaging modalities in support of cardiovascular diagnosis and stenosis treatment would be of great value if it could be done on rapid timescales and in a manner that aids physicians. Addressing the various competing obstacles to these goals represents a significant technical challenge. The use of co-registered data and signals such as AO, ECG, systolic transitions, diastolic transitions, etc., along with the generation of various interlaced, static, combined, and fused data sets including streams of data having live subsets of frames and stored, historical, or otherwise non-live subsets of frames, may be used in various embodiments.

[0074] In part, the present disclosure relates to systems and methods for overlaying angiographic co-registered images and / or data (i.e., images or data or both) onto a live angiogram. Additionally, the present disclosure relates to systems and methods for combining angiographic co-registered information with one or more live angiographic feeds. Directly viewing contrast and pullback features of a non-contrast live angiographic feed reduces errors due to using only a reference frame.

[0075] If various data collection and diagnostic systems, such as angiography systems using OCT, IVUS, or other intravascular imaging systems, use imaging probes with radiopaque markers to track a given probe, the angiography data may be co-registered with the intravascular data. Both of these data sets may be co-registered, correlated, or cross-correlated with the live angiography feed using pressure signals, EKG signals, dicrotic notch signals and position, and other timing signals used to monitor aortic pressure or other pressure signals.

[0076] The correct map landing zone can be identified upon completion of the first and second phases. The map can be, for example, an overlay on the live angiography feed. Thus, the correct landing zone can correspond to the correct location on the patient's vessel where the marker is to be drawn in the live angiography feed. Although cardiac phasing and motion correction are described above as occurring during different phases, in other examples, cardiac phasing and motion correction can be performed in different orders, at overlapping times, or simultaneously.

[0077] After correcting the angiographic co-registered images for both cardiac activity and respiratory / locomotion for each frame, the stent landing zone, along with the intravascular imaging pullback features from the contrast angiographic images, may be mapped onto the live angiographic video feed shown in FIG. 9.

[0078] FIG. 9 illustrates an exemplary method for co-registering intravascular and / or image data with an extraluminal image and then co-registering or pairing the co-registered image with a live angiography feed.

[0079] In block 910, intravascular image data and extravascular image data are acquired simultaneously. Intravascular image data may be acquired during the pullback. The intravascular image data may be OCT, IVUS, NIRS, etc. The extraluminal image data may be acquired before, during, and / or after the pullback. The extravascular image data may be angiographic images. Both data sets are generated, transmitted, processed, and stored in a central computer. The images may be co-registered based on markers or common reference points.

[0080] In block 920, the intravascular image data may be co-registered with the extraluminal image data. After data acquisition, a determination is made as to which angiographic frame represents the closest reference point to the initial OCT frame. This determination may be made by either a human user or an automatic computer routine. Either a human user or a computer algorithm may inspect the angiographic data and estimate which frame corresponds to the initial frame of the pullback. For example, the computing device 112 may include one or more modules configured to automatically operate to capture angiographic images, tag each image with its acquisition time, and assist in co-registration of the acquisition-time tagged intravascular image data. Additionally, the computing device 112 may include one or more modules configured to automatically detect radiopaque markers on each angiographic image corresponding to an intravascular image. The co-registration module calculates the path of the intravascular imaging catheter on all angiographic images corresponding to the intravascular image acquisition during the pullback of the probe through the vessel being imaged.

[0081] In one embodiment, side branches detected in an OCT frame of data using OCT image processing may be used as input to improve co-registration with angiography data. For example, in one embodiment, each OCT frame includes a flag (yes / no) indicating whether side branches are present or not. Furthermore, once co-registration is obtained, the location of stents, calcium deposits, lipid deposits, thrombus, thin capped fibroatheroma (TCFA or "vulnerable plaque"), vessel normalization, side branch detection, FFR values ​​(which may be calculated as vascular resistance ratio (VRR) values ​​based on OCT image data), lumen size values, stents, and various other data described herein may be overlaid on the angiography or OCT image taking into account the co-registration between the data sets.

[0082] At block 930, the system may perform a morphology assessment. In some examples, the morphology assessment may be cardiac motion and / or rigid displacement. To account for the identified cardiac motion, the system may phase align the image frames. In some examples, to account for the identified rigid displacement, the system may apply a rigid transformation technique.

[0083] At block 940, the system can pair the co-registered and corrected image with a live angiographic image and / or video. The corrected image can be, for example, co-registered endoluminal and extraluminal image data with phase-matched and / or rigid body displacement techniques applied. In some examples, the co-registered and corrected image can be paired with a fluoroscopic video. Using the endovascular and extravascular image data sets stored in the memory device, the computing device can pair the co-registered image with the live fluoroscopic video using the same or similar reference points marked during co-registration. For example, the co-registered image and the live fluoroscopic video are paired using any or all of the distance from the point of interest, the time of capture, and reference points such as markers.

[0084] At block 950, the system may output the corrected image overlaid on either or both of the live angiographic image and the fluoroscopic image. According to some examples, the output may include annotations shown in association with the vessels displayed in the live angiographic feed. The annotations may match or substantially match locations identified during a previous intravascular imaging procedure for stent placement or other procedures.

[0085] The above is useful in co-registering live angiographic or fluoroscopic data. In particular, real-time correlation is used to effectively identify the portion of the cardiac cycle that can be tracked over time against other data or signals, such as AO pressure data or ECG data for angiographic data, and to correct for rigid body motion. This then facilitates selecting any frame from a set of angiographic image frames and replacing such angiographic frame with a live angiographic frame. Stent mapping or substitution may be used, or picture-in-picture representation may be used, or others. Various interlacing techniques may be used. This frame exchange supports the use of a library of images from angiograms previously generated with contrast agents.

[0086] In one embodiment, during the endovascular pullback, the image frame is dark under the angiogram. There is a discrete period for the pullback that may be tracked and indexed relative to the angiogram frame and aligned relative to the AO pressure data, ECG data, dicrotic notch, or other timing data. In one embodiment, all subsequent interlacing of the frames may be performed using the AO data or other timing data. One or more displays may be used to pull frames from the live feed using a frame grabber and display the live angiogram data, either interlaced with the angiogram frames or combined with the endovascular data. Intravascular image markers, such as OCT / IVUS markers, and other information may be combined, overlaid, or otherwise used with the live angiogram feed. This fusion or combination of various types of image data with the live angiogram data may be used to assist with various results, such as knowing exactly where the stent landing zone is located.

[0087] 10 shows an example of overlaying annotations from an intravascular imaging procedure onto a live angiography feed. Data from co-registered images can be overlaid onto the live angiography image in accordance with aspects of the present disclosure described above.

[0088] In this particular example embodiment, for example, Figure 10 shows a live angiogram image 1001 displayed with stent planning markers 1002, 1003, and 1004 overlaid on the angiogram image. The stent planning markers 1002, 1003, 1004 may correspond to annotations made by a physician during an intravascular imaging procedure. For example, the intravascular imaging procedure may be performed using an OCT probe, an IVUS probe, or other type of probe. The procedure may identify portions of a patient's blood vessels that have narrowing due to plaque, calcium deposits, etc.

[0089] These stent planning markers 1002-1004 can be based on markers from the OCT pullback that are mapped onto the angiogram images. The relative positions of these markers move in different angiogram images as the patient's cardiac cycle moves the various arteries in the heart. However, by correlating both the intravascular and angiogram images to the patient's cardiac cycle and rigid body motion, the stent planning markers 1002-1004 can be overlaid on each angiogram image in a manner that maintains their proper position relative to the artery in which the stent is to be placed. Thus, the system provides more accurate placement of stent planning markers onto the live angiogram feed from the intravascular imaging procedure. These markers and other indicia help reduce regional misplacement in procedures such as stent delivery and placement by reliably targeting the areas that the physician planned to address in the intravascular imaging procedure.

[0090] FIG. 11 shows an example of an unfolded screen with grid coordinates overlaid on the screen. The left screen shows a live fluoroscopy angiography feed 1110. The right screen shows a reference angiography image 1120 with a stent plan marked between brackets 1121. Both images 1110 and 1120 reflect the field of view of the imaging device. Objects in the field of view should correspond to each other. For example, objects displayed in the live angiography feed should correspond to objects displayed in the reference image. In some examples, the displayed objects are portions of blood vessels.

[0091] The system can place a grid layer over the displayed images. In some examples, the grid can be inherent or visible to the user. The grid layer can be associated with the field of view of each image such that the grid layer remains constant as the image changes based on the position of the imaging device. The grid layer can create coordinates for the image. Each grid layer can include an origin from which the system will calculate coordinates.

[0092] Boxed areas on the display highlight areas of interest (AOI) 1130 and 1131 selected by the user. A reference angiogram 1120 with a stent plan 1121 overlaid can be used as a guide to aid in accurate positioning of the device. The reference angiogram 1120 can be co-registered as described above. In some instances, the grid overlay is not visible to the user and is simply implicit for the system to use.

[0093] The system may include a feature that allows a user to zoom and pan the reference angiographic image. The zoom feature may be capable of simultaneously zooming and panning the reference angiographic image 1120 and the live angiographic feed 1110. This would provide a user with the ability to zoom in on an AOI in the live angiographic feed 1110 while the user is deploying an intravascular device, such as an interventional device. The system may include a processor having code that operates the steps necessary to facilitate the zoom feature. In some examples, the grid layer may be adjusted based on the selection of the zoom feature. For example, if the user selects to enlarge the image by a factor of 2, the grid layer may also be enlarged to reflect the size of the image.

[0094] The system may be configured to correlate the two images by transforming the coordinates of a first image to match the coordinates of a second image. For example, to correlate a live angiography feed 1110 with a reference angiography image 1120, the system may transform the coordinates of the reference angiography image 1120 to match the coordinates of the live angiography feed 1110. Transforming the coordinates may include determining an origin as the base coordinate or 0,0 of each image. In some examples, the images may have different origins, which may cause the system to have to consider different origins when correlating the two images. In the example shown in FIG. 11, the first origin 1141 or 0,0 of the live angiography feed 1110 is located at the center of the image. The origin 1140 or 0,0 of the reference angiography image 1120 is in the upper left corner of the image. It should be understood that in other examples, the origin of each image may be in a different location than that shown in FIG. 11.

[0095] In some examples, the system is configured to receive a zoom input from a user. The zoom input can be how much the user wants to enlarge or reduce the image, such as a magnification number. For example, the user can select an AOI and decide to zoom the image and video screen by 3 times. The system can use the zoom input to calculate a scale factor. The scale factor can be calculated using the ratio between the new image and the original image. For example, for a zoom input of 3 times, the ratio would be 3:1 and the scale factor would be 3.

[0096] A user may select an area within any image for review, such as AOI 1131 of reference angiogram image 1120. The display may magnify the area around the selected point, such as a magnified image of the AOI. In some examples, the system may select two points within the user-selected area, such as the top left and bottom right coordinates of the AOI. The selected points may be transformed using the determined scale factor.

[0097] The selected point may further be converted from absolute coordinates to distance coordinates. When the system correlates the first image to the second image, the absolute coordinates of the first image may be calculated relative to an origin of the first image, such as the second origin 1140 of the reference angiography image 1120. The distance coordinates of the first image may be calculated by determining the distance from each absolute coordinate to the origin of the second image, such that the distance coordinates of the first image and the origin coordinates of the second image match. For example, if the first image is a reference image and the second image is a live angiography feed with an origin at the center of the live angiography feed, the distance coordinates would be calculated from the absolute coordinates of the reference image to the center of the reference image. Specifically, in the reference angiography image 1120, the absolute coordinates of the top left coordinate of the AOI 1131 may be understood as -3, -4, and the distance coordinates of the top left coordinate of the AOI 1131 may be understood as 10, -9. The distance coordinates may be utilized to convert the remaining coordinates of the AOI. The system can determine the remaining coordinates based on the absolute coordinates of the remaining coordinates, the size and shape of the AOI, etc. For example, if the top left corner is determined to be 10,-9, the bottom right coordinate of the AOI can be determined to be 0,-2.

[0098] Once the scale factor has been applied and the distance coordinates determined, the system can correlate the distance coordinates of the reference angiogram image to the coordinates of the live angiogram feed so that the coordinates are equal. The system can operate simultaneously on the reference angiogram image 1120 and the live angiogram feed 1110.

[0099] 12 shows an example of an unfolded screen with grid coordinates overlaid on the screen. The left screen shows a live angiography feed 1210 of fluoroscopy. The right screen shows a reference angiography image 1220 with a stent plan marked between brackets 1221. This figure shows the unfolded screen with both the live angiography feed 1210 and the reference angiography image 1220 simultaneously zoomed to the AOI. After transforming the coordinates of the reference angiography image 1220 as described above, the system may be able to manipulate the live angiography feed 1210 and the reference angiography image 1220 simultaneously. The manipulations may be input to the system by the user, such as zoom and pan.

[0100] Although the invention has been described herein with reference to particular embodiments, it is to be understood that these embodiments are merely illustrative of the principles and applications of the present invention. It is therefore to be understood that numerous modifications can be made to the illustrative embodiments and that other arrangements can be devised without departing from the spirit and scope of the invention as defined by the appended claims.

Claims

1. accessing a live imaging feed of the blood vessel using one or more processors; receiving, using the one or more processors, a reference image of the blood vessel; using the one or more processors to register the live imaging feed and the reference image such that objects represented in a field of view of the reference image correspond to objects represented in a field of view of the live imaging feed; receiving a user input associated with the reference image; adjusting the field of view of the reference image based on the user input, using the one or more processors; automatically adjusting the field of view of the live imaging feed based on the user input to the reference image, using the one or more processors; The method comprising:

2. The method of claim 1 , wherein the user input comprises a selected zoom level.

3. The step of registering the live imaging feed and the reference image includes: identifying an area of ​​interest on the reference image using the one or more processors; calculating, using the one or more processors, a scale factor based on a size of the object represented in the field of view of the reference image and a size of the object represented in the field of view of the live imaging feed, the field of view being associated with a grid; identifying, using the one or more processors, an origin of the grid associated with the live imaging feed; using the one or more processors to transpose the origin onto the reference image; calculating, using the one or more processors, a distance coordinate based on a distance between the area of ​​interest and the displaced origin; converting, using the one or more processors, coordinates of the area of ​​interest based on the scale factor and the distance coordinates to match coordinates of the live imaging feed; The method of claim 2 further comprising:

4. The method of claim 3 , wherein the scale factor is based on the selected zoom level and is calculated as a ratio of a zoomed reference image to a non-zoomed reference image.

5. The method of claim 3 , further comprising using the one or more processors to simultaneously display the transformed area of ​​interest on the reference image based on the transformed coordinates and the live imaging feed.

6. The method of claim 1 , wherein the object comprises at least a portion of the blood vessel.

7. The method of claim 1 , wherein the live imaging feed comprises extravascular images derived from an angiography, fluoroscopy, x-ray, nuclear magnetic resonance, or computed assisted tomography based imaging system.

8. 2. The method of claim 1, wherein the reference image comprises a static extraluminal image co-registered with endovascular data, the endovascular data being derived from an intravascular device that collects data using at least one of optical coherence tomography (OCT), intravascular ultrasound, near-infrared spectroscopy, or micro-OCT.

9. The method of claim 1 , wherein the reference image further comprises a surgical guide overlay.

10. The method of claim 1 , wherein in the steps of adjusting the reference image and adjusting the live imaging feed, the user input comprises panning the live imaging feed and the reference image simultaneously.

11. 1. A system including one or more processors, The one or more processors: Access live imaging feeds of blood vessels, receiving a reference image of the blood vessel; registering the live imaging feed with the reference image such that objects depicted in a field of view of the reference image correspond to objects depicted in a field of view of the live imaging feed; receiving a user input associated with the reference image; adjusting the field of view of the reference image based on the user input; and simultaneously adjusting the field of view of the live imaging feed based on the user input to the reference image. The system is configured as follows:

12. The system of claim 11 , wherein the user input comprises a selected zoom level.

13. The one or more processors: Identifying a selected area of ​​interest on the reference image; calculating a scale factor based on a size of the object represented in the field of view of the reference image and a size of the object represented in the field of view of the live imaging feed, the field of view being associated with a grid; Identifying an origin of the grid associated with the live imaging feed; Transposing the origin onto the reference image; calculating a distance coordinate based on a distance between the area of ​​interest and the transposed origin; Transforming coordinates of the selected area of ​​interest based on the scale factor and the distance coordinates to match coordinates of the live imaging feed. The system of claim 12 further configured to:

14. The system of claim 13 , wherein the scale factor is based on the selected zoom level and is calculated as a ratio of a zoomed reference image to a non-zoomed reference image.

15. The system of claim 13 , wherein the one or more processors are further configured to simultaneously display the transformed area of ​​interest on the reference image based on the transformed coordinates and the live imaging feed.

16. The system of claim 11 , wherein the object comprises at least a portion of the blood vessel.

17. The system of claim 11 , wherein the live imaging feed comprises extravascular images derived from an angiography, fluoroscopy, x-ray, nuclear magnetic resonance, or computed assisted tomography based imaging system.

18. 12. The system of claim 11, wherein the reference image includes a static extraluminal image co-registered with endovascular data, the endovascular data being derived from an intravascular device that collects data using at least one of optical coherence tomography (OCT), intravascular ultrasound, near-infrared spectroscopy, or micro-OCT.

19. The system of claim 11 , wherein the reference image further comprises a surgical guide overlay.

20. The system of claim 11 , wherein the user input comprises panning the live imaging feed and the reference image simultaneously.

21. 1. A method for providing annotations to a live angiography feed, comprising: identifying one or more portions of a blood vessel of a patient through an intravascular imaging procedure; matching frames in the live angiography feed with frames from the intravascular imaging procedure such that both frames are captured during corresponding portions of the patient's cardiac cycle; using a rigid body registration to correct for rigid body displacements; identifying spots on an image overlay that correspond to portions of the patient's blood vessel shown in the live angiography feed and the one or more portions identified during the intravascular imaging procedure based on the matched frames and the corrected respiratory motion; The method comprising:

22. 22. The method of claim 21, wherein the frames from the intravascular imaging procedure are co-registered frames of endoluminal and extraluminal image data.

23. The method of claim 22 , wherein the endoluminal data is derived from an OCT probe or an IVUS probe.

24. 23. The method of claim 22, wherein the endoluminal data is derived from an imaging system based on angiography, fluoroscopy, x-ray, nuclear magnetic resonance, or computer-assisted tomography.

25. 22. The method of claim 21, further comprising determining the cardiac cycle of the patient using cardiovascular timing parameters, a pressure signal and heart rate.

26. The method of claim 21 , wherein the rigid registration includes considering the rigid displacement of endovascular data.

27. 27. The method of claim 26, wherein the rigid body displacement is caused by respiration of the patient, movement of the patient, movement of a machine, or movement of a camera.

28. 23. The method of claim 22, wherein matching the frames in the live angiography feed with the frames from the intravascular imaging procedure is predicted by an artificial engineering technique.

29. The method of claim 22 , further comprising placing a marker on the live angiographic feed.

30. 22. The method of claim 21, further comprising correlating coordinates of the live angiography feed with coordinates of the frames from the intravascular imaging procedure to facilitate simultaneous zoom functionality.

31. a probe configured to capture endoluminal data; an imaging device configured to capture extraluminal image data; a computing device configured to receive the endoluminal data and the extraluminal image data; comprising The computing device comprises: Identifying one or more portions of a blood vessel of a patient through an intravascular imaging procedure; Matching frames in a live angiography feed with frames from the intravascular imaging procedure such that both frames are captured during corresponding portions of the patient's cardiac cycle; Use rigid body registration to compensate for rigid body displacements, Identifying spots on an image overlay that correspond to portions of the patient's blood vessel shown in the live angiography feed and the one or more portions identified during the intravascular imaging procedure based on the matched frames and the corrected respiratory motion.

1. A data collection system configured to:

32. 32. The system of claim 31, wherein the frames from the intravascular imaging procedure are co-registered frames of the endoluminal data and the extraluminal image data.

33. 33. The system of claim 32, wherein the endoluminal data is derived from an OCT probe or an IVUS probe.

34. 33. The system of claim 32, wherein the extraluminal image data is derived from an angiography, fluoroscopy, x-ray, nuclear magnetic resonance, or computed assisted tomography based imaging system.

35. 32. The system of claim 31, wherein the computing device is further configured to determine the cardiac cycle of the patient using cardiovascular timing parameters, a pressure signal, and heart rate.

36. The system of claim 31 , wherein the rigid registration includes taking into account the rigid displacement of endovascular data.

37. 37. The system of claim 36, wherein the rigid body displacement is caused by respiration of the patient, movement of the patient, movement of a machine, or movement of a camera.

38. 33. The system of claim 32, wherein matching the frames in the live angiography feed with the frames from the intravascular imaging procedure is predicted by an artificial engineering technique.

39. 32. The system of claim 31, wherein the computing device is further configured to place a marker on the live angiography feed.

40. 32. The system of claim 31, wherein the computing device is configured to correlate coordinates of the live angiography feed with coordinates of the frames from the intravascular imaging procedure to facilitate simultaneous zoom functionality.

41. 1. A non-transitory computer-readable medium storing instructions executable by one or more processors to perform a method for providing annotations to a live angiography feed, the method comprising: Identifying one or more portions of a blood vessel of a patient through an intravascular imaging procedure; matching frames in the live angiography feed with frames from the intravascular imaging procedure such that both frames are captured during corresponding portions of the patient's cardiac cycle; Use rigid body registration to compensate for rigid body displacements, Identifying spots on an image overlay that correspond to portions of the patient's blood vessel shown in the live angiography feed and the one or more portions identified during the intravascular imaging procedure based on the matched frames and the corrected respiratory motion. A non-transitory computer-readable medium, comprising:

42. 1. A method for simultaneously zooming a live angiography feed and a reference image, comprising: using one or more processors, identifying a first origin on the live angiographic feed, the origin being a center of the live angiographic feed; identifying, using the one or more processors, a second origin on the reference image, the second origin being different from the first origin; calculating, using the one or more processors, a scale factor based on a selected zoom level; transforming, using the one or more processors, selected coordinates of the reference image using the scale factor; determining, using the one or more processors, a distance between the transformed selected coordinates and a center of the reference image; using the one or more processors, aligning coordinates of the live angiography feed with coordinates of the reference image based on the distance from the transformed selected coordinate to the center; The method comprising:

Images