Live angiography registration

Abstract

The present disclosure relates to methods of providing live x-ray images (e.g., fluoroscopic images) in conjunction with angiography frames that have been co-registered with reference information that have been subjected to real-time motion correction. In part, this disclosure describes improvements to existing workflows in which a user must remember where vascular features from OCT are and use reference frames to estimate the location of desired stent placement. Disclosed herein are methods of associating a live imaging feed and a reference image such that a user can dynamically scale and translate the live imaging feed and the reference image.

Description

[0001] CROSS-REFERENCE TO RELATED APPLICATIONS

[0002] This application claims the benefit of the filing dates of U.S. Provisional Patent Application No. 63 / 323,335, filed on March 24, 2022, and U.S. Provisional Patent Application No. 63 / 477,877, filed on December 30, 2022, entitled “Live Angioplasty,” the disclosures of which are hereby incorporated by reference herein. Background Art

[0003] The present disclosure relates generally to the field of intravascular imaging and data collection systems and methods.

[0004] Coronary artery disease is one of the leading causes of death worldwide. The ability to better diagnose, monitor, and treat coronary artery disease is important in saving lives. During some diagnostic procedures, as part of stent planning or stent delivery, various medical and technical staff members review multiple sources of information, including angiographic and fluoroscopic images. Switching between these two data sources as part of stent planning and delivery can be tiring and often requires staff members to keep images and other data associations in mind when switching from the changing fluoroscopic view to the angiographic view. Additionally, manipulating the live angiographic feed and the reference image can be cumbersome and result in inaccurate associations between the two images. Summary of the invention

[0005] The present disclosure describes a method for registering angiographic images with and without contrast agents from different time points so that features from the contrast agent image can be accurately mapped onto the non-contrast agent image. This will enable the user to directly view features and pullback features from the contrast agent image in the non-contrast agent image, thereby reducing the error of using only a reference frame.

[0006] The present disclosure relates to a method comprising: accessing a live imaging feed of a blood vessel using one or more processors, receiving a reference image of the blood vessel, aligning the live imaging feed and the reference image so that an object represented in a field of view of the reference image corresponds to an object represented in a field of view of the live imaging feed, receiving user input associated with the reference image, adjusting the field of view of the reference image based on the user input, and automatically adjusting the field of view of the live imaging feed based on the user input to the reference image. According to aspects of the present disclosure, the user input may be a selected zoom level.

[0007] According to aspects of the present disclosure, the method may also include: aligning the live imaging feed and the reference image, including: using one or more processors to identify an area of ​​interest on the reference image; calculating a scale factor based on the size of an object represented in the field of view of the reference image and the size of an object represented in the field of view of the live imaging feed, wherein the field of view of the reference image and the field of view of the live imaging feed are associated with a grid; identifying an origin of the grid associated with the live imaging feed; transforming the origin into the reference image; calculating a distance coordinate based on the distance between the area of ​​interest and the transformed origin; and transforming the coordinates of the area of ​​interest to match the coordinates of the live imaging feed based on the scale factor and the distance coordinate. The scale factor can be based on a selected zoom level and is calculated as a ratio of the scaled reference image to the unscaled reference image. According to aspects of the present disclosure, the method may also include using one or more processors to simultaneously display the transformed area of ​​interest on the reference image based on the transformed coordinates and the live imaging feed.

[0008] According to aspects of the present disclosure, the object may include at least a portion of a blood vessel. The live imaging feed may include an extravascular image obtained from an imaging system based on angiography, fluoroscopy, x-ray, magnetic resonance imaging, or computer-assisted tomography. The reference image may include a static extraluminal image that has been co-registered with intravascular data, wherein the intravascular data may be obtained from an intravascular device that collects data using at least one of optical coherence tomography (OCT), intravascular ultrasound, near infrared spectroscopy, or micro-OCT.

[0009] According to aspects of the present disclosure, the reference frame may also include a surgical guidance overlay.In adjusting the reference image and the live imaging feed, the user input may include simultaneously panning the live imaging feed and the reference image.

[0010] The present disclosure relates to a system including one or more processors configured to access a live imaging feed of a vessel, receive a reference image of the vessel, align the live imaging feed with the reference image so that objects represented in the field of view of the reference image correspond to objects represented in the field of view of the live imaging feed, receive user input associated with the reference image, adjust the field of view of the reference image based on the user input, and simultaneously adjust the field of view of the live imaging feed based on the user input to the reference image. The user input may include a selected zoom level.

[0011] According to aspects of the present disclosure, one or more processors may also be configured to: identify a selected region of interest on a reference image; calculate a scale factor based on the size of an object represented in the field of view of the reference image and the size of an object represented in the field of view of the live imaging feed, wherein the field of view of the reference image and the field of view of the live imaging feed are associated with a grid; identify an origin of the grid associated with the live imaging feed; transform the origin to the reference image; calculate a distance coordinate based on the distance between the region of interest and the transformed origin; and transform the coordinates of the selected region of interest to match the coordinates of the live imaging feed based on the scale factor and the distance coordinate. The scale factor may be based on a selected zoom level and calculated as a ratio of the scaled reference image to the unscaled reference image. According to aspects of the present disclosure, one or more processors may also be configured to simultaneously display the transformed region of interest on the reference image based on the transformed coordinates and the live imaging feed.

[0012] The present disclosure relates to methods for providing live x-ray images (e.g., fluoroscopic images) in conjunction with real-time motion-corrected angiographic frames that have been co-registered with reference information. In part, the present disclosure describes an improvement over existing workflows in which a user must remember where vascular features from OCT are and use reference frames to estimate the location of desired stent placement. A method is disclosed herein in which motion-corrected OCT information is displayed directly on top of a fluoroscopic feed to a user.

[0013] One aspect of the present disclosure relates to a method for providing annotations on a live angiography feed, the method comprising: identifying one or more portions of a patient's blood vessel through an intravascular imaging procedure; matching frames in the live angiography feed with frames from the intravascular imaging procedure so that the two types of frames are captured during corresponding portions of the patient's cardiac cycle; correcting for rigid displacement using rigid registration; and identifying an area on an image overlay based on the matched frames and the corrected respiratory motion, the area corresponding to a portion of the patient's blood vessel shown in the live angiography feed and one or more portions identified during the intravascular imaging technique.

[0014] The method may further include the frames from the intravascular imaging process being frames that co-register the intraluminal data with the extraluminal data.

[0015] The method may further include obtaining intraluminal data from an OCT probe or an IVUS probe.

[0016] The method may also include obtaining the extraluminal data from an imaging system based on angiography, fluoroscopy, x-ray, magnetic resonance imaging, or computer-assisted tomography.

[0017] The method may also include determining a cardiac cycle of the patient using the cardiovascular timing parameter, the pressure signal, and the heartbeat.

[0018] The method may further include: rigid registration includes processing rigid displacement of the intravascular data, wherein the rigid displacement is caused by patient breathing, patient movement, mechanical movement, or camera displacement.

[0019] The method may also include matching frames in the live angiography feed with frames from the intravascular imaging procedure is predicted by manual techniques.

[0020] The method may further include placing markers on the live angiography feed.The method may further include associating coordinates of the live angiography feed with frames from the intravascular imaging procedure to facilitate a synchronized zoom feature.

[0021] Another aspect of the present disclosure relates to a data collection system comprising a probe configured to capture intraluminal data, an imaging device configured to capture extraluminal image data, and a computing device configured to receive the intraluminal data and the extraluminal image data, wherein the computing device is configured to: identify one or more portions of a patient's blood vessel through an intravascular imaging process; match frames in a live angiography feed with frames from the intravascular imaging process so that both types of frames are captured during corresponding portions of the patient's cardiac cycle; correct for rigid displacement using rigid registration; and identify an area on an image overlay based on the matched frames and the corrected respiratory motion, the area corresponding to a portion of the patient's blood vessel shown in the live angiography feed and one or more portions identified during the intravascular imaging technique.

[0022] The system may further include: intraluminal data is obtained from an OCT probe or an IVUS probe.

[0023] The system may also include: the extraluminal data is obtained from an imaging system based on angiography, fluoroscopy, x-ray, magnetic resonance imaging or computer-assisted tomography.

[0024] The system may also include a computing device configured to further determine a cardiac cycle of the patient using the cardiovascular timing parameter, the pressure signal, and the heartbeat.

[0025] The system may also include: rigid registration includes processing rigid displacement of the intravascular data.

[0026] The system may also include: the rigid displacement is caused by the patient's breathing, the patient's movement, the movement of the machine, or the displacement of the camera.

[0027] The system may also include matching frames in the live angiography feed with frames from the intravascular imaging procedure is predicted by manual techniques.

[0028] The system may also include a computing device configured to further place markers on the live angiography feed. The system may also include a computing device configured to associate coordinates of the live angiography feed with frames from the intravascular imaging procedure to facilitate a synchronized zoom feature.

[0029] Another aspect of the present disclosure relates to a non-transitory computer-readable medium storing instructions executable by one or more processors to provide a method for annotation on a live angiography feed, the method comprising: identifying one or more portions of a patient's blood vessel through an intravascular imaging procedure; matching frames in the live angiography feed with frames from the intravascular imaging procedure such that the two types of frames are captured during corresponding portions of the patient's cardiac cycle; correcting for rigid displacement using rigid registration; and identifying an area on an image overlay based on the matched frames and the corrected respiratory motion, the area corresponding to a portion of the patient's blood vessel shown in the live angiography feed and the one or more portions identified during the intravascular imaging technique.

[0030] Another aspect of the present disclosure relates to a method for simultaneously scaling a live angiography feed and a reference image, the method comprising: identifying, using one or more processors, a first origin on the live angiography feed, wherein the origin is a center of the live angiography feed; identifying, using one or more processors, a second origin on the reference image, wherein the second origin is different from the first origin; calculating, using one or more processors, a scaling factor based on a selected zoom level; converting, using one or more processors, selected coordinates of the reference image using the scaling factor; determining, using one or more processors, a distance between the converted selected coordinates and a center of the reference image; and aligning, using one or more processors, the live angiography feed coordinates and the reference image coordinates based on the distance from the converted selected coordinates to the center. BRIEF DESCRIPTION OF THE DRAWINGS

[0031] Figure 1 is an exemplary system according to aspects of the present disclosure.

[0032] Figure 2 The present invention is a flowchart illustrating a correction method in image analysis according to aspects of the present disclosure.

[0033] Figure 3 An exemplary graph illustrating tip position data in accordance with aspects of the present disclosure is shown.

[0034] Figure 4 2 is an exemplary graph illustrating beats per minute (BPM) prediction data in accordance with aspects of the present disclosure.

[0035] FIG. 5A to FIG. 5BAn example of automatic angiography segmentation according to aspects of the present disclosure is illustrated.

[0036] Figure 6 An example of visualization of time-matched frames between live angiography data and contrast agent image data is shown.

[0037] Figure 7 Examples of rigid displacements according to aspects of the present disclosure are shown.

[0038] Figure 8 An example of rigid motion correction according to aspects of the present disclosure is shown.

[0039] Fig. 9 1 is a flow chart illustrating an exemplary method of stent mapping and deployment according to aspects of the present disclosure.

[0040] Fig.10 A fluoroscopic frame with an OCT pullback feature and a reference angiography frame are shown side by side.

[0041] Fig.11 is an example of a deployment screen according to aspects of the present disclosure.

[0042] Fig.12 is an example of a deployment screen according to aspects of the present disclosure. DETAILED DESCRIPTION

[0043] The present disclosure provides annotations for displaying on a live angiography feed that mark vascular features and / or locations for desired stent placement. The annotations can be identified during the intravascular imaging process (e.g., during the withdrawal of an intravascular imaging probe). Angiography images with and without contrast agents can be registered from different time points so that features from contrast agent images can be accurately mapped to images without contrast agents. Users can directly view features and withdrawal features from contrast agent images in the images without contrast agents, thereby reducing errors from using only reference frames. The present disclosure also provides methods for associating live angiography feeds and reference angiography images so that users can dynamically zoom and pan the live angiography feeds and reference angiography images.

[0044] Figure 1A data collection system 100 for use in collecting intravascular data and extravascular data is illustrated. The system may include a data collection probe 104. The data collection probe 104 may be, for example, an OCT probe, an IVUS catheter, a micro OCT probe, a near infrared spectroscopy (NIRS) sensor, or any other device that can be used to image a blood vessel 102. In some examples, the data collection probe 104 may be a pressure line, a flow meter, etc. Although the examples provided herein relate to intravascular imaging devices, such as OCT probes, the use of OCT probes is not meant to be limiting. For example, an IVUS catheter, a pressure line, or another intravascular data collection device may be used in conjunction with or in place of an OCT probe. A guidewire (not shown in the figure) may be used to introduce the probe 104 into the blood vessel 102. The probe 104 may be introduced and collect data while being withdrawn along the length of the blood vessel. The intravascular data set or image data frame may be used to identify features, such as vascular stenosis caused by plaque, calcium deposits, and the like.

[0045] OCT is a catheter-based imaging modality that uses light to peer into and generate images of the coronary artery walls for study. Using coherent light, interferometry, and micro-optics, OCT can provide video-rate in vivo tomography with micron-level resolution within diseased blood vessels. The use of fiber-optic probes to view subsurface structures at high resolution makes OCT particularly useful for minimally invasive imaging of internal tissues and organs. This level of detail possible with OCT allows the user to diagnose as well as monitor the progression of coronary artery disease. IVUS imaging uses high-frequency sound waves to create images within blood vessels.

[0046] Intravascular imaging of parts of a patient's body provides physicians and others with a useful diagnostic tool. For example, intravascular imaging of the coronary arteries can reveal the location of narrowing or stenosis. This information helps cardiologists choose between invasive coronary artery bypass surgery and less invasive catheter-based procedures such as angioplasty or stent delivery.

[0047] The probe 104 may be connected to a subsystem 108 via an optical fiber 106. The subsystem 108 may include a light source (e.g., a laser), an interferometer having a sample arm and a reference arm, various optical paths, a clock generator, a photodiode, and other OCT, IVUS, micro-OCT, NIRS, and / or pressure line components.

[0048] The probe 104 may be connected to an optical receiver 110. According to some examples, the optical receiver 110 may be a balanced photodiode based system. The optical receiver 110 may be configured to receive light collected by the probe 104. The probe 104 may be coupled to the optical receiver 110 via a wired connection or a wireless connection.

[0049] The system 100 may also include an external imaging device 120 or be configured to receive data from an external imaging device 120. For example, the external imaging device may be an imaging system based on angiography, fluoroscopy, X-ray, magnetic resonance imaging, computer-assisted tomography, etc. The external imaging device 120 may be configured to non-invasively image the blood vessel 102. According to some examples, the external imaging device 120 may obtain one or more images before, during, and / or after the retraction of the data collection probe 104.

[0050] The external imaging device 120 may communicate with the subsystem 108. According to some examples, the external imaging device 120 may be wirelessly coupled to the subsystem 108 via a communication interface such as Wi-Fi or Bluetooth. In some examples, the external imaging device 120 may communicate with the subsystem 108 via a cable such as an optical fiber. In yet another example, the external imaging device 120 may be indirectly communicatively coupled to the subsystem 108 or the computing device 112. For example, the external imaging device 120 may be coupled to a separate computing device (not shown) that communicates with the computing device 112. As another example, image data from the external imaging device 120 may be transferred to the computing device 112 using a computer-readable storage medium.

[0051] Subsystem 108 may include computing device 112. One or more steps may be performed automatically or without user input to navigate images, enter information, select and / or interact with input, etc. In some examples, one or more steps may be performed based on user input received via mouse clicks, keyboard, touch screen, verbal commands, etc.

[0052] The computing device may include one or more processors 113 , memory 114 , instructions 115 , data 116 , and one or more modules 117 .

[0053] The one or more processors 113 may be any conventional processor, such as a commercially available microprocessor. Alternatively, the one or more processors may be a dedicated device, such as an application specific integrated circuit (ASIC) or other hardware-based processor. Figure 1 The processor, memory, and other elements of device 110 are functionally described as being in the same block, but one of ordinary skill in the art will appreciate that a processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be stored in the same physical housing. Similarly, the memory may be a hard disk or other storage medium located in a housing different from the housing of device 112. Thus, references to a processor or computing device will be understood to include references to a collection of processors or computing devices or memories that may or may not operate in parallel.

[0054] The memory 114 may store information accessible to the processor, including instructions 115 that may be executed by the processor 113, and data 116. The memory 114 may be a type of memory operable to store information accessible to the processor 113, including a non-transitory computer-readable medium, or other medium storing data that may be read with the aid of an electronic device, such as a hard disk, a memory card, a read-only memory ("ROM"), a random access memory ("RAM"), an optical disk, and other writable and read-only memories. The subject matter disclosed herein may include different combinations of the foregoing, whereby different portions of the instructions 115 and data 116 are stored on different types of media.

[0055] Memory 114 can be retrieved, stored or modified by processor 113 according to instruction 115. For example, although the present disclosure is not limited to a specific data structure, data 116 can be stored in a computer register, in a relational database as a table with multiple different fields and records, an XML document or a flat file. Data 116 can also be formatted in a computer-readable format (for example, but not limited to binary values, ASCII or Unicode). As a further example only, data 116 can be stored as a bitmap composed of pixels, which is stored in a compressed or uncompressed manner, or stored as various image formats (for example, JPEG), vector-based formats (for example, SVG) or computer instructions for drawing graphics. In addition, data 116 can include information sufficient to identify relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations), or information used by functions to calculate relevant data.

[0056] Instructions 115 may be any set of instructions (e.g., machine code) directly executed by processor 113 or instructions (e.g., scripts) executed indirectly. In this regard, the terms "instructions," "applications," "steps," and "programs" may be used interchangeably herein. Instructions may be stored in an object code format for direct processing by a processor, or in any other computing device language, including scripts or collections of independent source code modules that are interpreted on demand or pre-compiled. The functions, methods, and routines of the instructions will be described in more detail below.

[0057] Module 117 may include a co-registration module and a live angiography co-registration module. According to some examples, these modules may include software (e.g., pre-processing software), transformations, matrices, and other software-based components for processing image data or responding to patient triggers to facilitate co-registration of different types of image data by other software-based components, or otherwise performing such co-registration. These modules may include lumen detection using scan-line-based or image-based methods, stent detection using scan-line-based or image-based methods, indicator generation, stent expansion assessment and evaluation, stent landing zone detection, and indication for deployed stents; angiography and intravascular imaging co-registration; and other modules that support and are programmed to perform the methods disclosed herein.

[0058] The computing device 112 may be adapted to co-register the intravascular data with the luminal image. For example, the computing device 112 may access a co-registration module to co-register the intravascular data with the luminal image. The luminal image may be an extraluminal image, such as an angiogram, x-ray, or similar image. The co-registration module may co-register the intravascular data (e.g., an intravascular image, pressure readings, virtual flow reserve (VFR), fractional flow reserve (FFR), resting full circulation ratio (RFR), flow velocity, etc.) with the extraluminal image. In some examples, the co-registration module may co-register the intravascular data with an intraluminal image (e.g., an intraluminal image captured by an OCT probe, an IVUS probe, a micro-OCT probe, or a link).

[0059] In one example, the co-registration module can co-register the intraluminal data captured during the withdrawal with one or more extraluminal images. For example, the extraluminal image frame can be pre-processed. Various matrices such as convolution matrices, Hessian matrices, etc. can be applied on a per-pixel basis to change the intensity, remove or otherwise modify a given angiographic image frame. As described herein, the pre-processing stage can enhance, modify and / or remove features of the extraluminal image to increase the accuracy, processing speed, success rate and other characteristics of the subsequent processing stage. The blood vessel centerline can be determined and / or calculated. In some examples, the blood vessel centerline can be superimposed or otherwise displayed relative to the pre-processed extraluminal image. According to some examples, the blood vessel centerline can represent the trajectory of the collection probe 104 retracting through the blood vessel during the withdrawal. In some examples, the centerline can be referred to as a trace. Additionally or alternatively, a marker band or a marker that cannot be transparent to radiation can also be detected in the extraluminal image frame. According to some examples, the extraluminal image frame can be co-registered with the data received by the collection probe 104 based on the position of the determined marker band.

[0060] The system may include a monitor 122. The monitor 122 may include, but is not limited to, an electrocardiogram monitor configured to generate data related to cardiac function and displaying various states of the subject, such as systole and diastole. Knowing the cardiac phase can be used to assist in tracking the centerline of a vessel, because the geometry of the heart, including the coronary arteries, is substantially the same at a particular cardiac phase, even during different cardiac cycles. In part, the present disclosure relates to systems and methods for identifying angiographic frames corresponding to portions of the cardiac cycle using cardiovascular system timing parameters and / or signals, such as an electrocardiogram (ECG) and pressure signals (e.g., an aortic pressure signal). Bimodal notches and other timing markers, such as those used to identify systole and diastole, may be used. In addition, the disclosed systems and methods may identify angiographic frames corresponding to a current fluoroscopic image based on real-time association with such signals and / or timing parameters.

[0061] Figure 2 is a flow chart showing motion correction to be applied to the co-registered images. Specifically, the motion correction module may account for cardiac motion, respiratory motion, body movement, etc. before overlaying the co-registered images on the live angiography feed. Motion correction may be applied to contrast agent image features to accurately overlay the contrast agent image features on top of the live angiography feed.

[0062] According to some examples, for each live fluoroscopic frame, a corresponding angiographic frame is identified based on signal or timing parameter association (e.g., by ECG association). Additionally, in some embodiments, as described above, an intravascular probe marker (e.g., a radiopaque marker) is used under angiography and associated with it as part of an intravascular imaging session. In this way, intravascular imaging facilitates angiographic association.

[0063] In the first stage, cardiac time matching 201 is performed. During cardiac time matching 201, cardiac motion is removed. Cardiac motion is related to the movement of the heart as it beats as displayed on the image. In this stage, cardiac motion is removed from the co-registered images of the angiography. In the second stage, rigid registration 202 is performed. During rigid registration 202, rigid motion is removed. Rigid motion is related to respiratory motion (such as expansion and contraction of the lungs during imaging) and patient movement. Additionally, any displacement of the camera over time can be approximated as rigid motion. In the third stage, after the angiography co-registered images are corrected for both cardiac activity and respiratory / displacement motion in each frame, the stent docking area and intravascular imaging pullback features can be mapped from the contrast agent angiography images to the live angiography video feed. Each of these stages will be described in turn below.

[0064] Due to the 2D projection of 3D vessels in angiography, cardiac motion is deformable.To correct for cardiac motion, at a particular moment during display of a live angiography feed, a contrast agent image is identified that is in the same phase of the cardiac cycle as the angiography frame being displayed.

[0065] Correcting for cardiac motion may include predicting the patient's beats per minute (BPM). The BPM will allow for finding an angiography frame from the intravascular imaging pullback that matches the current non-contrast fluoroscopic frame as it progresses.

[0066] At least two modes can be used to estimate BPM. In some examples, the mode can be an imaging mode, such as OCT, angiography, IVUS, NIRS, μOCT, etc. For both modes, the BPM prediction algorithm is based on a customized autocorrelation value within a limited range of window sizes.

[0067] Figure 3 and Figure 4 Various examples of determining cycle length or BPM or patient are illustrated. In some examples, the second mode may be aortic (AO) pressure data. The AO pressure may have a cycle defined over the course of a heartbeat. In some examples, the AO pressure may be used to determine or predict the patient's BPM. Figure 3 and Figure 4 As shown, the BPM can be determined based on points located at the same or substantially the same location in each cycle. AO pressure data or other timing or signal data (e.g., ECG) can be used to determine which portion of the cardiac cycle corresponds to a given AO pressure data. According to some examples, the AO pressure data can correspond to identifying the systolic and diastolic phases of the cardiac cycle or other trackable time periods relative to the cardiac cycle.

[0068] According to some examples, the first mode may be angiography video. According to some examples, the characteristics of the blood vessels and / or the characteristics of the imaging system may be shown in the contrast agent angiography image 501 ( Figure 5B ) and the non-contrast angiography image 502 ( Figure 5A ) are visible in both. Features of the imaging system can be, for example, markings on the cable tip 503 and / or the cable 504. The computing device can be configured to detect features or segment images of arteries or other structures into various component tissue types or regions of interest through an artificial intelligence (AI) system such as one or more machine learning models. In some examples, the machine learning model can be used to detect features in images without contrast agents. The machine learning model can be trained using raw angiographic images. In some examples, the machine learning model can be trained using labeled image training data (e.g., angiographic images with and without contrast agents).

[0069] The AI ​​system receives a plurality of training images, each training image annotated with the location of one or more features of the imaging system. The AI ​​system may receive training image data labeled with the location of features of the imaging system detected in each image. As part of receiving the plurality of training images, the system may separate the data into a plurality of sets, such as image frames for training, image frames for testing, and image frames for validation.

[0070] The AI ​​system processes multiple training images to annotate each image. The AI ​​system trains a machine learning model using the processed multiple training images. In some examples, the model is configured to generate as an output a segmentation map representing one or more predicted regions of interest (including portions of the input image predicted to be features of the imaging system).

[0071] The machine learning model can be trained according to any technique for supervised learning, and in general, the training techniques use a dataset in which at least some of the training examples are labeled. For example, the machine learning model can be one or more neural networks with model parameter values ​​that are updated as part of the training process using back-propagation of gradient descent, such as on a single image frame or on a batch of image frames.

[0072] In some examples, the machine learning model includes, at least in part, an encoder-decoder architecture with skip connections. As another example, the machine learning model can include one or more neural network layers as part of an autoencoder that is trained to learn a compact (encoded) representation of an image from unlabeled training data, such as images taken using OCT, IVUS, NIRS, OCT-NIRS, μOCT, or any other of a variety of other imaging techniques. The neural network layers can be further trained on input training images as described herein, and the machine learning model can benefit from a wider training set by being trained, at least in part, using unlabeled training images.

[0073] Training can be accomplished using a loss function that quantifies the difference between the location of a feature of the imaging system predicted by the system and the true location of the feature of the imaging system received as part of the training data for the input image. For example, the loss function can be a distance measurement between the predicted and true locations of the feature of the imaging system at one or more pairs of points on the predicted and true locations. In general, any loss function that compares each pixel between a training image with predicted annotations and a training image with true annotations can be used.

[0074] The system can perform training until it determines that one or more stopping criteria have been met. For example, the stopping criteria can be a preset number of epochs, a minimum improvement of the system between epochs as measured using a loss function, a predetermined amount of wall clock time has passed, and / or until a computational budget is exhausted, such as a predetermined number of processing cycles.

[0075] Existing models can be enhanced with existing machine learning models that are trained to predict non-lipid regions of interest and also to predict the location of features of an imaging system as one of a plurality of different output channels. The machine learning model can be trained to output a plurality of channels, each channel corresponding to a respective region of interest, e.g., the regions of interest can be represented as one or more output images and / or as segmentation maps or other data that characterize regions of an input image as corresponding to different regions of interest.

[0076] In part, the AI ​​system is designed so that it can be installed or combined with an imaging system, such as an intravascular imaging system, an ultrasound system, or an x-ray system (e.g., an angiography system or a fluoroscopy system). For example, the position of a given feature (e.g., the cable tip 503) can be plotted over time. By plotting the x- and y-positions of these features over time, the patient's BPM over time can be estimated.

[0077] Figure 5A and Figure 5B An example of automatic angiography segmentation is depicted. As mentioned above, Figure 5A Features detected by a deep learning model in a contrast-free image 502 are illustrated. For example, the detected features may be a cable tip 503 and a marker location 504. Figure 5B Illustrated are features detected by a deep learning model in a contrast agent image 501. These features may be cable tips 503, for example.

[0078] According to some examples, automatic angiography segmentation can be applied. For example, features detected by the machine learning model can be displayed in the contrast-free image 502. For example, such features can be displayed using a mask. As an example, different portions of a vessel can be identified using a mask with different colors, where each color corresponds to a segmented portion of the vessel. It should be understood that other types of masks can be applied to indicate any of a number of different features.

[0079] Once the AO pressure data or other timing data or signals are mapped to the cardiac cycle or other clock or timing subsystem, the data of interest can be co-registered with pre-computed angiography data that has been co-registered with the OCT / IVUS markers 504. In general, the systems and methods identify the cardiac cycle that tracks or corresponds to a particular frame of image data or other parameter of interest.

[0080] Figure 6 An example of a phase matched image frame 603 is illustrated. The phase matched image frame 603 may include a matched live angiography image frame 601 and a contrast agent image frame 602. The live angiography image frame 601 may correspond to or substantially correspond to the contrast agent angiography image frame 602. According to some examples, the contrast agent angiography image frame 602 may include intravascular data obtained during the co-registration of the angiography image frame with the intravascular image frame. The cardiac cycle may contain repeating patterns, such as repeated pressure rises, bimodal notches, and ECG pulses. These features of the live cardiac cycle data may be associated or matched with the first set of cardiac cycle data.

[0081] Figure 7 Rigid displacement is illustrated, wherein the position or placement of the guide cable 701 path and catheter 702 on the image is not aligned with the position of the blood vessel 703 in the angiographic image. Rigid displacement may be caused by, for example, rigid motion. Specifically, rigid displacement may be caused by rigid motion, such as respiratory motion (e.g., expansion and contraction of the lungs), patient movement during imaging, and camera shift over time. To handle rigid displacement, rigid transformation techniques can be used to compensate or correct for this type of motion. Features visible in both the contrast angiographic image and the non-contrast angiographic image are used in the rigid registration 202 step of motion correction.

[0082] Figure 8 Angiographic images after a rigid transformation has been applied are illustrated. For example, images 802A and 804A illustrate angiographic images without contrast agent overlaid with a centerline. For example, the centerline can be determined using angiographic images taken with contrast agent. As shown, the centerline in each image 802A, 804A is offset from the blood vessel. The system can apply a rigid transformation technique to adjust the positioning of the blood vessel centerline based on the determined rigid displacement. As shown in images 802B, 804B, after the system applies the rigid transformation technique, the centerline can be aligned or substantially aligned with the blood vessel in the image.

[0083] In one embodiment, the co-registered angiographic and intravascular images can be combined or co-registered with a live angiographic image or video such that intravascular markers, intravascular image data, intravascular data, and / or other parameters can be overlaid on the live angiographic image or video. Stent detection, stent dilatation, side branching, and other image data detected based on the collected intravascular data set can be linked to or displayed relative to the live angiographic and angiographic images as a result of the co-registration between and among the three image data sets.

[0084] These different data sets, extraluminal imaging, intravascular imaging, and live angiography, can be combined, interwoven, used, juxtaposed, or integrated in various ways to present a combination of co-registered data to an end user. Angiographic images with and without contrast agents can be registered from different time points so that features from the contrast agent images can be accurately mapped onto the non-contrast agent images. In this regard, diagnostic imaging can be performed using less contrast agent solution without sacrificing accuracy or image quality.

[0085] There is great value if angiographic images, fluoroscopic data, intravascular data, and / or other imaging modalities can be used on an accelerated time scale and in a manner that is helpful to the physician to support cardiovascular diagnosis and stenosis treatment. Addressing the various competing obstacles to achieving these goals represents a significant technical challenge. In various embodiments, co-registered data and signals (e.g., AO, ECG, systolic transition, diastolic transition) and other uses and generation of various interleaved, static, combined, and fused data sets can be used, including streams of data having a subset of live frames and a subset of stored, historical, or non-live frames.

[0086] In part, the present disclosure relates to systems and methods for overlaying angiographic co-registered images and / or data onto live angiography. In addition, the present disclosure relates to systems and methods for combining angiographic co-registration information with one or more live angiographic feeds. Directly viewing features and pullback features from contrast images in a live angiographic feed without contrast will reduce errors introduced by using only a reference frame.

[0087] Given that various data collection and diagnostic systems (e.g., angiography systems using OCT, IVUS) or other intravascular imaging systems use imaging probes with radiopaque markers to track a given probe, the angiographic data can be co-registered with the intravascular data. Both data sets can be co-registered, correlated, or cross-correlated with the live angiographic feed using pressure signals or other pressure signals used to monitor aortic pressure, EKG signals, bimodal notch signals and position and other timing signals.

[0088] After the first and second phases are completed, an accurate mapping landing zone can be identified. For example, the mapping can be an overlay for a live angiography feed. Accordingly, an accurate landing zone can be a landing zone where a marker corresponds to a correct location on the patient's blood vessel as depicted in the live angiography feed. Although cardiac timing matching and motion correction are described above as occurring in different phases, in other examples, cardiac timing matching and motion correction can be performed in a different order, at overlapping times, or simultaneously.

[0089] After correcting the co-registered angiographic images for both cardiac activity and respiratory / displacement motion in each frame, the stent docking zone as well as the intravascular imaging pullback features from the contrast angiographic images can be mapped onto the live angiographic video feed, as shown in FIG. Fig. 9 as shown in .

[0090] Fig. 9 Exemplary methods of co-registering intravascular images and / or image data with extraluminal images and then co-registering or pairing the co-registered images with a live angiography feed are illustrated.

[0091] In block 910, intravascular image data and extravascular image data are simultaneously acquired. The intravascular image data may be acquired during the pullback. The intravascular image data may be OCT, IVUS, NIRS, etc. The extravascular image data may be acquired before, during, and / or after the pullback. The extraluminal image data may be an angiographic image. Both data sets are generated, transmitted, processed, and stored in a central computer. The images may be co-registered based on markers or common reference points.

[0092] In box 920, the intravascular image data can be co-registered with the extraluminal image data. After data acquisition, it is determined which angiographic frame represents the reference point closest to the initial OCT frame. This determination can also be made by a human user or an automated computer routine. The human user or a computer algorithm can examine the angiographic data and estimate which frame corresponds to the initial frame of the withdrawal. For example, the computing device 112 may include one or more modules configured to automatically run and capture angiographic images, and mark each image by its acquisition time, and support co-registration of intravascular image data marked with acquisition time. In addition, the computing device 112 may include one or more modules configured to automatically detect a radiopaque marker on each angiographic image corresponding to the intravascular image. The co-registration module calculates the path of the intravascular imaging catheter on all angiographic images corresponding to the intravascular image acquisition during the withdrawal of the probe through the imaged vessel.

[0093] In one embodiment, side branches detected in an OCT frame of the data using OCT image processing can be used as input to improve co-registration with the angiography data. For example, in one embodiment, each OCT frame includes a flag (yes / no) indicating whether a side branch is present. In addition, once co-registration is achieved, the location of stents, calcium deposits, lipid deposits, thrombi, thin capped fibroatheromas (TCFA or "vulnerable plaques"), vessel normalization, side branch detection, FFR values ​​(which can be calculated as vascular resistance ratio (VRR) values ​​based on the OCT image data), lumen size values, stents, and various other data described herein can be overlaid on the angiography image or OCT image based on the co-registration between the data sets.

[0094] In block 930, the system can perform a morphological assessment. In some examples, the morphological assessment can be cardiac motion and / or rigid displacement. To account for any identified cardiac motion, the system can time-match the image frames. In some examples, to account for any identified rigid displacement, the system can apply a rigid transformation technique.

[0095] In box 940, the system can pair the co-registered images and the corrected images with live angiographic images and / or videos. For example, the corrected images can be co-registered intraluminal image data and extraluminal image data that have been time-matched and / or to which rigid displacement techniques have been applied. In some examples, the co-registered images and the corrected images can be paired with a fluoroscopic video. Using the intravascular image dataset and the extravascular image dataset stored on a storage device, the computing device can pair the co-registered images with the live fluoroscopic video using the same or similar reference points marked during co-registration. For example, the co-registered images and the live fluoroscopic video are paired using reference points (e.g., distance from a point of interest, time of capture, and / or markings, etc.).

[0096] In block 950, the system can output the corrected image overlaid on the live angiography image and / or fluoroscopic video. According to some examples, the output can include annotations shown associated with the blood vessels displayed in the live angiography feed. The annotations can be aligned or substantially aligned with locations identified during a previous intravascular imaging procedure for stent placement or other treatment.

[0097] The above process is very useful when co-registering live angiography or fluoroscopy data. Specifically, real-time association is used, which effectively identifies parts of the cardiac cycle that can be tracked over time relative to AO pressure data or other data or signals (e.g., ECG data for angiography data), and corrects rigid motion. Therefore, this facilitates selecting any frame from a set of angiography image frames and replacing such angiography frames with live angiography frames. Bracket mapping or replacement can be used, or picture-in-picture representations and other representations can be used. Various interleaving techniques can be used. This frame exchange supports the use of a library of images from angiography previously generated using contrast agents.

[0098] In one embodiment, during intravascular pullback, the image frame is dark under the angiography. There are discrete time periods for pullback, which can be tracked and indexed relative to the angiography frame and can be registered relative to AO pressure data, ECG data, bimodal notch or other timing data. In one embodiment, each subsequent interleaving of frames can be performed using AO data or other timing data. One or more displays can be used to show live angiography data by extracting frames from a live feed using a frame grabber and interleaving angiography frames or combining with intravascular data. Intravascular imaging markers (e.g., OCT / IVUS markers and other information) can be combined, superimposed or otherwise used with live angiography feeds. This fusion or combination of various types of image data with live angiography data can be used to support various results, such as knowing exactly where the stent docking area is located.

[0099] Fig.10 An example of overlaying annotations from an intravascular imaging procedure onto a live angiography feed is illustrated.According to aspects of the above disclosure, data from co-registered images may be overlaid on a live angiography image.

[0100] In this particular example, for example, Fig.10 A live angiography image 1001 is shown with stent planning markers 1002, 1003, and 1004 already overlaid on the angiography image. The stent planning markers 1002, 1003, 1004 may correspond to annotations made by a physician during an intravascular imaging procedure. For example, an intravascular imaging procedure may be performed using OCT, IVUS, or other types of probes. The procedure may identify portions of a patient's blood vessels that are narrowed due to plaque, calcium deposits, and the like.

[0101] These stent planning markers 1002 to 1004 can be based on markers from OCT pullback mapped onto angiographic images. The relative positions of these markers will move from one angiographic image to another because the patient's cardiac cycle will cause the various arteries within the heart to move. However, by associating both the intravascular image and the angiographic image with the patient's cardiac cycle and rigid motion, the stent planning markers 1002 to 1004 can be overlaid on each angiographic image in a manner that maintains their proper position relative to the artery where the stent is to be placed. Therefore, the system provides for more accurate placement of stent planning markers from intravascular imaging technology onto live angiographic feeds. These markers and other identifications help reduce geographic misalignment in processes such as stent delivery and placement by ensuring that physicians aim at the area they plan to treat during intravascular imaging.

[0102] Fig.11 A sample deployment screen with grid coordinates overlaid on the screen is illustrated. The left screen shows a live angiography feed 1110 of X-ray fluoroscopy. The right screen shows a reference angiography image 1120 with stent planning marked between brackets 1121. Both images 1110 and 1120 reflect the field of view of the imaging device. Objects in the two fields of view should correspond to each other. For example, objects displayed in the live angiography feed should correspond to objects displayed in the reference image. In some examples, the objects represented are portions of blood vessels.

[0103] The system can place a grid layer over the displayed image. In some examples, the grid can be internal or visible to the user. The grid layer can be associated with the field of view of each image so that when the image changes based on the imaging device position, the grid layer remains constant. The grid layer can create coordinates for the image. Each grid layer can include an origin from which the system will calculate the coordinates.

[0104] The square portion on the display highlights the area of ​​interest (AOI) 1130-1131 selected by the user. The reference angiogram image 1120 with the stent plan 1121 overlay can be used as a guide to help accurately position the device. The reference angiogram image 1120 can be co-registered as described above. In some examples, the grid overlay is not visible to the user, but is only used internally by the system.

[0105] The system can be equipped with a feature that can allow a user to zoom and pan the reference angiography image. The zoom feature can zoom and pan the reference angiography image 1120 and the live angiography feed 1110 simultaneously. This will provide the user with the ability to zoom in on the AOI in the live angiography feed 1110 when deploying an intravascular device (e.g., an interventional device). The system can be equipped with a processor with code that runs the necessary steps to facilitate the zoom function. In some examples, the grid layer can be adjusted based on the selection of the zoom feature. For example, if the user chooses to zoom in on the image by 2 times, the grid layer will also be enlarged to reflect the size of the image.

[0106] The system can be configured to associate two images by transforming the coordinates of the first image to match the coordinates of the second image. For example, to associate the live angiography feed 1110 with the reference angiography image 1120, the system can transform the coordinates of the reference angiography image 1120 to match the coordinates of the live angiography feed 1110. Transforming the coordinates can include determining an origin as the base coordinate for each image, or 0,0. In some examples, the images can have different origins such that the system must account for the different origins when associating the two images. Fig.11 In the example shown, the first origin 1141 or 0,0 of the live angiography feed 1110 is located at the center of the image. The origin 1140 or 0,0 of the reference angiography image 1120 is at the upper left corner of the image. It should be understood that in other examples, the origin for each image can be at a different position than the origin 1141 or 0,0. Fig.11 In different positions shown.

[0107] In some examples, the system is configured to receive a zoom input from a user. The zoom input can be the degree to which the user wants to enlarge or reduce the image, such as a magnification factor. For example, the user can select the AOI and decide to enlarge the image and video screen by 3 times. The system can use the zoom input to calculate a scale factor. The scale factor can be calculated using the ratio between the new image and the original image. For example, for a 3x zoom input, the ratio is 3:1 and the scale factor is 3.

[0108] The user can select an area within any image to examine in detail, such as AOI 1131 of reference angiography image 1120. The display can zoom in on the area around the selected point, such as a zoomed-in image of the AOI. In some examples, the system can select two points within the area selected by the user, such as the upper left corner coordinates and the lower right corner coordinates of the AOI. The selected points can be converted using a determined scale factor.

[0109] The selected point can be further converted from its absolute coordinates to distance coordinates. When the system associates the first image with the second image, the absolute coordinates of the first image can be calculated relative to the origin of the first image (e.g., the second origin 1140 of the reference angiography image 1120). By determining the distance from each absolute coordinate to the origin of the second image, the distance coordinates of the first image can be calculated so that the distance coordinates of the first image and the original coordinates of the second image match. For example, when the first image is a reference image and the second image is a live angiography feed and the origin is located at the center of the live angiography feed, the distance coordinates are calculated from the absolute coordinates of the reference image to the center of the reference image. Specifically, in the reference angiography image 1120, the absolute coordinates of the upper left coordinates of the AOI 1131 can be understood as -3, -4, and the distance coordinates of the upper left coordinates of the AOI 1131 can be understood as 10, -9. The distance coordinates can be used to convert the remaining coordinates of the AOI. The system can determine the remaining coordinates based on the absolute coordinates of the remaining coordinates, the size and shape of the AOI, etc. For example, if the upper left corner is determined to be 10, -9, the lower right coordinate of the AOI may be determined to be 0, -2.

[0110] Once the scale factor has been applied and the distance coordinates determined, the system can relate the distance coordinates of the reference angiography image to those of the live angiography feed so that the coordinates are equal.The system can manipulate the reference angiography image 1120 and the live angiography feed 1110 simultaneously.

[0111] Fig.12 A sample deployment screen is depicted with grid coordinates superimposed on the screen. The left screen shows a live angiography feed 1210 of x-ray fluoroscopy. The right screen shows a reference angiography image 1220 with the stent plan marked between brackets 1221. The depiction shows a deployment screen where both the live angiography feed 1210 and the reference angiography image 1220 have been simultaneously zoomed into the AOI. As described above, after converting the coordinates of the reference angiography image 1220, the system may be able to manipulate the live angiography feed 1210 and the reference angiography image 1220 simultaneously. Manipulation may be input into the system by a user, such as zooming and panning.

[0112] Although the present invention has been described herein with reference to specific embodiments, it should be understood that these embodiments are merely illustrative of the principles and applications of the present invention. Therefore, it should be understood that various modifications may be made to the exemplary embodiments without departing from the spirit and scope of the present invention as defined in the appended claims, and other arrangements may be conceivable.

Claims

1. A method comprising: accessing a live imaging feed of the blood vessel using one or more processors; receiving, using the one or more processors, a reference image of the blood vessel; aligning, using the one or more processors, the live imaging feed and the reference image so that objects represented in a field of view of the reference image correspond to objects represented in a field of view of the live imaging feed; receiving user input associated with the reference image; adjusting, using the one or more processors, a field of view of the reference image based on the user input; as well as A field of view of the live imaging feed is automatically adjusted based on the user input to the reference image using the one or more processors.

2. The method according to claim 1, wherein: The user input includes a selected zoom level.

3. The method according to claim 2, wherein: Aligning the live imaging feed with the reference image further comprises: identifying, using one or more processors, a region of interest on the reference image; calculating, using one or more processors, a scale factor based on a size of an object represented in a field of view of the reference image and a size of an object represented in a field of view of the live imaging feed, wherein the field of view of the reference image and the field of view of the live imaging feed are associated with a grid; identifying, using one or more processors, an origin of the grid associated with the live imaging feed; transforming the origin onto the reference image using one or more processors; calculating, using one or more processors, a distance coordinate based on the distance between the region of interest and the transformed origin; and The coordinates of the region of interest are converted, using one or more processors, based on the scale factor and the range coordinates to match the coordinates of the live imaging feed.

4. The method according to claim 3, wherein: The scale factor is based on the selected zoom level and is calculated as a ratio of a scaled reference image to an unscaled reference image. 5 . The method of claim 3 , further comprising simultaneously displaying a transformed region of interest on the reference image based on the transformed coordinates and the live imaging feed using one or more processors.

6. The method according to claim 1, wherein: The object includes at least a portion of a blood vessel.

7. The method according to claim 1, wherein: The live imaging feed includes extravascular images obtained from an imaging system based on angiography, fluoroscopy, x-ray, magnetic resonance imaging, or computer-assisted tomography.

8. The method according to claim 1, wherein: The reference image comprises a stationary extraluminal image co-registered with intravascular data, wherein the intravascular data is obtained from an intravascular device that collects data using at least one of optical coherence tomography (OCT), intravascular ultrasound, near infrared spectroscopy, or micro-OCT.

9. The method according to claim 1, wherein: The reference image also includes a surgical guidance overlay.

10. The method according to claim 1, wherein: In adjusting the reference image and the live imaging feed, the user input includes simultaneously panning the live imaging feed and the reference image.

11. A system comprising: One or more processors configured to: access to live imaging feeds of blood vessels; receiving a reference image of the blood vessel; aligning the live imaging feed and the reference image so that objects represented in the field of view of the reference image correspond to objects represented in the field of view of the live imaging feed; receiving user input associated with the reference image; adjusting the field of view of the reference image based on the user input; as well as A field of view of the live imaging feed is simultaneously adjusted based on the user input to the reference image.

12. The system according to claim 11, wherein: The user input includes a selected zoom level.

13. The system according to claim 12, wherein: The one or more processors are further configured to: identifying a selected region of interest on the reference image; calculating a scale factor based on a size of an object represented in a field of view of the reference image and a size of an object represented in a field of view of the live imaging feed, wherein the field of view of the reference image and the field of view of the live imaging feed are associated with a grid; identifying an origin of the grid associated with the live imaging feed; Transforming the origin onto the reference image; Calculating a distance coordinate based on the distance between the region of interest and the transformed origin; and The coordinates of the selected region of interest are transformed based on the scale factor and the range coordinates to match the coordinates of the live imaging feed.

14. The system according to claim 13, wherein: The scaling factor is a scaling factor based on the selected zoom level and calculated as a ratio of a scaled reference image to an unscaled reference image.

15. The system of claim 13, wherein: The one or more processors are further configured to simultaneously display a transformed region of interest on the reference image based on the transformed coordinates and the live imaging feed.

16. The system according to claim 11, wherein: The object includes at least a portion of a blood vessel.

17. The system of claim 11, wherein: The live imaging feed includes extravascular images obtained from an imaging system based on angiography, fluoroscopy, x-ray, magnetic resonance imaging, or computer-assisted tomography.

18. The system of claim 11, wherein: The reference image comprises a stationary extraluminal image co-registered with intravascular data, wherein the intravascular data is obtained from an intravascular device that collects data using at least one of optical coherence tomography (OCT), intravascular ultrasound, near infrared spectroscopy, or micro-OCT.

19. The system of claim 11, wherein: The reference image also includes a surgical guidance overlay.

20. The system of claim 11, wherein: The user input comprises simultaneously panning the live imaging feed and the reference image.

21. A method of providing annotations on a live angiography feed, the method comprising: identifying one or more portions of a patient's blood vessels through an intravascular imaging procedure; matching frames in the live angiography feed with frames from the intravascular imaging procedure such that both types of frames are captured during corresponding portions of the patient's cardiac cycle; Correction of rigid displacements using rigid registration; as well as An area on the image overlay is identified based on the matched frames and the corrected respiratory motion, the area corresponding to a portion of the patient's blood vessel shown in the live angiography feed and the one or more portions identified during the intravascular imaging technique.

22. The method according to claim 21, wherein: The frames from the intravascular imaging process are frames in which the intraluminal data and the extraluminal image data are co-registered.

23. The method according to claim 22, wherein: The intraluminal data are obtained from an OCT probe or an IVUS probe.

24. The method according to claim 22, wherein: The extraluminal data is obtained from angiography, fluoroscopy, x-ray, magnetic resonance imaging or computer-assisted tomography based imaging systems.

25. The method of claim 21, further comprising determining the cardiac cycle of the patient using cardiovascular timing parameters, a pressure signal, and a heartbeat.

26. The method according to claim 21, wherein: The rigid registration includes processing rigid displacements of the intravascular data.

27. The method according to claim 26, wherein: The rigid displacement is caused by the patient's breathing, the patient's movement, the movement of the machine or the displacement of the camera.

28. The method of claim 22, wherein: Matching frames in the live angiography feed with frames from the intravascular imaging procedure is predicted by manual techniques.

29. The method of claim 22, further comprising: Markers are placed on the live angiography feed.

30. The method of claim 21, further comprising: Coordinates of the live angiography feed are associated with frames from the intravascular imaging procedure to facilitate a synchronized zoom feature.

31. A data collection system comprising: a probe configured to capture intraluminal data; an imaging device configured to capture extraluminal image data; a computing device configured to receive the intraluminal data and the extraluminal data; Wherein, the computing device is configured to: identifying one or more portions of a patient's blood vessels through an intravascular imaging procedure; matching frames in the live angiography feed with frames from the intravascular imaging procedure such that both types of frames are captured during corresponding portions of the patient's cardiac cycle; Correction of rigid displacements using rigid registration; as well as A region on the image overlay is identified based on the matched frames and the corrected respiratory motion, the region corresponding to a portion of a blood vessel of the patient shown in the live angiography feed and one or more portions identified during the intravascular imaging technique.

32. The system of claim 31, wherein: The frames from the intravascular imaging process are frames in which the intraluminal data and the extraluminal data are co-registered.

33. The system of claim 32, wherein: The intraluminal data are obtained from an OCT probe or an IVUS probe.

34. The system of claim 32, wherein: The extraluminal data is obtained from angiography, fluoroscopy, x-ray, magnetic resonance imaging or computer-assisted tomography based imaging systems.

35. The system of claim 31, wherein: The computing device is configured to further determine the cardiac cycle of the patient using cardiovascular timing parameters, a pressure signal, and a heartbeat.

36. The system of claim 31, wherein: The rigid registration includes processing rigid displacements of the intravascular data.

37. The system of claim 36, wherein: The rigid displacement is caused by the patient's breathing, the patient's movement, the movement of the machine or the displacement of the camera.

38. The system of claim 32, wherein: Matching frames in the live angiography feed with frames from the intravascular imaging procedure is predicted by manual techniques.

39. The system of claim 31, wherein: The computing device is configured to further place markers on the live angiography feed.

40. The system of claim 31, wherein: The computing device is configured to correlate coordinates of the live angiography feed with frames from the intravascular imaging procedure to facilitate a synchronized zoom feature.

41. A non-transitory computer readable medium storing instructions executable by one or more processors to provide a method of providing annotations on a live angiographic feed, the method comprising: identifying one or more portions of a patient's blood vessels through an intravascular imaging procedure; matching frames in the live angiography feed with frames from the intravascular imaging procedure such that both types of frames are captured during corresponding portions of the patient's cardiac cycle; Use rigid registration to correct for rigid displacements; as well as An area on the image overlay is identified based on the matched frames and the corrected respiratory motion, the area corresponding to a portion of the patient's blood vessel shown in the live angiography feed and the one or more portions identified during the intravascular imaging technique.

42. A method for simultaneously scaling a live angiography feed and a reference image, the method comprising: identifying, using one or more processors, a first origin on the live angiography feed, wherein the origin is a center of the live angiography feed; identifying, using one or more processors, a second origin on the reference image, wherein the second origin is different from the first origin; calculating, using one or more processors, a scale factor based on a selected zoom level; converting, using one or more processors, the selected coordinates of the reference image using the scale factor; determining, using one or more processors, a distance between the transformed selected coordinates and a center of the reference image; and Live angiography feed coordinates and reference image coordinates are aligned using one or more processors based on the distance from the transformed selected coordinates to the center.

Images