Pullback cursor linking

Abstract

The present disclosure provides systems and methods for registering, or linking, vessel data of a region of interest captured during pullbacks that were obtained at different rimes or by different intravascular sensing modalities. The pullbacks may be registered based on vessel features, such as anatomical features or previously implanted medical devices, within the region of interest. Based on the identification of the same vessel feature in each pullback, the pullbacks can be registered such that vessel data captured at different times or by different intravascular sensing modalities can be easily compared, e.g., side by side.

Description

PULLBACK CURSOR LINKINGCROSS REFERENCE TO RELATED APPLICATION(S)

[0001] The present application claims the benefit of priority from United States Provisional App. No.63 / 711,965 filed on October 25, 2024, which is hereby incorporated by reference in its entirety.BACKGROUND

[0002] Intravascular images of vessels are typically obtained and used by physicians in determining courses of treatment and, subsequently, the outcomes of the treatment. To determine the efficacy of a treatment or the state of the vessel after the treatment, physicians typically have to guess or approximate whether a given image at a location along the vessel pre-treatment corresponds to an image at the same location along the vessel post-treatment. This can lead to inaccurate comparisons, such as when the physician wrongly chooses allegedly corresponding locations along the vessel pre- and post-treatment. BRIEF SUMMARY

[0003] The disclosure is generally directed to registering, or linking, vessel data of a region of interest captured during pullbacks that were obtained at different times. The pullbacks may be registered based on vessel features, such as anatomical features or previously implanted medical devices, within the region of interest. Based on the identification of the same vessel feature in each pullback, the pullbacks can be registered such that vessel data captured at different times can be easily compared, e.g., side by side. The registration and, therefore, linking of vessel data allows for representations of the vessel data to be linked such that a selection of a given location in a representation of a first pullback can be efficiently and accurately identified in a representation of a second pullback.

[0004] One aspect of the disclosure is directed to a method comprising receiving, by one or more processors, first vessel data of a region of interest of a vessel obtained during a first pullback; receiving, by one or more processors, second vessel data of at least the region of interest of the vessel obtained during a second pullback, wherein the second pullback is captured by a different intravascular sensing modality than the first pullback or captured at a different time than the first pullback; generating, by the one or more processors based on the first vessel data, a first two-dimensional representation of the region of interest of the vessel; generating, by the one or more processors based on the second vessel data, a second two-dimensional representation of at least the region of interest; providing for output, by the one or more processors, the first and second two-dimensional representations; identifying, by the one or more processors, a selection of a location along the first or second two-dimensional representation; automatically identifying, by the one or more processors, an equivalent location along the other two-dimensional representation; and providing for output, by the one or more processors, an indication corresponding to the selected location along the first or second two-dimensional representation and the equivalent location along the other two-dimensional representation.

[0005] Identifying the selection of the location along the first or second two-dimensional representation may occur automatically or may comprise receiving, by the one or more processors, an input corresponding to the selection of the location along the first or second two-dimensional representation.

[0006] The first vessel data and the second vessel data may comprise intravascular image data. The method may further comprise registering the first vessel data and the second vessel data. Registering the first and second vessel data may comprise receiving, by the one or more processors, an input corresponding to a selection of an initial location along the first or second two-dimensional representation; identifying, by the one or more processors, a selection of a comparable location along the other two-dimensional representation, wherein the initial location along the first or second two-dimensional representation and the comparable location along the other two-dimensional representation are substantially at a same location within the region of interest of the vessel; and registering, by the one or more processors based on the selection of the initial location and the comparable location, the first and second vessel data. When registering the first vessel data and second vessel data, the one or more processors may link the first and second two-dimensional representations. Registering the first and second vessel data may occur automatically.

[0007] The first pullback may be obtained before percutaneous coronary intervention, and the second pullback may be obtained post percutaneous coronary intervention. The percutaneous coronary intervention may comprise at least one of a stent, a balloon, atherectomy, optimization, or angioplasty.

[0008] The method may further comprise providing for output, by the one or more processors based on the selection of the location along the first or second two-dimensional representation, a first image frame of the first or second vessel data; and providing for output, by the one or more processors based on the identified equivalent location along the other two-dimensional representation, a second image frame of the other vessel data.

[0009] The method may further comprise providing for output, by the one or more processors, a connector linking the indication corresponding to the selected location along the first or second two-dimensional representation and the indication corresponding to the equivalent location.

[0010] Another aspect of the disclosure is directed to a system comprising one or more processors, the one or more processors may be configured to receive first vessel data of a region of interest of a vessel obtained during a first pullback; receive second vessel data of at least the region of interest of the vessel obtained during a second pullback, wherein the second pullback is captured by a different intravascular sensing modality than the first pullback or captured at a different time than the first pullback; generate, based on the first vessel data, a first two-dimensional representation of the region of interest of the vessel; generate, based on the second vessel data, a second two-dimensional representation of at least the region of interest; provide for output the first and second two-dimensional representations; identify a selection of a location along the first or second two-dimensional representation; automatically identify an equivalent location along the other two-dimensional representation; and provide for output an indication corresponding to the selected location along the first or second two-dimensional representation and the equivalent location along the other two-dimensional representation.

[0011] Identifying the selection of the location along the first or second two-dimensional representation may occur automatically or may comprise receiving, by the one or more processors, an input corresponding to the selection of the location along the first or second two-dimensional representation.

[0012] The first vessel data and the second vessel data may comprise intravascular image data. The one or more processors may be further configured to register the first vessel data and the second vessel data. Registering the first and second vessel data may comprise receiving, by the one or more processors, an input corresponding to a selection of an initial location along the first or second two-dimensional representation; identifying, by the one or more processors, a selection of a comparable location along the other two-dimensional representation, wherein the initial location along the first or second two-dimensional representation and the comparable location along the other two-dimensional representation are substantially at a same location within the region of interest of the vessel; and registering, by the one or more processors based on the selection of the initial location and the comparable location, the first and second vessel data. When registering the first vessel data and second vessel data, the one or more processors may link the first and second two-dimensional representations. Registering the first and second vessel data may occur automatically.

[0013] The first pullback may be obtained before percutaneous coronary intervention, and the second pullback may be obtained post percutaneous coronary intervention. The percutaneous coronary intervention may comprise at least one of a stent, a balloon, atherectomy, optimization, or angioplasty.

[0014] The one or more processors may be further configured to provide for output, by the one or more processors based on the selection of the location along the first or second two-dimensional representation, a first image frame of the first or second vessel data; and provide for output, by the one or more processors based on the identified equivalent location along the other two-dimensional representation, a second image frame of the other vessel data.

[0015] The one or more processors may be further configured to provide for output, by the one or more processors, a connector linking the indication corresponding to the selected location along the first or second two-dimensional representation and the indication corresponding to the equivalent location.

[0016] Yet another aspect of the disclosure is directed to one or more non-transitory computer readable storage medium storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising: receiving first vessel data of a region of interest of a vessel obtained during a first pullback; receiving second vessel data of at least the region of interest of the vessel obtained during a second pullback, wherein the second pullback may be captured by a different intravascular sensing modality than the first pullback or captured at a different time than the first pullback or by another intravascular sensing modality or a same modality; generating, based on the first vessel data, a first two-dimensional representation of the region of interest of the vessel; generating, based on the second vessel data, a second two-dimensional representation of at least the region of interest; providing for output the first and second two-dimensional representations; identifying a selection of a location along the first or second two-dimensional representation; automatically identifying an equivalent location along the other two-dimensional representation; and providing for output an indication corresponding to the selected location along the first or second two-dimensional representation and the equivalent location along the other two-dimensional representation.

[0017] Identifying the selection of the location along the first or second two-dimensional representation may occur automatically or may comprise receiving, by the one or more processors, an input corresponding to the selection of the location along the first or second two-dimensional representation.

[0018] The first vessel data and the second vessel data may comprise intravascular image data. The one or more processors may further perform operations comprising registering the first vessel data and the second vessel data. Registering the first and second vessel data may comprise receiving, by the one or more processors, an input corresponding to a selection of an initial location along the first or second two-dimensional representation; identifying, by the one or more processors, a selection of a comparable location along the other two-dimensional representation, wherein the initial location along the first or second two-dimensional representation and the comparable location along the other two-dimensional representation are substantially at a same location within the region of interest of the vessel; and registering, by the one or more processors based on the selection of the initial location and the comparable location, the first and second vessel data. When registering the first vessel data and second vessel data, the one or more processors may link the first and second two-dimensional representations. Registering the first and second vessel data may occur automatically.

[0019] The first pullback may be obtained before percutaneous coronary intervention, and the second pullback may be obtained post percutaneous coronary intervention. The percutaneous coronary intervention may comprise at least one of a stent, a balloon, atherectomy, optimization, or angioplasty.

[0020] The one or more processors may further perform operations comprising providing for output, by the one or more processors based on the selection of the location along the first or second two-dimensional representation, a first image frame of the first or second vessel data; and providing for output, by the one or more processors based on the identified equivalent location along the other two-dimensional representation, a second image frame of the other vessel data.

[0021] The one or more processors may further perform operations comprising providing for output, by the one or more processors, a connector linking the indication corresponding to the selected location along the first or second two-dimensional representation and the indication corresponding to the equivalent location.BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 is an example data collection system for collecting intravascular and extravascular data, according to aspects of the disclosure.

[0023] Figure 2 is an example interface screen which may implemented, for example, in the context of the data collection system of Figure 1, according to aspects of the disclosure.

[0024] Figure 3 is another example interface screen which may implemented, for example, in the context of the data collection system of Figure 1, according to aspects of the disclosure.

[0025] Figure 4 is a flow diagram illustrating a method of registering pullbacks, which may be implemented within the data collection system of Figure 1 , for implementing the example interface screens of Figures 2 and 3, according to aspects of the disclosure.DETAILED DESCRIPTION

[0026] The technology is generally directed to registering, or linking, two or more pullbacks of a region of interest of a vessel captured at different times or by different intravascular sensing modalities. Registering pullbacks captured at different times, such as before percutaneous coronary intervention (PCI) and after PCI, allows for a location indicator in a first pullback representation, e.g., a pre-PCI pullback representation, to be linked to a location indicator in a second pullback representation, e.g., a post-PCI pullback representation. Linking the localion indicators in the pullback representations allows for the screens provided via the graphical user interface (GUI) to be automatically updated upon receiving inputs, or user instructions, corresponding to a selection of a given location along the pullback representation. This allows for users, e.g., physicians, to easily view and, therefore, access, information pertaining to the selected location in the vessel. For example, if a first pullback was taken pre-PCI and a second pullback was taken post-PCI, a user can easily view and access the success of the PCI at a given location within the vessel by having pre- and post-PCI vessel data provided for output in response to a single input, or user instruction, selecting a location along the vessel representation.

[0027] The pullback may be a pullback of an intravascular sensing and / or imaging device, such as an optical coherence tomography (OCT) probe, an intravascular ultrasound (IVUS) catheter, a micro-OCT probe, a near infrared spectroscopy (NIRS) sensor, an optical frequency domain imaging (OFDI), or any other device that can be used to image or acquire sensing data on a blood vessel. The pullback of the intravascular sensing and / or imaging device may capture vessel data, such as image frames. The vessel data may be used to generate one or more representations of the vessel, such as graphical two-dimensional representations, longitudinal representations, three-dimensional representations, or the like. A representation of the vessel data may be generated for each pullback. For example, a first representation of the vessel may be generated based on vessel data obtained during the first pullback and a second representation of the vessel may be generated based on vessel data obtained during the second pullback. The first and second representation may be the same type of representation, e.g., both the first and second representations are two-dimensional representations, longitudinal representations, three-dimensional representations, etc.

[0028] The first and second vessel data are registered, and the first and second representations are linked. Registering the vessel data and linking the reprcsenlalions includes, for example, identifying one or more locations in the first or second vessel data and / or representation and, for each identified location, identifying a comparable, or equivalent, location in the other vessel data and / or representation. As an example, if one location is identified in the first vessel data and / or rcpresenlalion, then a comparable location would be idem! lied in the second vessel data and / or representation. Continuing with this example, if the location in the first vessel data and / or representation is identified as position X along the vessel, the comparable location in the second vessel data and / or representation would also be position X, or substantially position X, along the vessel.

[0029] The location may be identified based on an input, or user instruction, received by the system. For example, the system is configured to receive the input, or user instruction, corresponding to the selection of the localion along a first or second representation of the vessel. The input may be received, for example,via the GUI, such as through input via a touchscreen, mouse click, trackbar, cursor, etc. In other examples, the input may be received via other components, such as verbal instructions received via a microphone, gestures received via a camera or motion sensing device, etc. The localion may be, for example, one or more easily idenli liable locations along the vessel, such as a side branch, a bifurcation, a bend, a diameter of the vessel (such as a narrowing or widening of the vessel), one or more nodules of calcium, a proximal or distal end of a previously implanted device (such as a stent), a proximal or distal end of a guide catheter, a take-off angle of a device (such as a stent), etc. The system is further configured to receive another input, or user instruction, corresponding to the selection of the comparable localion on the other representation of the vessel. Based on the received inputs, the system registers the first and second vessel data and link the first and second representations based on the understanding that the selected local ions are at the same, or substantially the same, localion along the vessel in each pullback.

[0030] In some examples, the locations may be automatically idenli lied. For example, the system may automatically identify anatomical features and / or features of implanted medical devices within the first and second vessel data. The anatomical features may be one or more of a side branch, a bifurcation, a bend, a diameter of the vessel (such as a narrowing or widening of the vessel), one or more nodules of calcium, etc. The features of implanted medical devices may be one or more of a proximal or distal end of a previously implanted device (such as a stent), a proximal or distal end of a guide catheter, a take-off angle of a device (such as a stent), etc. The system may compare the anatomical features in the first and second vessel data and / or representations to determine which anatomical features are the same or substantially the same. In some examples, the system may determine that the anatomical features are the same or substantially the same based on a pattern, or sequence, of the anatomical features. For example, if the system identifies a side branch, then a bend, then a region of vessel having increased calcium burden in the first vessel data and / or representation, the system may identify a corresponding region in the second vessel data and / or representation that includes the same sequence of a side branch, then a bend, then a region having increased calcium burden. Based on the comparison, the system may register a given location in the first vessel data with the comparable localion in the second vessel data, thus linking the given location in the first representation with the comparable localion in the second representation. By registering a given location with a comparable location in the first and second vessel data, the system can register the remaining portions of the first and second vessel data and link the first and second representations.

[0031] An offset may be one or more consecutive frames in a first pullback that do not match corresponding frames in the second pullback. According to some examples, the vessel data and / or representations, once linked, provide for a constant relative offset between the vessel data and / or representations. For example, once linked, when a first frame, e.g., frame “x,” is selected in a first representation, the corresponding frame in a second frame would be frame x + 8, where 8 is the constant offset between the first and second representations. The constant 8 may be determined based on the frames selected when linking the vessel data and / or representations, whether via inputs or automatically. For example, 8 may be determined based on an analysis of the vessel features, such as side branches, or other landmark frames present in both sets of vessel data and / or representations.

[0032] As an example, if there are “n” number of landmark frames in a first pullback and “m” number of landmark frames in a second pullback, the system may be configured to correlate the two pullbacks. For example, the system can loop over all possible offsets 8 between the two pullbacks. For each offset 8, the data collection system 100 can be configured to identify a first target feature in the first pullback and a second target feature corresponding to the first target feature in the second pullback. A distance between the identified target feature and all other features in each pullback may be determined. Based on the determined distance between the identified target feature and all other features, one or more of the identified frames in the first pullback or second pullback may be linearly shifted to line up at least some of the features. Specifically, in some examples, a penalty score may be determined for each offset 8. The penalty score may correspond to a sum of displacements from each feature in the first pullback to the nearest feature in the second pullback, and vice versa. The offset 8 that yields the lowest combined penalty score may be chosen as the automatically linked offset 8. As such, this general alignment method may adjust the order or inclusion of one or more frames in one pullback to generally align with the one or more frames in the second pullback.

[0033] The linking of the vessel data and / or representations can be generalized to an offset function 8(n), which allows for a non-constant offset between the two pullbacks. A non-constant offset 8 accounts for motion artifacts, such as motion caused from the heart beating, present in both pullbacks. For example, once the frames in the pullbacks are generally aligned as described above, individual corresponding frames may be locally aligned. In some examples, one or more frames in a first pullback that do not match one or more corresponding frames in a second pullback may be identified. In some examples, the mismatched one or more frames may be identified by mapping points corresponding to features and determining that the mapped points do not match. Once this mapping is determined in one direction along the frames, the same mapping may also be used in the other direction along the frames. In some examples, a spline fit may be used to locally align the mismatched frames. The spline fit may include using a spline -based transformation to warp the one or more frames based on the mapped points. The spline fit may be any polynomial function, such as a cubic spline or a quadratic spline.

[0034] The vessel data and / or representations may be provided for output via the GUI, or display, or the system. For example, the interface screen may include the first and second representations of the pullbacks. In some examples, the interface screen may, additionally, include an image frame captured during the pullbacks. The representations and / or image frames of the first and second pullback may be provided side by side or vertically with respect to one another. For example, an image frame from the first pullback and the corresponding first representation may be provided on a first portion of the screen while an image frame from the second pullback and the corresponding second representation may be provided on a second portion of the screen such that the image frames are next to each other and the representations are next to each other. In some examples, the first portion may be a left side of the screen while the second portion may be a right side of the screen. In another example, the first portion may be a top portion of the screen while the second portion may be a bottom portion of the screen. In some examples, the representations and / or image frames of the first and second pullback may be adjustable by a user, such that either or bothof the representations or image frames can be resized, moved to a different portion of the screen, or otherwise adjusted.

[0035] The system may be configured to receive inputs, or user instructions, corresponding to a location along the first or second representations, once the vessel data is registered. For example, the first and second representations may be linked. The system may receive an input, or user instruction, corresponding to a selection of a location along the first representation. In other examples, the system may automatically identify a selection of a location along the first representation. The system may provide, as output, a location indicator at the selected localion on the first representation. The system may, additionally, provide as output the image frame of the first vessel data corresponding to the selected localion along the first representation. In response to receiving the selection of the localion along the first representation, the system automatically identifies the corresponding location in the second representation. The system provides, as output, a location indicator at the corresponding localion in the second representation. In some examples, the location indicators in the first and second representations may be linked, such that a line or connection is provided for output linking the location indicators. The system may, additionally, provide as output the image frame of the second vessel data corresponding to the idenli l ied corresponding localion in the second representation.

[0036] According to some examples, the system may be used before, during, or after surgical procedures to monitor changes to vessel data. The vessel data may include, but is not limited to, lumen contours, vessel diameters, vessel cross-sectional areas, lumen area, external elastic lamina (EEL) values, EEL diameters, EEL arcs, lesion locations, lesion size, plaque burdens, virtual flow reserve (VFR), fractional flow reserve (FFR), landing zones, treatment zones, virtual stents bounded by the landing zones or the like. In examples in which the surgical procedure includes a cardiovascular stent placement, for example, the physician may take actions that would alter vessel data. For example, a physician may debulk plaque buildup in a target region for stent placement. In some examples, a physician may capture an initial pullback including indications of plaque burden in a region of interest for stent placement. After debulking that plaque buildup, the physician may then capture a post-debulking pullback of the region of interest in the initial pullback. The system may register the initial and post- debulking pullbacks using a location indicator to track any changes in plaque burden before and after the debulking to assess whether a sufficient amount of plaque buildup was removed to implant a stent. In some examples, the physician may determine that a sufficient amount of plaque buildup was not removed and repeat this process by continuing to debulk plaque buildup and continuing to compare additional pullbacks to the initial or previous pullbacks unlil the stent can be placed. In some examples, the physician can include an annotation to accompany vessel data, such as an annotation for an indication of plaque burden, which can further aid in monitoring changes to vessel data during the surgery.

[0037] According to some examples, the system may be used to monitor changes to vessel data over time, for example, to track the effectiveness of a medication, lifestyle change, procedure, or surgery. In some examples, an initial or pre-treatment pullback includes indications in a region of interest of a vascular condition or disease that requires treatment, such as indications of plaque burden. The physician maysubsequently provide a treatment plan to reduce plaque burden, including, but not limited to, medications, such as aspirin, beta blockers, calcium channel blockers, and ACE inhibitors, or cholesterol-lowering drugs including statins; lifestyle changes, such as diet, stress, or exercise changes; procedures or surgery, including PCI, coronary artery bypass grafting (CABG), trans-myocardial laser revascularization or coronary endarterectomy, or carotid endarterectomy. At the conclusion of the treatment plan, which may be for any number of days, months, or years as determined by the physician, a post-treatment pullback of the region of interest in the initial pullback can be captured. The system may register the pre- and posttreatment pullbacks using a localion indicator to track any changes in plaque burden before and after the treatment to assess potential additional treatments. In examples when the post-treatment pullback is captured during the procedures or surgery, the potential additional treatments may include further procedures or surgery. For example, the system made be used to assess calcium modification in real-time during the procedure or surgery to assess whether further calcium modification is required. In some examples, the physician can include an annotation to accompany vessel data, such as an annotation for an indication of plaque burden, which can further aid in monitoring changes to vessel data over lime.

[0038] According to some examples, the system may be used to monitor changes to vessel data over time, for example, to track the development of vascular conditions or diseases. In some examples, an initial pullback including indications in a region of interest of a vascular condition or disease, such as indications of plaque burden, can be captured and analyzed. The physician may determine, based on the initial pullback, that treatment is not yet required. After any number of days, months, or years as determined by the physician, an additional pullback of the region of interest in the initial pullback may be captured. The system may register the initial and additional pullbacks, thereby allowing the user, e.g., physician, to track any changes in plaque burden to whether treatment is required. In some examples, the physician can include an annotation to accompany vessel data, such as an annotation for an indication of plaque burden, which can further aid in monitoring changes to vessel data over lime.

[0039] According to some examples, the system for merging pullbacks may be used to create a longer merged pullback using two or more pullbacks with at least one overlap region. For example, the system for merging pullbacks may be used to create a merged pullback having a length that is, potentially, larger than the length limit of the imaging system. A longer merged pullback enables the physician to assess vessel data at several different points within a region of the vessel. For example, the physician may use a longer merged pullback to consider the difference in the amount of plaque throughout a longer region of a vessel.

[0040] In some examples, the physician may use the system for linking cursors in conjunction with the system for merging pullbacks to register two or more longer merged pullbacks. In such examples, the physician may use the registered longer pullbacks, for example, to assess vessel data from different limes within a larger region of the vessel than a region without the system to merge longer pullbacks. For example, the registered longer pullbacks may be used in connection with a surgical operation, assessing progress with a treatment plan, or assessing progress of a vascular condition or disease, as disclosed herein.

[0041] While the technology is primarily described in connection with linking or merging two pullbacks, in some examples, the technology can be used to link or merge more than two pullbacks.Example System

[0042] Figure 1 illustrates a data collection system 100 for use in collecting vessel data, such as intravascular and extravascular data. The system 100 may include a non-invasive imaging system 120, an intravascular device 104, a storage device 106, subsystem 108, and network 102. The subsystem 108 may include an optical receiver 110, computing device 112, and display 118.

[0043] The non-invasive imaging system 120 may be, for example, a nuclear magnetic resonance, x-ray, computer aided tomography, or other suitable non-invasive imaging technology. In particular, the non-invasive imaging system may be an angiography system configured to generate pre-contrast X-ray angiograms (XAs) before a radio opaque contrast agent has been used, high dose contrast XAs taken while a radio-opaque contrast agent is present in one or more blood vessels of interest, and low dose contrast XAs taken when little or no contrast agent is present. The high dose contrast XAs may, in some examples, be cine images, and the low dose contrast XAs may be fluoroscopic images. For example, the angiography system 120 may include a fluoroscopy system. The angiography system 120 may be configured to noninvasively image a subject S such that frames of angiography data, typically in the form of frames of image data, are generated. According to some examples, the x-ray imaging may occur while a device 104 is being delivered via the vessel such that a blood vessel in region R of subject S is imaged using angiography. In some examples, high dose XAs may be generated before the intravascular device 104 is delivered via the vessel, while the device is being delivered, and / or after the device has been delivered. The imaging results of a non-invasive scan may be provided for output on display 118. The high dose and low dose XAs may be correlated such that the device and / or treatment zone may be dynamically visualized during a procedure. As discussed above and herein, the non-invasive imaging system may be an angiography system. However, the non-invasive imaging system 120 may use various other imaging technologies and, therefore, references to the non-invasive imaging system 120 being an angiography system are not intended to be limiting.

[0044] The non-invasive imaging system 120 may be in communication with a storage device 106 via network 102. In some examples, the imaging system 120 may be in direct communication with storage device 106, e.g., without having to be connected via network 102. The storage device 106 may be a workstation or server. The data collected and / or generated by imaging system 120 may be stored and managed by the storage system 106. In some examples, a subsystem, a server or workstation handles the functions of storage device 106. According to some examples, the imaging system 120 may generate electromagnetic radiation, such as x-rays. The imaging system 120 may, in some examples, receive such radiation after passing through the subject S. In turn, data storage 106 may use the signals from the imaging system 120 to image one or more regions of the subject S including region R. In some examples, storage device 106 may be integrated with imaging device 120.

[0045] The region of interest R may be a subset of the vascular or peripherally vascular system, such as a particular blood vessel. The region R may be a region for the delivery of a device 104. The device may be,for example, a stent, balloon, or the like. In some examples, the device 104 may be an intravascular device, such as an OCT probe, an IVUS catheter, a micro-OCT probe, a NIRS sensor, an OFDI, or any other device that can be used to image a blood vessel. In some examples, the device 104 may be a pressure wire, a flow meter, etc. The device 104 may include a device tip, one or more radiopaque markers, an ophcal fiber, a torque wire, or the like. Additionally, the device tip may include one or more data collecting subsystems such as an optical beam director, an acoustic beam director, a pressure detector sensor, other transducers or detectors, and combinations of the foregoing.

[0046] In examples where the device 104 includes an optical beam director, the optical fiber may be in optical communication with the device 104 and / or with the beam director. The torque wire may define a bore in which an optical fiber is disposed. According to some examples, the device 104 may include the sheath such as a polymer sheath (not shown) which forms part of a catheter. The optical fiber, which in the context of an intravascular system is a portion of the sample arm of an interferometer, may be optically coupled to subsystem 108 and / or a patient interface unit (PIU).

[0047] A guide wire, not shown, may be used to introduce the device 104 into the blood vessel. In examples where the device 104 is an intravascular data collection device, the device 104 may be introduced and pulled back along a length of a blood vessel while collecting data. In examples where the device 104 is a treatment device, such as a device to deliver a balloon or stent, the device 104 may be introduced to the vessel and guided to a treatment zone.

[0048] The device 104 may be connected to a subsystem 108. According to some examples, the device 104 may be connected to subsystem 108 via an optical fiber. The subsystem 108 may include a light source, such as a laser, an interferometer having a sample arm and a reference arm, various optical paths, a clock generator, photodiodes, and other OCT, IVUS, micro-OCT, NIRS, and / or pressure wire components. The device 104 may be connected to an optical receiver 110. According to some examples, the optical receiver 110 may be a balanced photodiode based system. The optical receiver 110 may be configured to receive light collected by the device 104. The device 104 may be coupled to the optical receiver 110 via a wired or wireless connection.

[0049] The system 100 may further include, or be configured to receive data from, a non-invasive imaging system 120. The non-invasive imaging system 120 may be, for example, an imaging system based on angiography, fluoroscopy, x-ray, nuclear magnetic resonance, computer aided tomography, etc. non-invasive imaging system 120 may be configured to noninvasively image the blood vessel. According to some examples, the non-invasive imaging system 120 may obtain one or more images before, during, and / or after a pullback of the data collection device 104. Non-invasive imaging system 120 may be used to image a patient such that diagnostic decisions can be made and various possible treatment options such as stent placement can be carried out. These and other imaging systems can be used to image a patient externally or internally to obtain raw data, which can include various types of image data.

[0050] The non-invasive imaging system 120 may be in communication with subsystem 108. According to some examples, the non-invasive imaging system 120 may be wirelessly coupled to subsystem 108 via network 102. For example, the non-invasive imaging system 120 may be wirelessly coupled to subsystem108 via a communications interface, such as Wi-Fi or Bluetooth. In some examples, the non-invasive imaging systeml20 may be in communication with subsystem 108 via a wire, such as an optical fiber. In yet another example, external imaging device 120 may be indirectly communicatively coupled to subsystem 108 or computing device 112. For example, the non-invasive imaging device 120 may be coupled to a separate computing device (not shown) that is in communication with computing device 112. As another example, data from the imaging system 120 may be transferred to the computing device 112 using a computer-readable storage medium, from storage device 106 via network 102, or the like.

[0051] The subsystem 108 may include a computing device 112. One or more steps may be performed automatically or without user input to navigate images, input information, select and / or interact with an input, etc. In some examples, one or more steps may be performed based on receiving a user input by mouse clicks, a keyboard, a touch screen, verbal commands, etc. The computing device 112 may include one or more processors 113, memory 114, instructions 115, data 116, and one or more modules 117.

[0052] The one or more processors 113 may be any conventional processor, such as a commercially available microprocessor. Alternatively, the one or more processors may be a dedicated device such as an application specific integrated circuit (ASIC) or other hardware-based processor. Although Figure 1 functionally illustrates the processor, memory, and other elements of device 112 as being within the same block, it will be understood by those of ordinary skill in the art that the processor, computing device, or memory may actually include multiple processors, computing devices, or memories that may or may not be stored within the same physical housing. Similarly, the memory may be a hard drive or other storage media located in a housing different from that of device 112. Accordingly, references to a processor or computing device will be understood to include references to a collection of processors or computing devices or memories that may or may not operate in parallel.

[0053] Memory 114 may store information that is accessible by the processors, including instructions 115 that may be executed by the processors 113, and data 116. The memory 114 may be a type of memory operative to store information accessible by the processors 113, including a non-transitory computer-readable medium, or other medium that stores data that may be read with the aid of an electronic device, such as a hard-drive, memory card, read-only memory (ROM), random access memory (RAM), optical disks, as well as other write-capable and read-only memories. The subject matter disclosed herein may include different combinations of the foregoing, whereby different portions of the instructions 115 and data 116 are stored on different types of media.

[0054] Memory 114 may be retrieved, stored or modified by processors 113 in accordance with the instructions 115. For instance, although the present disclosure is not limited by a particular data structure, the data 116 may be stored in computer registers, in a relational database as a table having a plurality of different fields and records, XML documents, or flat files. The data 116 may also be formatted in a computer-readable format such as, but not limited to, binary values, ASCII or Unicode. By further way of example only, the data 116 may be stored as bitmaps comprised of pixels that are stored in compressed or uncompressed, or various image formats (e.g., JPEG), vector-based formats (e.g., SVG) or computer instructions for drawing graphics. Moreover, the data 116 may comprise information sufficient to identifythe relevant information, such as numbers, descriptive text, proprietary codes, pointers, references to data stored in other memories (including other network locations) or information that is used by a function to calculate the relevant data.

[0055] The instructions 115 can be any set of instructions to be executed directly, such as machine code, or indirectly, such as scripts, by the processor 113. In that regard, the terms “instructions,” “application,” “steps,” and “programs” can be used interchangeably herein. The instructions can be stored in object code format for direct processing by the processor, or in any other computing device language including scripts or collections of independent source code modules that are interpreted on demand or compiled in advance. Functions, methods and routines of the instructions are explained in more detail below.

[0056] According to some examples, the computing device 112 may include receiving, either by a wired connection or via a wireless connection, data from the device 104. The data may include, for example, intravascular data including intravascular imaging data, pressure data, temperature data, flow data, or the like. The data received by the computing device 112 from imaging system 120, storage device 106, and / or device 104 may be used to dynamically visualize the device 104 on one or more extraluminal images. In some examples, where the device is an intravascular data collection device, the data received from the device may be used to determine plaque burden, FFR measurements at one or more locations along the vessel, calcium angles, EEL detections, calcium detections, proximal frames, distal frames, EEL-based metrics, stent / no stent decisions, scores, recommendations for debulking and other procedures, evidence based recommendations informed by automatic detection of regions / features of interest, stent planning, etc.

[0057] In examples, the device 104 is an intravascular data collection device, the data obtained by device 104 and / or imaging system 120 may be processed by one or more modules to provide information about the blood vessel including lumen contours, vessel diameters, vessel cross-sectional areas, lumen area, EEL values, EEL diameters, EEL arcs, lesion locations, lesion size, plaque burdens, VFR, FFR, landing zones, treatment zones, virtual stents bounded by the landing zones or the like. According to some examples, the modules 117 may include a dynamic visualization module, an EEL detection module, a lumen detection module, a lesion detection module, a co-regislralion module, and / or a registration and / or linking module.

[0058] The dynamic visualization module may automatically correlate high dose and low dose extraluminal images such that a vessel map may be overlaid on the live low dose extraluminal images, a treatment zone identified on the high dose extraluminal images and / or vessel map may be overlaid on the live low dose extraluminal images, and / or the device 104 may be tracked on the live low dose extraluminal images with reference to the vessel map. The dynamic visualization module may correlate the high and low dose XA based on motion features detected in both the high and low dose extraluminal images. In some examples, the dynamic visualization module may use Al, such as one or more Al or machine learning (ML) models, to identify the working vessel, detect markers on the device, and track the motion of the device.

[0059] The EEL detection module may automatically detect and measure the EEL diameter of a given intravascular image frame taken during a pullback of a vessel. The lumen detection module mayautomatically detect and measure the lumen diameter of a given intravascular image frame taken during a pullback of the vessel. The lesion detection module may automatically detect lesions within the vessel based on vessel data obtained from the device 104 and / or imaging system 120. The side branch detection module may process the vessel data obtained from the device 104 and / or imaging system 120 to detect one or more side branches of the blood vessel.

[0060] The computing device 112 may be adapted to co-register vessel data obtained during a pullback of the device 104 with intravascular image and / or an extraluminal image. For example, the computing device 112 may be configured to receive and store extraluminal image data, such as image data generated by imaging system 120 and obtained by a frame grabber, which may be a device configured to capture frames from analog video signal or digital video stream. The computing device 112 may be configured to receive and store intravascular image data, such as image data generated by device 104 and obtained by the frame grabber. In some examples, computing device 112 may access the co-regislralion module to co-register the vessel data with the luminal image. The luminal image may be an extraluminal image, such as an angiograph, x-ray, or the like. The co-registration module may co-register intravascular data, such as an intravascular image, plaque burden, EEL measurement, lumen diameter measurements, pressure readings, VFR, FFR, resting full-cycle ratio (RFR), flow rates, etc. with the extraluminal image. In some examples, the co-regislralion module may co-register intravascular data with an intraluminal image, such as an intraluminal image captured by an OCT probe, IVUS probe, micro-OCT probe, or the like.

[0061] In one example, the co-registration module may co-register intraluminal data captured during a pullback with one or more extraluminal images. For example, the extraluminal image frames may be pre-processed. Various matrices such as convolution matrices, Hessians, and others can be applied on a per pixel basis to change the intensity, remove, or otherwise modify a given angiography image frame. As discussed herein, the preprocessing stage may enhance, modify, and / or remove features of the extraluminal images to increase the accuracy, processing speed, success rate, and other properties of subsequent processing stages. A vessel centerline may be determined and / or calculated. In some examples, the vessel centerline may be superimposed or otherwise displayed relative to the pre-processed extraluminal image. According to some examples, the vessel centerline may represent a trajectory of the device 104, such as an intravascular device, through the blood vessel during a pullback. In some examples, the centerline may be referred to as a trace. Additionally or alternatively, marker bands or radiopaque markers may be detected in the extraluminal image frames. According to some examples, the extraluminal image frames and the data received by device 104 may be co-registered based on the determined location of the marker bands.

[0062] The registration or linking module may, in some examples, register two or more pullbacks of the device 104. For example, each pullback of the device 104 may be captured at different times, such as before and after percutaneous coronary intervention (PCI). For example, a first pullback of the device 104 may be performed pre -PCI and a second pullback of the device 104 may be performed post-PCI. In some examples, a first pullback of the device 104 through the vessel of interest may be captured at a first lime and a second pullback of the device 104 through the vessel of interest may be captured at a second time, different than the first lime. In such an example, the first and second pullbacks may both be captured pre- or post-PCI.

[0063] The registration module may register, or link, the two pullbacks such that the receipt of user i nslruclions through a GUI or other input device to select a given frame of the first pullback will cause the system to identify a corresponding frame in the second pullback. The corresponding frame in the second pullback may be a frame, e.g., an image frame, of the same or substantially same location along the vessel of interest captured in the first and second pullbacks.

[0064] According to some examples, the modules may additionally or alternatively include a video processing software module, a preprocessing software module, an image file size reduction software module, a catheter removal software module, a shadow removal software module, a vessel enhancement software module, a blob enhancement software module, a Laplacian of Gaussian filter or transform software module, a guide wire detection software module, an anatomic feature detection software module, stationary marker detection software module, a background subtraction module, a Frangi vesselness software module, an image intensity sampling module, a moving marker software deleclion module, iterative centerline testing software module, a background subtraction software module, a morphological close operation software module, a feature tracking software module, a catheter deleclion software module, a bottom hat filter software module, a path deleclion software module, a Dijkstra software module, a Viterbi software module, fast marching method based software modules, a vessel centerline generation software module, a vessel centerline tracking module software module, a Hessian software module, an intensity sampling software module, a superposition of image intensity software module and / or other suitable software modules as described herein. According to some examples, the modules may include software such as preprocessing software, transforms, matrices, and / or other software -based components that are used to process image data or respond to patient triggers to facilitate co-registration of different types of image data by other software -based components or to otherwise perform annotation of image data to generate ground truths and other software, modules, and / or functions suitable for implementing various features of the disclosure. The modules can include lumen detection using a scan line based or image-based approach, stent detection using a scan line based or image-based approach, indicator generation, apposilion bar generation for stent planning, guide wire shadow indicator to prevent confusion with dissention, side branches and missing data, and / or others.

[0065] In some examples, the modules may be configured to process the vessel data obtained by the device 104 and / or imaging system 120 using artificial intelligence (Al) algorithms, machine learning techniques, or the like.

[0066] The subsystem 108 may include a display 118 for outputting content to a user. The display 118 may be integrated with the computing device 112, or it may be a standalone unit electronically coupled to the coinpuling device 112. The display 118 may output intravascular data relating to one or more features detected in the blood vessel and / or obtained during a pullback. For example, the output may include, without limitation, cross-sectional scan data, longitudinal scans, three-dimensional representations generated based on intraluminal and / or extraluminal images, diameter graphs, image masks, lumen border, plaque sizes, plaque circumference, visual indicia of plaque localion, visual indicia of risk posed to stentexpansion, flow rate, suggested treatment zones, or the like. The display 118 may identify features with text, arrows, color coding, highlighting, contour lines, or other suitable human or machine-readable indicia.

[0067] According to some examples the display 118 may include a GUI. The display 118 may be a touchscreen display in which a user can provide an input to navigate images, input information, select and / or interact with an input, etc. In some examples, the display 118 and / or computing device 112 may include an input device, such as a trackpad, mouse, keyboard, etc. that allows a user to navigate images, input information, select and / or interact with an input, etc.

[0068] In some examples, the information input by the user may be annotations. For example, the system may be configured to receive annotations to the one or more representations of the vessel. The annotations may be, in some examples, an indication of plaque burden, FFR measurements at one or more locations along the vessel, calcium angles, EEL detections, calcium detections, proximal frames, distal frames, EEL-based metrics, stent / no stent decisions, scores, recommendations for debulking and other procedures, evidence based recommendations informed by automatic detection of regions / features of interest, stent planning, etc. In some examples, the annotations may be a treatment device landing zone, balloon device zone, vessel prep device zone, and / or lesion related zone. For example, the system may receive an input corresponding to a proximal and distal location along the vessel corresponding to a proximal and distal location of a treatment device landing zone, balloon device zone, vessel prep device zone, and / or lesion related zone.

[0069] According to some examples, the annotations may be automatically determined by the system. For example, the system may, based on vessel data, determine one or more of plaque burden, FFR measurements at one or more locations along the vessel, calcium angles, EEL detections, calcium detections, proximal frames, distal frames, EEL-based metrics, stent / no stent decisions, scores, recommendations for debulking and other procedures, evidence based recoin me ndal ions informed by automatic detection of regions / features of interest, stent planning, a treatment device landing zone, balloon device landing zone, vessel prep device zone, lesion related zone, etc. The system may automatically provide the plaque burden, FFR measurements at one or more locations along the vessel, calcium angles, EEL detections, calcium detections, proximal frames, distal frames, EEL-based metrics, stent / no stent decisions, scores, recommendations for debulking and other procedures, evidence based recommendations informed by automatic detection of regions / features of interest, stent planning, a treatment device landing zone, balloon device landing zone, vessel prep device zone, lesion related zone, etc. for output as one or more annolalions on at least one of the vessel representations.

[0070] In some examples, inputs received by the system corresponding to input information and / or annotations (collectively “annotations”) may be provided for output on display 118. Annotations received with respect to one vessel representation may be provided for display on the vessel representation in which the annotations were received. In some examples, the annotations may be provided for display on a plurality of vessel representations, such as the extraluminal representations, graphical two-dimensional representations, longitudinal representations, three-dimensional representations, or the like, the annolalions may be provided for output on one, some, or all the rcpresenlalions. According to some examples, whenthe system receives updated annotations on at least one of the representations, the annotations on the other representations may be updated to correspond to the updated annotations.

[0071] The display 118 alone or in combination with computing device 112 may allow for toggling between one or more viewing modes in response to user inputs. For example, a user may be able to toggle between different intravascular data, images, etc. recorded during each of the pullbacks. In some examples, the user may be able to toggle between different representations, such as a graphical two-dimensional representation, longitudinal representation, a cross-sectional representation, a three-dimensional representation, intravascular images, color images, black and white images, live images, or the like. The graphical two-dimensional representation may include, for example, a graphical representation of the vessel where a first axis corresponds to a location along the vessel of interest and a second axis corresponds to another value, such as diameter, pressure-based measurements or metrics (e.g., FFR, VFR, etc.), or any other value derived from the vessel data. In some examples, the graphical two-dimensional representation and / or the longitudinal represenlalion of the vessel of interest may be symmetric about the longest axis of the representation.

[0072] The representations of the vessel may provide various viewing angles and section views. In some examples, EEL positions, diameters thereof, or other EEL based parameters may be output for display relative to angular measurements, detected calcium arcs, plaque burden, or the like.

[0073] In some examples, one or more visual representations of the images may include an indication of a lesion location, lesion severity, lesion length, or the like. Additionally or alternatively, the indication of the lesion may be color coded, where each color represents the severity, length, or other measurement related to the lesion.

[0074] In some examples, the output may include candidate treatment zones. The candidate treatment zone may be, for example, a candidate stent landing zone. For example, the output may include an indication corresponding to a candidate proximal landing zone for a stent and a candidate distal landing zone for a stent. The candidate treatment zone may be determined based on the determined plaque burden, lesion locations, lesion length, or the like. The indications may be provided on any of the vessel representations, e.g., the three-dimensional representation, the longitudinal representation, the graphical representation, image data such as the external images, etc.

[0075] According to some examples, the display 118 and / or computing device 112 may be configured to receive one or more inputs corresponding to a selection on one or more representations. For example, an input may be received corresponding to a selection of an image frame on the longitudinal represenlalion. In response, the other representations provided for output may be updated to display a corresponding indication or image frame. For example, the extraluminal image may be updated to have an indication along the vessel corresponding to the location of the image frame selected in the longitudinal representation, a circumferential indication may be provided on a three-dimensional represenlalion corresponding to the location of the image frame selected in the longitudinal representation, the cross-sectional image frame may be updated to correspond to the image frame selected in the longitudinalrepresentation, etc. In some examples, the vessel data associated with the selected location may be updated and provided for display.

[0076] As discussed with respect to example interface screens shown in Figures 2 and 3, the system is configured to receive inputs or user instructions via the GUI (e.g., display 118) and / or computing device 112, to output a given image frame of the vessel of interest captured during a pullback. The given frame may be selected at a point along the length of the vessel of interest captured during the pullback. In response to receive the input, or user instruction, the system is configured to idenli fy the corresponding image frame in another pullback of the vessel of interest. The system, via display 118, provides an indication of the corresponding image frame in the other pullback in response to receiving the input, or user instruction, in conjunction with the initial pullback.

[0077] In some examples, the display 118, alone or in combination with computing device 112, may present one or more menus as output to the physician, and the physician may provide input in response by selecting an item from the one or more menus. For example, the menu may allow a user to show or hide various features. As another example, there may be a menu for selecting blood vessel features to display.Linked Pullbacks

[0078] Figure 2 illustrates an example display 200 of a GUI which may be implemented by display 118 of system 100 of Figure 1. The display 200 includes a first porlion 202 and a second portion 212. The first portion 202 may include one or more representations of a first pullback and the second porlion 212 includes one or more representations of a second pullback. The representations include an image frame 204, 214 and two-dimensional representations 206, 216.

[0079] The image frame 204, 214 corresponds to the image frame of a selected location on the two-dimensional representations 206, 216. The selected locations may be identified on the two-dimensional representation 206, 216 by identifiers 208, 218.

[0080] The image frames 204, 214 may include annotations or values associated with the vessel at the given frame. For example, as shown with respect to image frame 204, the annotations may include an indication of the calcium arc 205 for the frame 204. The calcium arc 205 may be represented numerically, e.g., 187 degrees, or visually, e.g., as a coaxial arc or ring around the perimeter of the intravascular image frame 204. Additional annotations may include an indication of the lumen 209, the EEL 211 , or the like.

[0081] The two-dimensional representations 206, 216 may, in some examples, be a symmetrical representation of the vessel. The representation may be symmetrical about a longest axis the representation.

[0082] In some examples, the two-dimensional representations 206, 216 may include a first axis corresponding to the location along the vessel and a second access corresponding to other data associated with the vessel. The other data associated with the vessel may include, for example, diameter values, EEL values, FFR values, VFR values, etc. In some examples, the two-dimensional representations 206, 216 may be graphical representations. As shown in display 200, the two-dimensional representations 206, 216 are generated based on diameter values of the vessel and are symmetrical about the longest axis of the representation. As shown, the two-dimensional representations 206, 216 extend horizontally across thescreen 200. However, in some examples, one or both two-dimensional representations 206, 216 may extend horizontally or vertically on the screen 200.

[0083] Information associated with the vessel determined based on the pullbacks may be provided relative to the two-dimensional representations 206, 216. For example, the two-dimensional representations 206, 216 may be color coded such that a given color may indicate the presence of calcium, plaque, etc. while another color indicates EEL or other data associated with the vessel. As shown, the two-dimensional representations 206, 216 include indicators 207, 217 identifying the location of side branches in the vessel. While only one side branch is identified in each representation 206, 216 with reference numbers 207, 217, respectively, the representations 206, 216 may include multiple side branches.

[0084] According to some examples, the first portion 202 may be based on vessel data captured during a pre -PCI pullback and the second portion 212 may be based on vessel data captured during a post-PCI pullback. In such an example, the representations in the second portion 212 may include an indication of the PCI. As shown in Figure 2, the PCI may be a stent placed within the vessel. Information associated with the stent may be provided relative to one or more of the representations in the second portion 212. For example, the percent expansion, e.g., 73%, an indication of apposition 219, proximal and distal boundaries of the stent 221, 223, or the like may be provided relative to the two-dimensional representation 216, as shown, and / or relative to the image frame 214.

[0085] As shown in Figure 2, the representations in the first portion 202 and the second portion 212 are not linked. In particular, movement of a given identifier, e.g., identifier 208, does not cause movement of the other identifier, e.g., identifier 218. Unlinked representations result in the user, e.g., physician, having to select the location in each representation, e.g., in each two-dimensional representation 206, 216, that they consider to be the same location. This is not accurate and can lead to the user viewing and / or comparing frames and related information of the vessel from two different locations. Such a comparison would be unhelpful and uninformative when analyzing whether PCI has been successful, whether there have been changes in a particular region within the vessel, or the like as the selected locations do not necessarily correspond to one another.

[0086] Figure 3 illustrates another example display 300 of a GUI, which may be implemented by display 118 of system 100 of Figure 1, in which the vessel data and / or representations in the first and second portions 202, 212 are linked. Linking the vessel data and / representations in the first and second portions 202, 212 allows the user to efficiently and effectively view two or more image frames at the same, or substantially the same, location in the vessel captured at two or more different times, e.g., from two or more different pullbacks. The vessel data and / or representations are linked locationally such that movement of an identifier in one representation, e.g., identifier 208 in representation 206, cause the identifier in the other representation, e.g., identifier 218 in representation 216, to move automatically and synchronously. Linking the vessel data and / or representations allows the user to scroll, or play, through the representations simultaneously such that the user can evaluate the anatomical features of the vessel in the same location captured at two different times.

[0087] To link the vessel data and / or representations, data collection system 100 may be configured to automatically identify a selection of a location on a first two-dimensional representation. In other examples, the GUI or computing device 112, may receive an input or user instruction corresponding to the selection of a location on a first two-dimensional representation. For purposes of this example, the first two-dimensional representation will be two-dimensional representation 206. The location may be an easily identifiable location in the representation 206, such as a given side branch, e.g., side branch 207, a narrowing of the vessel, a proximal or distal end of a previously implanted medical device, e.g., a stent, or the like. The GUI or computing device 112 may receive another input or user instruction corresponding to an equivalent location on the second two-dimensional representation. For purposes of this example, the second two-dimensional representation will be two-dimensional representation 216. The equivalent location would be the location of the same side branch, e.g., side branch 217, the same narrowing of the vessel, the same proximal or distal end of a previously implanted medical device, e.g., a stent, or the like. The selected location and equivalent location establish a corresponding location in the representations and / or vessel data such that the representation and / or vessel data proximal and distal to the selected location and equivalent location are locationally linked.

[0088] Once linked, the screen 300 may include an indication of the linking by connecting the location identifiers 208, 218 with a connector 320. As shown, connector 320 is a dashed line extending between the location identifiers 208, 218. While the linking of the identifiers 208, 218 is visually shown on the screen 300, the identifiers 208, 218 may be linked without a visual indication, e.g., without connector 320.

[0089] According to some examples, the vessel data and / or representations may be automatically linked. For example, the data collection system 100 may be configured to automatically identify equivalent locations in the vessel data and / or representations 206, 216. For example, as the data collection system 100 is configured to identify anatomical features in the vessel data, the data collection system 100 may be configured to determine a pattern, or sequence, of anatomical features and / or vessel features. The sequence may include, for example, a side branch, a bend, another side branch, a bifurcation, a diameter of the vessel (such as a narrowing or widening of the vessel), one or more nodules of calcium, a proximal or distal end of a previously implanted device (such as a stent), a proximal or distal end of a guide catheter, a take-off angle of a device (such as a stent), etc. The data collection system 100 may compare the features identified in the first and second vessel data and / or representations to determine which, if any, are the same between the first and second vessel data and / or representations. The comparison may include, for example, comparing a sequence of the features to ensure that the features are at the same location within the vessel. Based on the comparison, the data collection system 100 can register the first and second vessel data and link the first and second representations based on the corresponding features.

[0090] In some examples, the linking of the vessel data and / or representations is based on there being a constant offset 8 between the sets of vessel data and / or the representations. For example, with a constant offset 8, a given frame “x” in a first pullback, e.g., a first set of vessel data, will correspond to a frame "x + 8” in a second pullback, e.g., a second set of vessel data. The offset 8 may be determined based on the input, or user instructions, selecting the vessel data and / or frames in the representations that are equivalent.In some examples, the offset 8 may be determined automatically. For each offset 8, the data collection system 100 can be configured to identify a first target feature in the first pullback and a second target feature corresponding to the first target feature in the second pullback. A distance between the idenlil'ied target feature and all other features in each pullback may be determined. Based on the determined distance between the identified target feature and all other features, one or more of the idenlil'ied frames in the first pullback or second pullback may be linearly shifted to line up at least some of the features. Specifically, in some examples the data collection system 100 can be configured to determine a penalty score, corresponding to the sum of displacements from each feature in a first pullback to the nearest feature in a second pullback, and vice versa. The features may be, for example, identifiable vessel features, such as side branches, or other vessel features that are detectable by the data collection system 100. The offset 8 that results in the lowest combined penalty score may be chosen as the offset 8 for the linked vessel data and / or representations. Such general alignment may adjust the order or inclusion of one or more frames in one pullback to generally align with the one or more frames in the second pullback.

[0091] According to some examples, the offset 8 can be generalized to be an offset function 8(n), which would provide a non-constant offset 8 between the sets of vessel data and / or the representations. A nonconstant offset 8 can account for motion artifacts.

[0092] When automatically linking, or registering, the vessel data and / or representations, the data collection system 100 may be configured to compensate for motion artifacts, changes to the vessel between the pullbacks, and the like. For example, if the first pullback was pre-PCI and the second pullback was post-PCI, such as after a stent was implanted, the vessel geometry pre- and post-PCI would be different due to the stent. However, vessel features, such as side branches, bifurcations, and the like, would not change due to PCI. Accordingly, the data collection system 100 may use the unchanged anatomical features for linking the vessel data and / or representations.

[0093] The data collection system 100 may be configured to compensate, or account for, motion artifacts during the pullbacks. During each pullback, the heart is bearing, which causes movement of the heart and vessels. If the catheter is moving in alignment with the heart in each pullback, there will be little to no motion artifacts as the catheter of the device 104 would be stationary relative to the heart and vessels. However, such alignment is not common and, therefore, the data collection system 100 has to account for the movement caused by the beating of the heart. Such movement may be accounted for using a nonconstant offset 8, as described above. For example, once the frames are generally aligned as described above, the data collection system 100 can be configured to locally align individual corresponding frames. In some examples, the data collection system 100 can be configured to identify one or more frames in a first pullback that do not match one or more corresponding frames in a second pullback. In some examples, the mismatched one or more frames may be identified by mapping points corresponding to features and determining that the mapped points do not match. Once this mapping is determined in one direction along the frames, the same mapping may also be used in the other direction along the frames. In some examples, a spline fit may be used to locally align the mismatched frames. The spline fit may include using a spline-based transformation to warp the one or more frames based on the mapped points. The spline fit may be any polynomial function, such as a cubic spline or a quadratic spline.Although Figure 3 depicts image frames at the same, or substantially the same, location in the vessel captured at different times, it will be understood by those of ordinary skill in the art that the methods and systems for linking pullbacks may also be used for image frames at the same, or substantially the same, location in the vessel captured by different intravascular sensing modalities. Intravascular sensing modalities may include an OCT probe, an IVUS catheter, a micro-OCT probe, a NIRS sensor, an OFDI, or any other device that can be used to image or otherwise collect data on a blood vessel. The images from the different medical sensing devices may be captured at the same lime or at different times.Merged Pullbacks

[0094] The data collection system 100 may be configured to merge the pullback captured at different points in time. For example, a first pullback may be captured at a first lime. The first pullback may capture a first length of the vessel. A second pullback may be captured at a second time, different than the first time. The second pullback may capture a second length of the vessel as well as a portion of the first length of the vessel. The data collection system 100 may be configured to merge the first pullback and the second pullback such that one continuous pullback is generated based on the first and second vessel data obtained during the first and second pullbacks, respectively.

[0095] To merge the pullbacks, the data collection system 100 may identify the portion of the second pullback that includes at least a portion of the first length of the vessel. In some examples, the data collection system 100 may identify the region of overlap in a process similar to automatically registering. For example, the data collection system 100 is configured to identify vessel features, such as anatomical features, previously implanted medical devices, etc. The data collection system 100 may identify a pattern or sequence of vessel features in each pullback. Based on a comparison of the sequence of vessel features, the data collection system 100 can determine a portion, if any, of the first and second pullbacks that is overlapping.

[0096] In some examples, the data collection system 100 may be configured to determine a location of the vessel features with respect to a proximal or distal end of the guide catheter. The data collection system 100 may use the position of the vessel features to determine which, if any, are equivalent between the first and second pullback. If the data collection system 100 identi lies any of the vessel features as being equivalent, e.g., present in both the first and second pullback, the data collection system 100 may identify the vessel features and the corresponding length of pullback as being overlapping between the first and second pullback. The data collection system 100 may be configured to link the pullbacks and align the overlapping length as described in connection to Figure 3.

[0097] According to some examples, the data collection system 100 is configured to align the first and second pullbacks to generate a third, merged pullback based on the corresponding vessel features and / or identified portion of the pullbacks that are overlapping. The third pullback, e.g., the merged pullback, may include the first length of the vessel from the first pullback and the second length of the vessel from the second pullback.

[0098] To determine which vessel data to use when generating the merged pullback, the data collection system 100 may determine to keep the some of the vessel data from the first and / or second pullback. For example, in the region of overlap between the first and second pullbacks, the data collection system 100 may identify which vessel data to use and which vessel data to discard when generating the merged pullback. To make this determination, the data collection system would estimate motion artifacts in the overlap region of the first and second pullbacks. The data collection system 100 may be configured to filter the vessel data from the pullbacks in three-dimensions, e.g., three-dimensional convolution along the longitudinal axis of the pullback.

[0099] Typically, when processing signals, the signals are one-dimensional signals and are convoluted along lime. However, for the vessel data captured during the pullbacks, the signals, e.g., vessel image data, is a three-dimensional signal. The three-dimensional signals may be reduced. The three-dimensional signals are convoluted such that the pullbacks are slid across one another such that the pullbacks overlap at least in part and subsequently processed with each other. The processing may include, for example, integrating, mu I liplying, and / or summing the frames of each pullback. As an example, the area of the signals can be determined such that the areas are convolved.

[0100] The result of the merged pullback may be a continuous pullback which can be provided for output via a screen on the GUI or display 118. The continuous pullback includes the first and second length of the respective first and second pullbacks. Accordingly, rather than showing the first and second pullbacks as separate representations, e.g., separate two-dimensional representations, the merged pullback allows for multiple pullbacks to be provided as a single representation.Example Methods

[0101] Figure 4 is a flow diagram for an example method of linking, or registering, two pullbacks captured at different times or by different intravascular sensing modalities. The following operations do not have to be performed in the precise order described below. Rather, various operations can be handled in a different order or simultaneously, and operations may be added or omitted.

[0102] In block 410, first vessel data of a region of interest obtained during a first pullback is received, for example, by the data collection system 100 of Figure 1. The first vessel data may include, for example, intravascular image data, extraluminal image data, data collected via sensors within the vessel, information determined based on intraluminal and / or extraluminal images of the vessel, etc.

[0103] In block 420, second vessel data of at least the region of interest obtained during a second pullback is received, for example, by the data collection system 100 of Figure 1. The second pullback occurs at a different time or is captured by a different intravascular sensing modality than the first pullback . For example, the first pullback may occur pre -PCI, pre-stent, etc. while the second pullback may occur postPCI, post-stent, etc. The second vessel data may, similar to the first vessel data, include, for example, intravascular image data, extraluminal image data, data collected via sensors within the vessel, information determined based on intraluminal and / or extraluminal images of the vessel, etc.

[0104] In block 430, a first two-dimensional representation of the region of interest is generated based on the first vessel data.

[0105] In block 440, a second two-dimensional representation of at least the region of interest is generated based on the second vessel data.

[0106] In block 450, the first and second two-dimensional representations are provided for output. For example, the first and second two-dimensional representations are provided via a GUI, such as display 118 of the data collection system of Figure 1.

[0107] According to some examples, the first vessel data and the second vessel data are registered, or linked. In some examples, the first two-dimensional representation and the second two-dimensional representation are registered or linked. Registering, or linking, the first and second vessel data and / or representations includes, for example, receiving, via the GUI or computing device 112, an input corresponding to a selection of an initial location along the first or second two-dimensional represenlalion. The initial localion may correspond to a location of a vessel feature, such as a side branch, a bifurcation, a bend, a diameter of the vessel (such as a narrowing or widening of the vessel), one or more nodules of calcium, a proximal or distal end of a previously implanted medical device (such as a stent), a proximal or distal end of a guide catheter, a take-off angle of a device (such as a stent), or the like. Another input may be received by the GUI or computing device 112. The other input may be a selection of a comparable location along the other two-dimensional representation. The comparable localion may correspond to the same vessel feature used for selecting the initial localion. For example, the initial localion and the comparable localion are the same, or substantially the same, location within the region of interest of the vessel. The first and second vessel data and / or representations are registered, or linked, based on the selected initial location and comparable localion.

[0108] In some examples, the first and second vessel data and / or representations are registered automatically. For example, the data collection system 100 may be configured to automatically identify comparable, or equivalent, locations in the first and second vessel data and / or representations. As an example, the data collection system 100 may identify vessel features in the first vessel data and / or representation and identify one or more of the vessel features from the first vessel data and / or representation in the second vessel data and / or representation. In some examples, the data collection system may compare the sequence of vessel features in the first and second vessel data and / or representations to determine which are comparable, or equivalent, vessel features. Based on the comparison and the determination that at least some of the vessel features identified in the first and second vessel data and / or representations are comparable, the data collection system 100 can register the first and second vessel data and link the first and second representations.

[0109] In block 460, an input corresponding to a selection of a location along the first or second two-dimensional representation is received. The input may be, for example, a user instruction received via the GUI or computing device 112 of the data collection system 100 of Figure 1.

[0110] In block 470, an equivalent location along the other two-dimensional represenlalion is automatically identified by, for example, the data collection system 100 of Figure 1. For example, if the input is received selecting a location along the first two-dimensional representation, an equivalent location along the second two-dimensional representation is automatically idenli lied. The equivalent location maybe, for example, the location along the vessel in the other, e.g., second, two-dimensional representation corresponding to the selected localion in the first two-dimensional representation.

[0111] In block 480, an indication corresponding to the selected location along the first or second two-dimensional representation and the equivalent location along the other two-dimensional representation is provided for output. For example, the indication of the selected location and the equivalent location may be provided for output on or relative to the first and second two-dimensional representations on the screen of the GUI, such as display 118 of the data collection system 100 of Figure 1. In some examples, a connector may be provided for output. The connector may be a line or other marking to visually link the indications of the selected location and equivalent location. For example, the connector may be a line or other marking that extends between the first location in the first frame and the second location in the second frame.

[0112] According to some examples, a first image frame of the first or second vessel data is provided for output based on the selection of the location along the first or second two-dimensional representation. A second image frame of the other vessel data is provided for output based on the identified equivalent location. For example, if input selecting the location is received on the first two-dimensional representation, an image frame of the first vessel data corresponding to the selected location is provided for output. Continuing with this example, an image frame of the second vessel data corresponding to the equivalent location is provided for output. The image frame of the first vessel data and the image frame of the second vessel data provide a side by side comparison of that location of the vessel captured at two different times or by different intravascular sensing modalities.

[0113] The aspects, features, and examples of the disclosure are to be considered illustrative in all respects and are not intended to limit the disclosure, the scope of which is defined only by the claims. Other examples, modifications, and usages will be apparent to those skilled in the art without departing from the spirit and scope of the claimed invention.

[0114] Throughout the application, where compositions are described as having, including, or comprising specific components, or where processes are described as having, including or comprising specific process steps, it is contemplated that compositions of the present teachings also consist essentially of, or consist of, the recited components, and that the processes of the present teachings also consist essentially of, or consist of, the recited process steps.

[0115] In the application, where an element or component is said to be included in and / or selected from a list of recited elements or components, it should be understood that the element or component can be any one of the recited elements or components and can be selected from a group consisting of two or more of the recited elements or components. Further, it should be understood that elements and / or features of a composition, an apparatus, or a method described herein can be combined in a variety of ways without departing from the spirit and scope of the present teachings, whether explicit or implicit herein.

[0116] The use of the terms “include,” “includes,” “including,” “have,” “has,” or “having” should be generally understood as open-ended and non-limiting unless specifically stated otherwise.

[0117] The use of the singular herein includes the plural (and vice versa) unless specifically stated otherwise. Moreover, the singular forms “a,” “an,” and “the” include plural forms unless the context clearly dictates otherwise. In addition, where the use of the term “about” is before a quantitative value, the present teachings also include the specific quantitative value itself, unless specifically stated otherwise. As used herein, the term “about” refers to a ±10% variation from the nominal value. All numerical values and ranges disclosed herein are deemed to include “about” before each value.

[0118] It should be understood that the order of steps or order for performing certain actions is immaterial so long as the present teachings remain operable. Moreover, two or more steps or actions may be conducted simultaneously.

[0119] Where a range or list of values is provided, each intervening value between the upper and lower limits of that range or list of values is individually contemplated and is encompassed within the invention as if each value were specifically enumerated herein. In addition, smaller ranges between and including the upper and lower limits of a given range are contemplated and encompassed within the invention. The listing of exemplary values or ranges is not a disclaimer of other values or ranges between and including the upper and lower limits of a given range.

Claims

CLAIMS1. A method, comprising:receiving, by one or more processors, first vessel data of a region of interest of a vessel obtained during a first pullback;receiving, by one or more processors, second vessel data of at least the region of interest of the vessel obtained during a second pullback, wherein the second pullback is captured by a different intravascular sensing modality than the first pullback or captured at a different time than the first pullback;generating, by the one or more processors based on the first vessel data, a first two-dimensional representation of the region of interest of the vessel;generating, by the one or more processors based on the second vessel data, a second two-dimensional representation of at least the region of interest;providing for output, by the one or more processors, the first and second two-dimensional representations;identifying, by the one or more processors, a selection of a location along the first or second two-dimensional representation;automatically identifying, by the one or more processors, an equivalent location along the other two-dimensional representation; andproviding for output, by the one or more processors, an indication corresponding to the selected location along the first or second two-dimensional representation and the equivalent location along the other two-dimensional representation.

2. The method of claim 1 , wherein identifying the selection of the location along the first or second two-dimensional representation occurs automatically.

3. The method of claim 1, wherein identifying the selection of the location along the first or second two-dimensional representation comprises receiving, by the one or more processors, an input corresponding to the selection of the location along the first or second two-dimensional representation.

4. The method of claim 1 , wherein the first vessel data and the second vessel data comprise intravascular image data.

5. The method of claim 1, further comprising registering the first vessel data and the second vessel data.

6. The method of claim 5, wherein registering the first and second vessel data comprises: receiving, by the one or more processors, an input corresponding to a selection of an initial location along the first or second two-dimensional representation;receiving, by the one or more processors, an input corresponding to a selection of a comparable location along the other two-dimensional representation, wherein the i nilial location along the first or second two-dimensional representation and the comparable location along the other two-dimensional representation are substantially at a same location within the region of interest of the vessel; and registering, by the one or more processors based on the selection of the initial location and the comparable location, the first and second vessel data.

7. The method of claim 5, wherein when registering the first and second vessel data, the one or more processors are further configured to link the first and second two-dimensional representations.

8. The method of claim 5, wherein registering the first and second vessel data occurs automatically.

9. The method of claim 1 , wherein the first pullback is obtained before percutaneous coronary intervention and the second pullback is obtained post percutaneous coronary intervention.

10. The method of claim 9, wherein the percutaneous coronary intervention comprises at least one of a stent, a balloon, atherectomy, optimization, or angioplasty.

11. The method of claim 1 , further comprising:providing for output, by the one or more processors based on the selection of the location along the first or second two-dimensional representation, a first image frame of the first or second vessel data; andproviding for output, by the one or more processors based on the identified equivalent location along the other two-dimensional representation, a second image frame of the other vessel data.

12. The method of claim 1, further comprising providing for output, by the one or more processors, a connector linking the indication corresponding to the selected location along the first or second two-dimensional representation and the indication corresponding to the equivalent location.

13. A system, comprising:one or more processors, the one or more processors configured to:receive first vessel data of a region of interest of a vessel obtained during a first pullback; receive second vessel data of at least the region of interest of the vessel obtained during a second pullback, wherein the second pullback is captured by a different intravascular sensing modality than the first pullback or captured at a different time than the first pullback;generate, based on the first vessel data, a first two-dimensional representation of theregion of interest of the vessel;generate, based on the second vessel data, a second two-dimensional representation of at least the region of interest;provide for output the first and second two-dimensional representations;identify a selection of a location along the first or second two-dimensional representation;automatically identify an equivalent location along the other two-dimensional representation; andprovide for output an indication corresponding to the selected localion along the first or second two-dimensional representation and the equivalent location along the other two- dimensional representation.

14. The system of claim 13, wherein identifying the selection of the location along the first or second two-dimensional representation occurs automatically.

15. The system of claim 13, wherein identifying the selection of the location along the first or second two-dimensional representation comprises receiving, by the one or more processors, an input corresponding to the selection of the location along the first or second two-dimensional representation.

16. The system of claim 13, wherein the first vessel data and the second vessel data comprise intravascular image data.

17. The system of claim 13, wherein the one or more processors are further configured to register the first vessel data and the second vessel data.

18. The system of claim 15, wherein, when registering the first and second vessel data, the one or more processors are further configured to:receive an input corresponding to a selection of an i nilial location along the first or second two-dimensional representation;receive an input corresponding to a selection of a comparable localion along the other two-dimensional representation, wherein the initial localion along the first or second two-dimensional representation and the comparable location along the other two-dimensional representation are substantially at a same location within the region of interest of the vessel; andregister, based on the selection of the initial location and the comparable location, the first and second vessel data.

19. The system of claim 17, wherein when registering the first and second vessel data, the one or more processors are further configured to link the first and second two-dimensionalrepresentations.

20. The system of claim 17, wherein registering the first and second vessel data occurs automatically.

21. The system of claim 13, wherein the first pullback is obtained before percutaneous coronary intervention and the second pullback is obtained post percutaneous coronary intervention.

22. The system of claim 21, wherein the percutaneous coronary intervention comprises at least one of a stent, a balloon, atherectomy, optimization, or angioplasty.

23. The system of claim 13, wherein the one or more processors are further configured to: provide for output, based on the selection of the location along the first or second two-dimensional representation, a first image frame of the first or second vessel data; andprovide for output, based on the identified equivalent location along the other two-dimensional representation, a second image frame of the other vessel data.

24. The system of claim 13, wherein the one or more processors are further configured to provide for output a connector linking the indication corresponding to the selected location along the first or second two-dimensional representation and the indication corresponding to the equivalent location.

25. One or more non-transitory computer-readable media for storing instructions that, when executed by one or more processors, cause the one or more processors to perform operations comprising:receiving first vessel data of a region of interest of a vessel obtained during a first pullback; receiving second vessel data of at least the region of interest of the vessel obtained during a second pullback, wherein the second pullback is captured by a different intravascular sensing modality than the first pullback or captured at a different time than the first pullback;generating, based on the first vessel data, a first two-dimensional representation of the region of interest of the vessel;generating, based on the second vessel data, a second two-dimensional representation of at least the region of interest;providing for output the first and second two-dimensional representations;identifying a selection of a location along the first or second two-dimensional representation; automatically identifying an equivalent location along the other two-dimensional representation; andproviding for output an indication corresponding to the selected location along the first or second two-dimensional representation and the equivalent location along the other two-dimensionalrepresentation.

26. The non-transitory computer-readable media of claim 25, wherein identifying the selection of the location along the first or second two-dimensional representation occurs automatically .

27. The non-transitory computer-readable media of claim 25, wherein identifying the selection of the location along the first or second two-dimensional representation comprises receiving, by the one or more processors, an input corresponding to the selection of the localion along the first or second two-dimensional representation.

28. The non-transitory computer-readable media of claim 25, wherein the first vessel data and the second vessel data comprise intravascular image data.

29. The non-transitory computer-readable media of claim 25, wherein the operations further comprise registering the first vessel data and the second vessel data.

30. The non-transitory computer-readable media of claim 29, wherein when registering the first and second vessel data the operations further comprise:receiving an input corresponding to a selection of an inilial location along the first or second two-dimensional representation;receiving an input corresponding to a selection of a comparable location along the other two-dimensional representation, wherein the initial location along the first or second two-dimensional representation and the comparable localion along the other two-dimensional representation are substantially at a same location within the region of interest of the vessel; andregistering, based on the selection of the initial location and the comparable location, the first and second vessel data.

31. The non-transitory computer-readable media of claim 29, wherein when registering the first and second vessel data the operations further comprise linking the first and second two-dimensional representations.

32. The non-transitory computer-readable media of claim 29, wherein registering the first and second vessel data occurs automatically.

33. The non-transitory computer-readable media of claim 25, wherein the first pullback is obtained before percutaneous coronary intervention and the second pullback is obtained post percutaneous coronary intervention.

34. The non-transitory computer-readable media of claim 33, wherein the percutaneouscoronary intervention comprises at least one of a stent, a balloon, atherectomy, optimization, or angioplasty.

35. The non-transitory computer-readable media of claim 25, wherein the operations further comprise:providing for output, based on the selection of the location along the first or second two-dimensional representation, a first image frame of the first or second vessel data; andproviding for output, based on the identified equivalent location along the other two-dimensional representation, a second image frame of the other vessel data.

36. The non-transitory computer-readable media of claim 25, wherein the operations further comprise providing for output a connector linking the indication corresponding to the selected location along the first or second two-dimensional representation and the indication corresponding to the equivalent localion.

Images