Alignment of multiple series of intravascular images

Abstract

This disclosure provides a method for processing intravascular ultrasound (IVUS) images from different runs through a vessel to generate a mapping between frames of each IVUS run and a graphical user interface (GUI) that graphically presents the IVUS runs in relation to each other. In some examples, a vascular reference is identified in the frame of each IVUS run, and one or both runs are offset in time, distance, and / or angle so that the frames are aligned to the identified vascular reference. Furthermore, this disclosure provides aligning intravascular images angularly to the viewpoint of external images of the vessel.

Description

Technical Field

[0001] The present disclosure generally relates to intravascular ultrasound (IVUS) imaging systems. In particular, and without limitation, the present disclosure relates to correlating frames in a first series of IVUS images with frames in a second series of IVUS images.

Background Art

[0002] Ultrasonic devices that can be inserted into patients have proven to have diagnostic capabilities for various diseases and disorders. For example, intravascular ultrasound (IVUS) imaging systems are used as an imaging modality to diagnose occluded blood vessels and provide information to assist physicians in selecting and placing stents and other devices to restore or increase blood flow.

[0003] An IVUS imaging system includes a control module (having a pulse generator, image acquisition and processing components, and a monitor), a catheter, and a transducer disposed within the catheter. The catheter containing the transducer is disposed within a lumen or cavity within or proximate to an imaging target area such as the blood vessel wall or patient tissue proximate to the blood vessel wall. The pulse generator within the control module generates an electrical pulse, which is delivered to the transducer and converted into an acoustic pulse that is transmitted through the patient tissue. The patient tissue (or other structure) reflects the acoustic pulse, and the reflected pulse is absorbed by the transducer and converted into an electrical pulse. The converted electrical pulse is delivered to the image acquisition and processing components and converted into an image that can be displayed on the monitor.

[0004] Physicians often capture a series of IVUS images at different stages of treatment. However, conventional tools and systems do not allow physicians to compare these different series of IVUS images beyond providing a specific set of measurements obtained from the images. Therefore, there is a need to provide a graphical interface for correlating IVUS images of the same vessel acquired at different times and displaying these images in relation to one another.

[0005] Machine learning (ML) is the study of computer algorithms that improve through experience. Typically, ML algorithms build models based on sample data called training data. These models can be used to perform inferences (e.g., predictions or decision-making) without being explicitly programmed. As is understood, the quality of the inferences a model makes depends on the training data. Therefore, it is necessary to provide a larger and more complete knowledge corpus for training these ML models. [Overview of the project]

[0006] This summary is provided to introduce, in a simplified form, the selection of concepts that will be further explained in the detailed description below. This summary is not intended to necessarily identify the main or essential features of the subject matter described in the claims, nor is it intended to be an aid in determining the scope of the subject matter described in the claims.

[0007] Generally, the present disclosure provides for processing raw IVUS images, automatically detected lumens and vascular boundaries to identify regions of interest, or more specifically, start and end points that include multiple frames of interest in a series of IVUS images.

[0008] In some embodiments, the present disclosure may be implemented as a method for a computing device. The method may include the processor receiving a first set of intravascular ultrasound (IVUS) images of a patient's blood vessel, the first set of IVUS images comprising a first set of frames; the processor receiving a second set of intravascular ultrasound (IVUS) images of a patient's blood vessel, the second set of IVUS images comprising a second set of frames; the processor determining an offset to the first set of frames based at least partially on the second set of frames; the processor applying the offset to the first set of frames to generate an offset set of IVUS images; and the processor generating a graphical user interface (GUI), the GUI comprising a display of the offset set of IVUS images and the second set of IVUS images.

[0009] In a further embodiment of the method, determining an offset to a first plurality of frames includes identifying one frame of the first plurality of frames including a vascular reference, identifying one frame of a second plurality of frames including a vascular reference, and determining an offset to the first plurality of frames that, when applied, aligns one frame of the first plurality of frames including a vascular reference with one frame of the second plurality of frames including a vascular reference.

[0010] In a further embodiment of the method, the offset includes a first offset and a second offset, wherein determining the offset for a first plurality of frames includes identifying a first frame among the first plurality of frames including a first vascular reference, identifying a second frame among the second plurality of frames including a first vascular reference, determining a first offset for the first plurality of frames, which, when applied to a first segment of the first plurality of frames, aligns the first frame of the first plurality of frames to the first frame of the second plurality of frames, identifying a second frame among the first plurality of frames including a second vascular reference, identifying a second frame among the second plurality of frames including a second vascular reference, determining a second offset for the first plurality of frames, which, when applied to a second segment of the first plurality of frames different from the first segment, aligns the second frame of the first plurality of frames to the second frame of the second plurality of frames, wherein the second offset is different from the first offset.

[0011] In further embodiments of the method, the first offset includes an offset distance and the second offset includes an offset angle, or the first offset includes an offset distance or an offset angle and the second offset includes both an offset distance and an offset angle.

[0012] In a further embodiment of the method, identifying one frame from a first plurality of frames including a vascular criterion, and identifying one frame from a second plurality of frames including a vascular criterion, includes running a machine learning (ML) model to infer one frame from the first plurality of frames including a vascular criterion, and running the ML model to infer one frame from the second plurality of frames including a vascular criterion.

[0013] In further embodiments of the method, the vascular criterion is one of the following: lumen shape, vessel shape, side branch position, calcium morphology, plaque distribution, or guide catheter position. In a further embodiment of the method, determining an offset to a first plurality of frames includes calculating a correlation score for each frame of the first plurality of frames based on the frame-by-frame correlation with a second plurality of frames, identifying one frame of the first plurality of frames having the highest correlation score and one frame of the second plurality of frames associated with the highest correlation score, and determining an offset to the first plurality of frames that, if applied, aligns one frame of the first plurality of frames having the highest correlation score with one frame of the second plurality of frames associated with the highest correlation score.

[0014] In a further embodiment of the method, the offset is an offset distance, and the method further includes: calculating a correlation score for each frame of the first plurality of frames based on frame-by-frame correlation with an angular offset with a second plurality of frames; identifying one of the first plurality of frames having the highest correlation score and one of the second plurality of frames or a rotated version of the frame associated with the highest correlation score; and determining an offset angle relative to the first plurality of frames, which, when applied, aligns one of the first plurality of frames having the highest correlation score with one of the second plurality of frames associated with the highest correlation score, based on one of the second plurality of frames or a rotated version of the frame associated with the highest correlation score, and a series of offset IVUS images is generated by applying the offset distance and offset angle to the first plurality of frames.

[0015] In a further embodiment of the method, determining an offset to a first plurality of frames includes calculating a correlation score for each frame of the first plurality of frames based on frame-by-frame correlation with an angular offset with a second plurality of frames; identifying one of the first plurality of frames having the highest correlation score and one of the second plurality of frames or a rotated version of the frame associated with the highest correlation score; and determining an offset to the first plurality of frames that, when applied, aligns one of the first plurality of frames having the highest correlation score with one of the second plurality of frames associated with the highest correlation score.

[0016] In a further embodiment of the method, the offset of the first plurality of frames is a distance offset, an angle offset, or a distance and angle offset. In further embodiments, the method may include receiving a second series of IVUS images from an intravascular imaging device and receiving a first series of IVUS images from a memory storage device.

[0017] In a further embodiment of the method, a first series of IVUS images are captured during percutaneous coronary intervention (PCI) preparation. In a further embodiment of the method, a second series of IVUS images are captured during the pre-PCI procedure, the PCI procedure, and the post-PCI procedure, or during the post-PCI procedure.

[0018] In a further embodiment of the method, the GUI includes longitudinal views of a first set of IVUS images and a second set of IVUS images, the longitudinal views being set to a common scale.

[0019] In some embodiments, the present disclosure may be implemented as an apparatus for an intravascular imaging system. The apparatus may comprise a processor and a memory device coupled to the processor, the memory device containing a set of instructions executable by the processor, which, when executed by the processor, cause the intravascular imaging system to perform any of the methods outlined herein.

[0020] In some embodiments, the present disclosure can be implemented as at least one machine-readable storage device. The at least one machine-readable storage device may comprise a plurality of instructions that, in response to being executed by a processor of an intravascular ultrasound (IVUS) imaging system, cause the processor to perform any of the methods outlined herein.

[0021] In some embodiments, the present disclosure may be implemented as an apparatus for an intravascular imaging system. The apparatus may include an apparatus for an intravascular imaging system, the apparatus for an intravascular imaging system comprising a display, a processor coupled to the display, and a memory device coupled to the processor, the memory device comprising a set of instructions executable by the processor, the set of instructions, when executed by the processor, causes the intravascular imaging system to receive a first set of intravascular ultrasound (IVUS) images of a patient's blood vessel, the first set of IVUS images comprising a first set of frames, receive a second set of intravascular ultrasound (IVUS) images of a patient's blood vessel, the second set of IVUS images comprising a second set of frames, determine an offset to the first set of frames based at least partially on the second set of frames, apply the offset to the first set of frames to generate an offset set of IVUS images, generate a graphical user interface (GUI), the GUI includes displaying the offset set of IVUS images and the second set of IVUS images, and display the GUI on the display.

[0022] In a further embodiment of the apparatus, the command set causes the intravascular imaging system to identify one frame from a first plurality of frames including a vascular reference, identify one frame from a second plurality of frames including a vascular reference, and determine an offset relative to the first plurality of frames that, when applied, aligns one frame from the first plurality of frames including a vascular reference with one frame from the second plurality of frames including a vascular reference.

[0023] In a further embodiment of the apparatus, the offset includes a first offset and a second offset, and the set of commands causes the intravascular imaging system to further identify a first frame of a first plurality of frames including a first vascular reference, identify a second frame of a second plurality of frames including a first vascular reference, determine a first offset to the first plurality of frames which, when applied to a first segment of the first plurality of frames, aligns the first frame of the first plurality of frames to the first frame of the second plurality of frames, identify a second frame of a first plurality of frames including a second vascular reference, identify a second frame of a second plurality of frames including a second vascular reference, and determine a second offset to the first plurality of frames which, when applied to a second segment of the first plurality of frames different from the first segment, aligns the second frame of the first plurality of frames to the second frame of the second plurality of frames, wherein the second offset is different from the first offset.

[0024] In a further embodiment of the apparatus, the first offset includes an offset distance and the second offset includes an offset angle, or the first offset includes an offset distance or an offset angle and the second offset includes both an offset distance and an offset angle.

[0025] In a further embodiment of the device, the instruction set causes the intravascular imaging system to execute a machine learning (ML) model to infer one frame out of a first plurality of frames including vascular criteria, and further causes the ML model to execute to infer one frame out of a second plurality of frames including vascular criteria.

[0026] In a further embodiment of the device, the vascular criteria is one of a lumen shape, a vascular shape, a collateral position, a calcium morphology, a plaque distribution, or a guide catheter position. In some embodiments, the present disclosure can be implemented as at least one machine-readable storage device. The at least one machine-readable storage device, in response to being executed by a processor of an intravascular ultrasound (IVUS) imaging system, causes the processor to receive a first series of intravascular ultrasound (IVUS) images of a patient's blood vessel, the first series of IVUS images including a first plurality of frames; receive a second series of intravascular ultrasound (IVUS) images of the patient's blood vessel, the second series of IVUS images including a second plurality of frames; determine an offset for the first plurality of frames based at least in part on the second plurality of frames; apply the offset to the first plurality of frames to generate an offset series of IVUS images; generate a graphical user interface (GUI), the GUI including a display of the offset series of IVUS images and the second series of IVUS images; and transmit the GUI to a display coupled to the IVUS imaging system, and can include a plurality of instructions that cause the above to be performed.

[0027] In a further embodiment of at least one machine-readable storage device, the execution of a set of instructions causes the IVUS imaging system to calculate a correlation score for each frame of the first plurality of frames based on its frame-by-frame correlation with a second plurality of frames; to identify one of the first plurality of frames having the highest correlation score and one of the second plurality of frames associated with the highest correlation score; and, if applied, to determine an offset to the first plurality of frames that aligns one of the first plurality of frames having the highest correlation score with one of the second plurality of frames associated with the highest correlation score.

[0028] In a further embodiment of at least one machine-readable storage device, the offset is an offset distance, and the execution of the instruction set causes the IVUS imaging system to calculate a correlation score for each frame of the first plurality of frames based on frame-by-frame correlation with an angular offset with a second plurality of frames; to identify one of the first plurality of frames having the highest correlation score and one of the second plurality of frames or a rotated version of the frame associated with the highest correlation score; and to further determine an offset angle relative to the first plurality of frames that, when applied, aligns one of the first plurality of frames having the highest correlation score with one of the second plurality of frames associated with the highest correlation score, and a series of offset IVUS images are generated by applying the offset distance and offset angle to the first plurality of frames.

[0029] In a further embodiment of at least one machine-readable storage device, the execution of the set of instructions causes the IVUS imaging system to calculate a correlation score for each frame of the first plurality of frames based on a per-frame correlation with an angular offset from a second plurality of frames, identify one frame of the first plurality of frames having the highest correlation score and one frame or a rotated version of a frame of the second plurality of frames associated with the highest correlation score, and determine an offset for the first plurality of frames that, when applied, aligns one frame of the first plurality of frames having the highest correlation score with one frame of the second plurality of frames associated with the highest correlation score, based on one frame or a rotated version of a frame of the second plurality of frames associated with the highest correlation score.

[0030] In a further embodiment of at least one machine-readable storage device, the offset of the first plurality of frames is a distance offset, an angular offset, or a distance and angular offset.

Brief Description of the Drawings

[0031] To facilitate identification of any element or operation, the most significant digit of the reference number refers to the drawing number in which that element first appears. [Figure 1] FIG. 1 is a diagram showing an IVUS imaging system according to an embodiment of the present disclosure. [Figure 2] FIG. 2 is a diagram showing an exemplary angiographic image of a blood vessel. [Figure 3A] FIG. 3A is a diagram showing an IVUS image of a blood vessel. [Figure 3B] FIG. 3B is a diagram showing an IVUS image of a blood vessel. [Figure 4] FIG. 4 is a diagram showing an IVUS image correlation and visualization system according to at least one embodiment of the present disclosure. [Figure 5A]Figure 5A shows an exemplary frame-by-frame correlation between a frame in one set of IVUS images and a frame in another set of IVUS images, according to at least one embodiment of the present disclosure. [Figure 5B] Figure 5B shows a plot of correlation scores that can be generated according to the exemplary frame-by-frame correlation in Figure 5A. [Figure 6A] Figure 6A shows another exemplary frame-by-frame correlation between a frame in a set of IVUS images and a frame from another set of IVUS images, as well as an angularly offset version of the frame, according to at least one embodiment of the present disclosure. [Figure 6B] Figure 6B shows a plot of correlation scores that can be generated according to the exemplary frame-by-frame correlation in Figure 6A. [Figure 7] Figure 7 shows another IVUS image correlation and visualization system according to at least one embodiment of the present disclosure. [Figure 8A] Figure 8A shows a series of IVUS images of aligned blood vessels based on at least one embodiment of the present disclosure. [Figure 8B] Figure 8B shows a series of IVUS images of aligned blood vessels based on at least one embodiment of the present disclosure. [Figure 9A] Figure 9A shows exemplary segment-by-segment alignment and offset identification according to at least one embodiment of the present disclosure. [Figure 9B] Figure 9B shows exemplary segment-by-segment alignment and offset identification according to at least one embodiment of the present disclosure. [Figure 10A] Figure 10A shows another exemplary segment-by-segment alignment and offset specification according to at least one embodiment of the present disclosure. [Figure 10B] Figure 10B shows another exemplary segment-by-segment alignment and offset specification according to at least one embodiment of the present disclosure. [Figure 11]Figure 11 shows a graphical user interface (GUI) according to at least one embodiment of the present disclosure. [Figure 12] Figure 12 shows a logic flow for determining the mapping between different IVUS runs of a blood vessel according to at least one embodiment of the present disclosure. [Figure 13] Figure 13 shows time warping along the longitudinal offset of two sets of IVUS images based on extracted and vectorized features of a set of IVUS images, according to at least one embodiment of the present disclosure. [Figure 14] Figure 14 shows an exemplary machine learning (ML) system suitable for use with the exemplary embodiments of this disclosure. [Figure 15] Figure 15 shows another IVUS image correlation and visualization system according to at least one embodiment of the present disclosure. [Figure 16A] Figure 16A shows exemplary extravascular images and identified criteria. [Figure 16B] Figure 16B shows an example of an IVUS run frame rotated to align with the reference angle observed in the external image of Figure 16A. [Figure 16C] Figure 16C shows an example of an IVUS run frame rotated to align with the reference angle observed in the external image of Figure 16A. [Figure 16D] Figure 16D shows an example of an IVUS run frame rotated to align with the reference angle observed in the external image of Figure 16A. [Figure 17] Figure 17 shows a GUI according to at least one embodiment of the present disclosure. [Figure 18] Figure 18 shows a computer-readable storage medium. [Figure 19] Figure 19 is a schematic diagram of the machine. [Modes for carrying out the invention]

[0032] Having broadly described the features and technical advantages of this disclosure above, the following detailed description of this disclosure will be better understood. Those skilled in the art will understand that the disclosed embodiments can readily be used as a basis for modifying or designing other structures to accomplish the same objectives of this disclosure. Novel features of this disclosure, both in terms of their configuration and operation, will be better understood from the following description, when considered in relation to the accompanying drawings, along with further objectives and advantages. However, it should be expressly understood that each drawing is provided for illustrative and explanatory purposes only and is not intended to define the limitations of this disclosure.

[0033] As stated above, this disclosure relates to processing IVUS recordings, or in other words, processing a series of IVUS images, with respect to patient IVUS images and lumens (e.g., blood vessels). Accordingly, an exemplary IVUS imaging system, a patient's blood vessel, and a series of IVUS images will be described.

[0034] A suitable IVUS imaging system includes, but is not limited to, one or more transducers positioned at the distal end of a catheter configured and positioned for percutaneous insertion into a patient.

[0035] Figure 1 schematically shows one embodiment of the IVUS imaging system 100. The IVUS imaging system 100 includes a catheter 102 that can be connected to a control system 104. The control system 104 may include, for example, a processor 106, a pulse generator 108, and a drive unit 110. In at least some embodiments, the pulse generator 108 generates electrical pulses that can be input to one or more transducers (not shown) placed inside the catheter 102.

[0036] In some embodiments, mechanical energy from the drive unit 110 can be used to drive an imaging core (not shown) located within the catheter 102. In at least some embodiments, electrical signals transmitted from one or more transducers can be input to a processor 106 for processing. In at least some embodiments, the processed electrical signals from one or more transducers can be used to form a series of images, which are described in more detail below. For example, a scan converter can be used to map a scan line sample (e.g., a radial scan line sample) to a two-dimensional orthogonal grid, which can be used as the basis for a series of IVUS images that can be displayed to the user.

[0037] In at least some embodiments, the processor 106 may also be used to control one or more functions of other components of the control system 104. For example, the processor 106 may be used to control at least one of the frequency or duration of electrical pulses transmitted from the pulse generator 108, or the rotational speed of the imaging core by the drive unit 110. In addition, if the IVUS imaging system 100 is configured for automatic retraction, the drive unit 110 may control the speed and / or length of the retraction.

[0038] Figure 2 shows an extravascular image 200 of a patient's blood vessel 202. As described, an IVUS imaging system (e.g., IVUS imaging system 100) is used to capture a series of intraluminal images or "records" of a blood vessel, such as blood vessel 202. For example, when an IVUS catheter (e.g., catheter 102) is inserted into blood vessel 202 and catheter 102 is withdrawn from its distal end 204 to its proximal end 206, a recording or series of IVUS images is captured. Catheter 102 can be withdrawn manually or automatically (e.g., under the control of a drive unit 110). A series of IVUS images captured between the distal end 204 and the proximal end 206 is often referred to as images from an IVUS run.

[0039] Figures 3A and 3B show two-dimensional (2D) representations of IVUS images of vessel 202. For example, Figure 3A shows an IVUS image 300a showing a longitudinal view of the IVUS recording of vessel 202 between its proximal end 206 and distal end 204.

[0040] Figure 3B shows image frame 300b, which displays an on-axial (or short-axis) view of vessel 202 at point 302. In other words, image frame 300b is a single frame or single image from a series of IVUS images that can be captured between the distal end 204 and the proximal end 206, as described herein. As mentioned above, physicians often capture IVUS runs (e.g., a series of IVUS images) at different stages of treatment. For example, IVUS images may be captured before a percutaneous coronary intervention (PCI) procedure or after a PCI procedure (e.g., stent placement, balloon dilation, rotavation, etc.) has been performed.

[0041] This disclosure provides a graphical user interface that allows IVUS runs at different time frames to be aligned frame by frame and correlated, enabling physicians to observe correlated IVUS runs and gain a more direct understanding of vascular treatment, for example, by observing differences in lesion characteristics in parallel comparisons.

[0042] Figure 4 shows an IVUS image correlation and visualization system 400 according to several embodiments of the present disclosure. Generally, the IVUS image correlation and visualization system 400 is a system for processing, correlating, and presenting a series of IVUS images of the same vessel. The IVUS image correlation and visualization system 400 can be implemented in a commercially available IVUS guidance or navigation system, such as the AVVIGO® guidance system available from Boston Scientific®. The present disclosure offers advantages over conventional IVUS navigation systems in that no prior or conventional system has provided the ability to correlate IVUS runs performed at different times.

[0043] In some embodiments, the IVUS image correlation and visualization system 400 may be implemented as part of the control system 104. Alternatively, the control system 104 may be implemented as part of the IVUS image correlation and visualization system 400. As shown in the figure, the IVUS image correlation and visualization system 400 includes a computing device 402. Optionally, the IVUS image correlation and visualization system 400 includes an IVUS imaging system 100 and a display 404.

[0044] While IVUS is frequently used as an exemplary intravascular imaging modality in this disclosure, it should be noted that this disclosure may also be provided for aligning frames from different captured runs longitudinally and / or angularly using a variety of other intravascular imaging modalities, such as optical coherence tomography (OCT).

[0045] The computing device 402 can be any of a variety of computing devices. In some embodiments, the computing device 402 may be integrated into and / or implemented by the console of the display 404. In some embodiments, the computing device 402 may be a workstation or server communicatively connected to the IVUS imaging system 100 and / or the display 404. In yet other embodiments, the computing device 402 may be provided by a cloud-based computing device, such as computing as a service system accessible over a network (e.g., the Internet, an intranet, or a wide area network). The computing device 402 may include a processor 406, memory 408, input and / or output (I / O) devices 410, a network interface 412, and IVUS imaging system acquisition circuitry 414.

[0046] Processor 406 may include circuitry or processor logic, such as one of various commercially available processors. In some examples, processor 406 may include multiple processors, multithreaded processors, multicore processors (whether the multiple cores coexist on the same or separate dies), and / or other types of multiprocessor architectures in which multiple physically separate processors are linked in some way. In addition, in some examples, processor 406 may include a graphics processing portion, as well as dedicated memory, multithreading, and / or other parallel processing capabilities. In some examples, processor 406 may be an application-specific integrated circuit (ASIC) or a field-programmable integrated circuit (FPGA).

[0047] Memory 408 may include logic, part of which may include an array of integrated circuits forming a non-volatile memory for persistently storing data, or a combination of non-volatile and volatile memory. It should be understood that memory 408 may be based on any of various technologies. In particular, the array of integrated circuits included in memory 120 may be arranged to form one or more types of memory, such as dynamic random access memory (DRAM), NAND memory, or NOR memory.

[0048] The I / O device 410 can be any of a variety of devices for receiving input and / or providing output. For example, the I / O device 410 may include a keyboard, mouse, joystick, foot pedal, display, touch-enabled display, haptic feedback device, LED, etc.

[0049] The network interface 412 may include logic and / or functions to support communication interfaces. For example, the network interface 412 may include one or more interfaces that operate according to various communication protocols or standards for communication directly or over a network communication link. Direct communication may be achieved through the use of communication protocols or standards described in one or more industry standards (including derivatives and variations). For example, the network interface 412 may enable communication via buses such as PCIe (Peripheral Component Interconnect Express), NVMe (Non-Volatile Memory Express), Universal Serial Bus (USB), System Management Bus (SMBus), SAS (e.g., Serial Attached Small Computer System Interface (SCSI) interface, Serial AT Attachment (SATA) interface). Furthermore, the network interface 412 may include logic and / or functions that enable communication via various wired or wireless network standards (e.g., 902.11 communication standards). For example, the network interface 412 may be configured to support wired communication protocols or standards such as Ethernet®. As another example, the network interface 412 may be configured to support wireless communication protocols or standards such as Wi-Fi, Bluetooth®, ZigBee®, LTE, 5G, etc.

[0050] The IVUS imaging system acquisition circuit 414 may include a custom-made or specially programmed circuit configured to receive or transmit signals to and from the IVUS imaging system 100, including signals including IVUS runs, a series of IVUS images, or indications of one or more frames of IVUS images.

[0051] Memory 408 may contain instruction sets 416. During operation, the processor 406 can execute instruction sets 416 to cause the computing device 402 to receive a series of IVUS images from multiple IVUS runs of a blood vessel (e.g., from an IVUS imaging system 100, etc.) and save the recordings in memory 408 as IVUS image 418a, IVUS image 418b, etc. For example, the processor 406 can execute instruction sets 416 to receive information elements from the IVUS imaging system 100, including a display of IVUS images captured while the catheter 102 is being withdrawn from the distal end 204 to the proximal end 206, and these images include a display of the anatomical structure and / or structure of the blood vessel 202, including the blood vessel wall and plaque. Furthermore, it should be understood that the processor 406 can execute instruction sets 416 to receive IVUS images from multiple runs through the blood vessel (e.g., pre-PCI, post-PCI, at different times, etc.). It should be understood that IVUS images 418a and 418b may be stored in various image formats, or even in non-image formats or data structures that include representations of vessels 202. Furthermore, IVUS images 418a and 418b may contain multiple "frames" or individual images, which can be collinearly represented to form an image of vessels 202, such as that represented by IVUS image 300a.

[0052] This disclosure provides a method for correlating IVUS images 418a and 418b frame by frame and presenting a correlated view of the images in a graphical user interface. In some examples, a processor 406 can execute a set of instructions 416 to identify the IVUS run frame mapping 420 from IVUS images 418a and 418b using a machine learning (ML) model and infer the mapping (see, for example, Figure 4). In some embodiments, a processor 406 can execute a set of instructions 416 to identify the IVUS run frame mapping 420 from IVUS images 418a and 418b using frame-by-frame correlation or segment-by-segment correlation (see, for example, Figure 4). In some embodiments, this can be enabled using standard image processing techniques and / or ML inference. In other examples, a memory 408 can execute a set of instructions 416 to determine one or more fiducials (e.g., via machine learning, via an image processing algorithm, etc.) and determine the IVUS run frame mapping 420 from the identified fiducials (see, for example, Figure 7). Each method for determining the IVUS run frame mapping 420 differs slightly and is described separately, but once the IVUS run frame mapping 420 is identified, the methods become similar. Furthermore, in some embodiments, the IVUS run may be mapped to and / or aligned with angiographic images of the vessels (see, e.g., Figure 6). It is important to note that Figures 4, 7, and 15 show the IVUS image correlation and visualization systems 400, 700, and 1500, respectively. This is done for clarity when describing the various alignment techniques disclosed herein. However, it is important to note that the alignment technique described with respect to one system (e.g., 400) may be used in combination with the alignment technique disclosed with respect to another system (e.g., 700 and / or 1500).For example, it's important to understand that one alignment technique can be used to align a frame longitudinally, while another technique can be used to align a frame angularly.

[0053] Refer to Figure 4 here. Once the IVUS run frame mapping 420 is generated, a frame-by-frame correlation between IVUS image 418a and IVUS image 418b can be generated from the IVUS run frame mapping 420. For example, memory 408 can execute instruction set 416 to correlate each frame of IVUS image 418a with the corresponding frame of IVUS image 418b. Furthermore, memory 408 can execute instruction set 416 to generate a graphical user interface (GUI) 424 that displays the correlated and / or related frames of IVUS image 418a based on the IVUS run frame mapping 420.

[0054] In an example where ML is used to generate an IVUS run frame mapping 420, the processor 406 can execute instruction set 416 to run or "run" the ML model 422 using IVUS images 418a and 418b as input to generate the IVUS run frame mapping 420. The ML model 422 can infer the IVUS run frame mapping 420 from IVUS images 418a and 418b. Memory 408 stores a copy of the ML model 422, and the processor 406 can execute the ML model 422 to generate the IVUS run frame mapping 420. In general, the ML model 422 can be any of various ML models. Examples of ML models as assumed herein, as well as examples of training ML models, are described below.

[0055] In some embodiments, the disclosure may be provided for aligning IVUS runs based on the correlation between each frame of one IVUS run and all frames of another IVUS run. The processor 406 can execute a set of instructions 416 to determine the frame-by-frame correlation 426 of the IVUS runs. For example, the processor 406 can execute a set of instructions 416 to iterate through each frame of the IVUS image 418a and calculate the correlation between all frames of the IVUS image 418b (for example, using criteria, using ML, using background subtraction, using cross-correlation, or equivalent). The processor 406 can then execute a set of instructions 416 to identify the most closely correlated frame in the IVUS image 418b for each frame in the IVUS image 418a. For example, Figure 5A shows image frame 502 (for example, from IVUS image 418a, etc.) and image frames 504a, 504b, 504c, etc. (for example, from IVUS image 418b, etc.). The processor 406 can execute instruction set 416 to calculate the correlation (e.g., correlation value, score, etc.) between image frame 502 and image frames 504a, 504b, 504c, etc. Figure 5B shows the plot 506 of the calculated correlation. Plot 506 graphs the correlation score between a specific frame from one set of IVUS images (e.g., image frame 502) and frames from another set of IVUS images (e.g., frames 504a, 504b, 504c, etc.). As can be seen from the figure, the correlation score values ​​are plotted on the y-axis 508, and the frame numbers from the second set of IVUS images are plotted on the x-axis 510.

[0056] In some embodiments, frame-by-frame correlations may be determined for each frame at different rotation angles. The processor 406 may execute instruction set 416 to identify the correlation between each frame in one set of IVUS images (e.g., IVUS image 418a) and a frame from another set of IVUS images (e.g., IVUS image 418b) at multiple rotation angles. Figure 6A shows an example of this. During operation of the IVUS image correlation and visualization system 400, the processor 406 may execute instruction set 416 to calculate correlation scores between a frame from one set of IVUS images (e.g., image frame 502 from IVUS image 418a) and a frame from another set of IVUS images (e.g., image frame 504a from IVUS image 418b) and rotated versions of the image frames (e.g., rotated image frames 602a and 602b). Similar to the frame-by-frame correlation described above, processor 406 may execute instruction set 416 to calculate a correlation score between each frame from one set of IVUS images (e.g., IVUS image 418a) and each frame from another set of IVUS images (e.g., IVUS image 418b, etc.) and its rotated version. Figure 6B shows a plot 604 of the calculated correlations. Plot 604 graphs the correlation score between a specific image frame from one set of IVUS images (e.g., image frame 502) and an image frame from another set of IVUS images (e.g., image frame 504a) and its rotated version (e.g., rotated image frames 602a and 602b). As can be seen from the figure, the correlation score values ​​are plotted on the y-axis 606 and the rotation angle is plotted on the x-axis 608.

[0057] In some examples, the processor 406 may execute instruction set 416 to generate rotated image frames at all possible rotation angles (e.g., rotated image frames 602a, 602b, etc.). In such examples, 359 rotated image frames are generated. In other examples, the processor 406 may execute instruction set 416 to generate rotated image frames at all possible rotation angles subsets (e.g., every 2 degrees, every 5 degrees, every 10 degrees, every 15 degrees, every 20 degrees, every 30 degrees, every 45 degrees, etc.).

[0058] In general, the IVUS run frame mapping 420 may include instructions for offsets (e.g., time, distance, rotation, etc.) to be adjusted to align one (or each) of the IVUS images 418a and IVUS images 418b. As used herein, the term align means aligning the frames of the images longitudinally and / or angularly.

[0059] In some embodiments, the processor 406 may execute a set of instructions 416 to receive one bookmark (or more bookmarks) that identifies one frame from IVUS image 418a and / or IVUS image 418b. The IVUS run frame mapping 420 may be adjusted to align one or more bookmarks. In some embodiments, this mapping is not linear. For example, a frame from IVUS image 418a may be adjusted linearly (e.g., by a first distance) and / or rotated (e.g., by a first angle) based on its correlation with a frame from IVUS image 418b, while adjacent frames within IVUS image 418a may be adjusted linearly (e.g., by a second distance) and / or rotated (e.g., by a second angle) based on its correlation with the same or different frames from IVUS image 418b.

[0060] Figure 7 shows an IVUS image correlation and visualization system 700 according to several embodiments of the present disclosure. Generally, the IVUS image correlation and visualization system 700, like the IVUS image correlation and visualization system 400, is a system for processing, correlating, and presenting multiple sets of IVUS images of the same vessel. For simplicity of explanation, many of the components of the IVUS image correlation and visualization system 400 are referenced and reused in the IVUS image correlation and visualization system 700.

[0061] As described above with respect to Figure 4 and the IVUS image correlation and visualization system 400, the disclosure provides for generating IVUS run-frame mappings 420 for IVUS images 418a and IVUS images 418b. In some embodiments, a processor 406 may execute a set of instructions 416 to identify vascular criteria 702 in IVUS images 418a and IVUS images 418b. In some embodiments, vascular criteria 702 may be anatomical criteria of one or more coronary arteries (e.g., lumen shape, vessel shape, side branch location, calcium morphology, plaque distribution, guide catheter location, etc.). In some embodiments, the processor 406 may execute a set of instructions 416 to identify vascular criteria 702 from IVUS images 418a and IVUS images 418b using an image processing algorithm (e.g., a geometric image recognition algorithm for identifying lumen profiles, etc.). In other embodiments, a memory 408 may include one or more ML models 704 configured to infer vascular criteria 702 from IVUS images (e.g., IVUS images 418a, 418b, etc.). For example, memory 408 may include ML model 704, which may include one or more ML models trained to infer criteria (e.g., side branch position, calcium morphology, guide catheter position, etc.). Thus, processor 406 can run ML model 704 to identify vascular criteria 702 in the frames of IVUS images 418a and IVUS images 418b.

[0062] The processor 406 may generate an IVUS run frame mapping 420 from vascular criteria 702 by executing a set of instructions 416 to pair, for example, frames from IVUS images 418a and 418b in which the same anatomical criteria are identified. Once the IVUS run frame mapping 420 is obtained, the processor 406 may execute a set of instructions 416 to correlate each frame of IVUS image 418a with the corresponding frame of IVUS image 418b. Furthermore, the processor 406 may execute a set of instructions 416 to generate a GUI 424 that displays the correlated and / or related frames of IVUS image 418a based on the IVUS run frame mapping 420.

[0063] It should be understood that in some embodiments, the processor 406 may execute instruction set 416 to identify a criterion within a single frame of each IVUS run (e.g., IVUS images 418a and 418b). For example, the vascular criterion 702 may include a side branch identified in the frame of IVUS image 418a and the same side branch identified in the frame of IVUS image 418b. In other embodiments, the processor 406 may execute instruction set 416 to identify multiple criteria within multiple frames. In such examples, the multiple criteria do not need to be identical. For example, as described above, the vascular criterion 702 may include the side branch position in the frame of IVUS image 418a and the same side branch position in the frame of IVUS image 418b, as well as the guide catheter position in another frame of IVUS image 418a and the guide catheter position in another frame of IVUS image 418b. Multiple examples are not limited to this relationship.

[0064] Figure 8A shows multiple IVUS runs set for scale 802. Multiple sets of IVUS runs, i.e., IVUS images 804a, 804b, and 804c, are shown in the figure. It should be understood that each set of IVUS images (e.g., the set of IVUS images 804a, 804b, and 804c) contains multiple frames. As described above, in some embodiments, the criterion is identified within one or more frames of each set of IVUS images. This figure shows the criterion 806 identified within the frames of each set of IVUS images 804a, 804b, and 804c. The IVUS run frame mapping 420 may be generated from frames identified as representing (representing, corresponding to, depicting, etc.) the criterion 806. For example, the processor 406 may execute a set of instructions 416 to identify frames (e.g., frames from images 804a, 804b, and 804c, etc.) that contain the vascular criterion 806 (or vascular criterion) from each set of IVUS images. The processor 406 can execute a set of instructions 416 to identify an offset relative to the frames of a set (or more sets) of IVUS images, and when this offset is applied, the frames in each set of IVUS images are aligned on the scale 802. In some embodiments, the offset may be a time offset, a distance offset, an angular offset, or any combination of time offset, distance offset, and / or angular offset. Furthermore, it should be understood that the scale 802 may be any scale on which an IVUS run is represented or graphically presented. For example, some IVUS runs are graphically displayed relative to a retraction scale having distal and proximal points along the retraction. As an example, the retraction scale may be expressed in units of distance (e.g., millimeters). With respect to the offset, the offset may be generated such that frames identified as representing the same reference (e.g., vascular reference 806) are shifted or adjusted so that when the offset is applied, the frames are aligned on the scale.

[0065] For example, Figure 8B shows multiple IVUS images from Figure 8A, re-set to scale 802. However, frames from the set of IVUS images 804a and 804c are adjusted based on identified offsets (e.g., by IVUS run frame mapping 420) to align frames that show a reference on scale 802. For example, IVUS image 804a is adjusted by offset 808a so that IVUS image 804a is shifted relative to scale 802, while IVUS image 804c is adjusted by offset 808b so that IVUS image 804c is shifted relative to scale 802. By applying offsets 808a and 808b to IVUS images 804a and 804c, respectively, the IVUS images are aligned to scale 802, and in particular, frames showing vascular reference 806 in IVUS images 804a, 804b, and 804c are aligned relative to scale 802. For example, as shown in this figure, when IVUS images 804a, 804b, and 804c are aligned based on offsets 808a and 808b, the reference 806 identified in each IVUS run is aligned. Note that offsets 808a and 808b are illustrated as longitudinal offsets, i.e., as the distance that offsets the frame along scale 802. However, offsets 808a and 808b could instead be offset angles (e.g., the angle that rotates the frame), or both offset distance and offset angle. Furthermore, although only a single offset is shown per IVUS run in this specification (e.g., offset 808a for IVUS image 804a and offset 808b for IVUS image 804c), note that multiple offsets may be provided for each run (e.g., for multiple frames within a segment, for each frame, for only a portion of the frames, etc.).

[0066] Various techniques and workflows are provided for identifying the longitudinal and / or angular offset of frames in a set of IVUS images (e.g., IVUS image 418a) and aligning the frames with frames in another set of IVUS images (e.g., IVUS image 418b). While Figures 8A and 8B show only longitudinal alignment, it should be noted that this disclosure can be performed to align IVUS runs longitudinally, angularly, and / or longitudinally and angularly.

[0067] In some embodiments, the processor 406 may execute a set of instructions 416 to longitudinally align frames from IVUS image 418a segment by segment with respect to frames from IVUS image 418b. For example, in some embodiments, the processor 406 may execute a set of instructions 416 to identify segments based on vascular criteria 702. Figure 9A shows IVUS image 418a and identified criteria 902a and 902b. As described above, these criteria may include side branches, lumen shape, vessel shape, calcium morphology, plaque distribution, etc. The processor 406 may execute a set of instructions 416 to group frames from IVUS image 418a into segments based on identified criteria 902a and 902b. For example, Figure 9A shows frames from IVUS image 418a grouped into segments 904a, 904b, and 904c. Therefore, frame offsets in a set of IVUS images (e.g., IVUS image 418a) can be generated for different segments using the identified vascular reference 702.

[0068] Figure 9B shows points representing the longitudinal offsets of each frame corresponding to references 902a and 902b. From these points, a plot 906 can be generated representing the longitudinal offset (plotted on the y-axis 908) for each frame (plotted on the x-axis 910) in IVUS run 418a. In some embodiments, plot 906 may be generated linearly between points (for example, as shown in Figure 9B). In other embodiments, the processor 406 may execute a set of instructions 416 to generate plot 906 based on one or more line fitting algorithms (e.g., raster-based line fitting). The longitudinal offsets for each frame in segments 904a, 904b, and 904c may be determined based on plot 906.

[0069] In some embodiments, the processor 406 may execute a set of instructions 416 to align frames from IVUS image 418a to frames from IVUS image 418b in a rotational direction based on vascular reference 702. For example, in some embodiments, the IVUS run frame mapping 420 may include an offset angle (e.g., for rotating frames). Figure 10A shows IVUS image 418a and identified references 1002a, 1002b, and 1002c. As described above, these references may include side branches, lumen shape, vessel shape, calcium morphology, plaque distribution, etc. As described above, frames corresponding to references 1002a, 1002b, and 1002c in IVUS image 418a may be mapped to specific frames in IVUS image 418b (e.g., based on vascular reference 702, etc.) and an offset angle between frames may be determined. In another embodiment, the offset angle may be determined based on calculating a correlation with each frame and the rotational version of each frame (for example, as described above with respect to Figures 6A and 6B).

[0070] Figure 10B shows points representing the offset angles of each frame corresponding to references 1002a, 1002b, and 1002c. From these points, plot 1004 can be generated representing the offset angle (plotted on the y-axis 1006) for each frame (plotted on the x-axis 1008) in IVUS run 418a. As described above, 1004 can be generated linearly and / or based on one or more linear fitting algorithms or line smoothing algorithms.

[0071] It should be understood that various techniques and workflows for determining alignment offsets are combinable. As used herein, “alignment offset” is intended to mean either or both an offset distance (e.g., for longitudinal alignment of frames) or an offset angle (e.g., for angular alignment of frames). For example, IVUS run frame mapping 420 may include either or both an offset distance and an offset angle. In some examples, the various offset derivation methods described herein can be combined segment by segment. For example, the alignment offset for frames in a first segment (e.g., segment 904a in Figure 9A) may be determined based on a first selection of the alignment methods disclosed herein, while the alignment offset for frames in another segment (e.g., segments 904b, 904c in Figure 9A) may be determined based on a second selection of the alignment methods disclosed herein. As a specific example, frames in segment 904a may be aligned using frame-by-frame correlation, and frames in segment 904b may be aligned using inference from an ML model. However, the claims are not limited to this example and may include any combination of techniques performed in each segment.

[0072] As described above, the GUI can be generated to present graphical representations of different IVUS runs in relation to each other, for example, when the frames are aligned as described herein. Figure 11 shows a GUI 1100 that may be generated according to some embodiments of the present disclosure. In some embodiments, the GUI 1100 may be the GUI 424 of Figures 4, 7, or 15. For example, the processor 406 may execute a set of instructions 416 to generate a GUI 424 having graphical components and arrangements as shown in the GUI 1100 of Figure 11. In such an example, the processor 406 may execute a set of instructions 416 to display the GUI 1100 on a display 404.

[0073] The GUI 1100 may include a graphical representation of IVUS images 418a and 418b. As shown in this example, the graphical representation of IVUS images 418a and 418b may include both axial views (e.g., axial view 1102a and axial view 1102b) and longitudinal views (e.g., longitudinal view 1104a and longitudinal view 1104b). As shown, the GUI 1100 may visualize and arrange the axial views 1102a and 1102b, as well as the longitudinal views 1104a and 1104b, horizontally (e.g., side by side). In other embodiments, the processor 406 may execute a set of instructions 416 to generate a GUI 1100 that visualizes the axial views 1102a and 1102b in a vertical arrangement.

[0074] Furthermore, the GUI 1100 may include a dual view slider bar 1106 and a dual view slider 1108. The dual view slider 1108 can be operated (e.g., via a touchscreen, mouse, joystick, etc.) to slide (or move) between frames of the IVUS image. When the dual view slider 1108 is moved, the processor 406 may execute a set of instructions 416 to regenerate the GUI 1100 to move the frame indicators 1110a and 1110b, which are positioned on the longitudinal views 1104a and 1104b, to match the position of the dual view slider 1108. Furthermore, the on-axial views 1102a and 1102b may change to correspond to frames from each individual IVUS run that match the positions of the frame indicators 1110a and 1110b.

[0075] Accordingly, as provided herein, one or both IVUS runs can be adjusted to align the IVUS runs with each other (for example, based on the offset distance and / or offset angle). Thus, a user (e.g., a physician) can observe different IVUS runs (e.g., before and after a PCI run) in such a way that the position of the vessel and the corresponding reference point are aligned in a visualization, for example, as shown in GUI1100.

[0076] In some embodiments, more than two IVUS runs may be presented in the GUI. For example, Figures 8A and 8B show three IVUS runs that are shifted to align with each other. Thus, the GUI 1100 may be generated to present a graphical representation for each of these three IVUS runs.

[0077] Figure 12 shows a logic flow 1200 for aligning different IVUS runs according to several embodiments of the present disclosure. The logic flow 1200 may be carried out by an IVUS image correlation and visualization system described herein, such as the IVUS image correlation and visualization system 400, 700, for example. For clarity rather than limitation, the logic flow 1200 will be described with reference to the IVUS image correlation and visualization system 400.

[0078] Logic flow 1200 may begin in block 1202. In block 1202, “Receive a first series of IVUS images of the patient’s blood vessel,” a first series of IVUS images may be received, captured via an IVUS catheter percutaneously inserted into the patient’s blood vessel. For example, an information element including a display of IVUS image 418a may be received from an IVUS imaging system 100 in which catheter 102 is percutaneously inserted (or has been inserted) into blood vessel 202. IVUS image 418a may include image frames representing images captured while catheter 102 is withdrawn from distal end 204 to proximal end 206. Processor 406 may execute instruction set 416 to receive an information element including a display of IVUS image 418a from the IVUS imaging system 100, or possibly directly from catheter 102.

[0079] Proceeding to block 1204, “Receive a second series of IVUS images of the patient’s blood vessel,” a second series of IVUS images captured via an IVUS catheter percutaneously inserted into the patient’s blood vessel may be received. For example, an information element including a display of IVUS image 418b may be received from an IVUS imaging system 100 in which catheter 102 is percutaneously inserted (or inserted) into blood vessel 202. Similar to IVUS image 418a, IVUS image 418b may include image frames representing images captured while catheter 102 is withdrawn from distal end 204 to proximal end 206. However, as described above and assumed herein, the distal end 204 and proximal end 206 in IVUS image 418a may be in different positions than the distal end 204 and proximal end 206 in IVUS image 418b. The processor 406 can execute instruction set 416 to receive information elements, including a display of an IVUS image 418b, from the IVUS imaging system 100, or possibly directly from the catheter 102.

[0080] Proceeding to block 1206, "Identify the mapping between frames in a first set of IVUS images and frames in a second set of IVUS images," the mapping between frames in a first set of IVUS images and frames in a second set of IVUS images can be identified. For example, processor 406 may execute instruction set 416 to generate an IVUS run frame mapping 420 based on ML model 422. In another embodiment, processor 406 may execute ML model 704 to identify vascular reference 702, and then identify the IVUS run frame mapping 420 from vascular reference 702. In yet another example, processor 406 may execute instruction set 416 to generate an IVUS run frame mapping 420 based on correlations (e.g., frame-by-frame correlation, frame-by-frame correlation with angular offset, etc.) as described above. In yet another example, processor 406 may execute instruction set 416 to generate an IVUS run frame mapping 420 segment by segment as described above.

[0081] In any of the embodiments described above, the IVUS run frame mapping 420 may include indications of offsets (e.g., time, distance, angle, etc.) of one or both sets of IVUS images that align the IVUS images longitudinally (e.g., as shown in Figure 8B) and / or angularly when applied. As described herein, the IVUS run frame mapping 420 may indicate offset distance and / or offset angle. Multiple examples are not limited to this relationship.

[0082] In some examples, the processor 406 may execute instruction set 416 to map frames based on longitudinal offset, as described herein. In such examples, the processor 406 may execute instruction set 416 to map frames based on partial overlap and time warping. It should be understood that one set of IVUS images (e.g., IVUS image 418a) may be captured at a first retraction velocity, and another set of IVUS images (e.g., IVUS image 418b) may be captured at a second retraction velocity different from the first retraction velocity. In yet another example, one set of IVUS images (e.g., IVUS image 418a) may be captured along a first retraction path through a blood vessel, while another set of IVUS images (e.g., IVUS image 418b) may be captured along a slightly different retraction path, or motion artifacts may appear in the captured IVUS images.

[0083] Therefore, while many examples describe aligning (or co-registering) IVUS images of different runs based on offset distance and / or angle, some embodiments provide that runs may be aligned (or co-registered) based on motion overlaps and / or time warping.

[0084] For example, Figure 13 shows plot 1300, which illustrates the alignment of features extracted and vectorized from two IVUS runs through a blood vessel. The extracted and vectorized feature 1302a may be generated from IVUS image 418a, and the extracted and vectorized feature 1302b may be generated from IVUS image 418b. These features may be aligned based on time warping along the longitudinal offset, as described herein. That is, as shown in this figure, the frames of the IVUS runs may be shifted by different amounts longitudinally to account for the various retraction velocities and paths within the blood vessel.

[0085] When proceeding to block 1208, "Generate a graphical user interface including a display of a first set of IVUS images and a second set of IVUS images, wherein at least one of a first set of IVUS images or a second set of IVUS images is offset based on a mapping (e.g., in time, distance, angle, etc.), and the first set of IVUS images is aligned longitudinally and / or angularly with the second set of IVUS images," a GUI may be generated. The GUI includes a graphical display of a first set of IVUS images and a second set of IVUS images, wherein any number of frames of the first and / or second set of IVUS images are offset (e.g., in time, distance, angle, etc.), and the first and / or second set of IVUS images are aligned longitudinally and / or angularly. For example, processor 406 may execute instruction set 416 to generate GUI 424 as described above. As a specific example, processor 406 may execute instruction set 416 to generate GUI 1100 as GUI 424 and display GUI 1100 on display 404.

[0086] As described above, in some embodiments, the processor 406 of the computing device 402 may execute a set of instructions 416 to generate an IVUS run frame mapping 420 using an ML model, or generate a vascular reference 702 from an ML model, and then generate an IVUS run frame mapping 420 from the vascular reference 702. In such examples, the ML model may be stored in the memory 408 of the computing device 402. It will be understood that the ML model should be trained before deployment. Figure 14 shows an ML environment 1400 that may be used to train an ML model that may later be used to generate (or infer) mappings or vascular references as described herein. The ML environment 1400 may include an ML system 1402, such as a computing device, to which an ML algorithm is applied to learn the relationship between an input and an inferred output. In this example, the ML algorithm may learn the relationship between an input (e.g., an IVUS image) and an output (e.g., a frame mapping or a vascular reference, depending on the embodiment).

[0087] The ML system 1402 may utilize experimental data 1408 collected during multiple previous procedures. The experimental data 1408 may include IVUS images from multiple IVUS runs for multiple patients. The experimental data 1408 may be co-located with the ML system 1402 (for example, stored in the storage 1410 of the ML system 1402), remotely from the ML system 1402 and accessed via the network interface 1504, or a combination of local and remote data.

[0088] Using experimental data 1408, training data 1412 may be formed. As described above, the ML system 1402 may include storage 1410 which may include a hard drive, solid-state storage, and / or random-access memory. Storage 1410 may hold the training data 1412. In general, the training data 1412 may include information elements or data structures that include a display of multiple sets of IVUS images and the corresponding desired output (e.g., mapping or vascular reference). It should be understood that if the desired output is an IVUS frame mapping, the input may be two (or possibly more) sets of IVUS images. In a specific example referring to Figure 4, if ML model 1424 is trained and deployed as ML model 422, the input may be multiple pairs of a first set of IVUS images and a second set of IVUS images (e.g., two or more IVUS runs), and the output may be mappings associated with each pair of the first and second sets of IVUS images (e.g., mappings between IVUS runs). In another example, referring to Figure 7, if ML model 1424 is trained and deployed as ML model 704, the input could be a single series of IVUS images (e.g., a single IVUS run), and the output could be frames within the IVUS images in which vascular criteria (or multiple criteria) are identified.

[0089] The training data 1412 can be applied to train the ML model 1424. Depending on the application, different types of models may be used to form the basis of the ML model 1424. For example, in this example, an artificial neural network (ANN) may be particularly suitable for learning associations between IVUS images (e.g., IVUS image 418a, IVUS image 418b, etc.) and references or frame mappings (e.g., IVUS run frame mapping 420, vascular reference 702, etc.). A convolutional neural network is also well suited to this task. In another example, the ML model 1424 may be based on a spatial transformer (e.g., a spatial transformation network, etc.). In yet another example, the ML model 1424 may be a multi-network, such as a Siamese network.

[0090] The ML model 1424 can be trained using any suitable training algorithm 1420. For example, the examples shown herein may be suitable for supervised training algorithms or reinforcement learning training algorithms. In the case of a supervised training algorithm, the ML system 1402 can be applied with IVUS images 1414 as input 1930, to which the expected output (e.g., mapping or criterion) can be generated by the ML model 1424. In a reinforcement learning scenario, the training algorithm 1420 may attempt to generate an ML model 1424 with the minimum error by maximizing the mapping of some or all (or weighted combinations) of the model input 1930 to the output 1426. In some embodiments, the training data 1412 may be split into “training” data and “test” data, one subset of the training data 1412 may be used to adjust the ML model 1424 (e.g., the weights inside the model), and another non-overlapping subset of the training data 1412 may be used to measure the accuracy of the ML model 1424 in order to infer (or generalize) the output 1426 from an “unseen” input 1930.

[0091] The ML model 1424 may be applied using a processor circuit 1406 which may include appropriate hardware processing resources operating on the logic and structure within the storage 1410. The development of the training algorithm 1420 and / or the trained ML model 1424 may depend at least in part on the hyperparameters 1422. In an exemplary embodiment, the hyperparameters 1422 of the model may be automatically selected based on logic 1428 which may include any known hyperparameter optimization techniques appropriate for the selected ML model 1424 and the training algorithm 1420 used. In any embodiment, the ML model 1424 may be retrained over time to adapt to new knowledge and / or updated experimental data 1424.

[0092] After the ML model 1424 is trained, it can be applied to new input data (e.g., IVUS image 418a, IVUS image 418b, etc.) (e.g., by a processor 406, etc.). This input to the ML model (e.g., ML model 422, ML model 702, etc.) can be formatted according to a predetermined model input 1930 that reflects the manner in which the training data 1412 was provided to the ML model 1424. The ML model 1424 can produce an output 1426 that may be, for example, a generalization or an IVUS run-frame mapping 420 or a vascular reference 702 as described above.

[0093] The above description relates to a particular type of ML system 1402 that applies supervised learning techniques when available training data having input / output pairs is given. However, the present invention is not limited to use in a particular ML paradigm, and other types of ML techniques may be used. For example, in some embodiments, the ML system 1402 may apply, for example, evolutionary algorithms or other types of ML algorithms and models to obtain IVUS run-frame mappings 420 (or possibly vascular references 702) from IVUS images 418a and / or IVUS images 418b.

[0094] In some embodiments, the ML model 1424 may be a conventional ML model, such as a neural network, a convolutional neural network, or an evolutionary artificial neural network. However, in some embodiments, the ML model 1424 may not be an ML model in the conventional sense. For example, the ML model 1424 may be a dynamic programming algorithm, the parameters of which are tuned using the training data 1412.

[0095] In some embodiments, the disclosure may be provided for aligning an IVUS run angularly with respect to a view of a vessel from an external imaging modality. For example, Figure 15 shows an IVUS image correlation and visualization system 1500 according to some embodiments of the disclosure. Generally, the IVUS image correlation and visualization system 1500 is a system for processing IVUS images and correlating and presenting them with external images of the same vessel. For the sake of simplicity, many of the components of the IVUS image correlation and visualization system 400 are referenced and reused when describing the IVUS image correlation and visualization system 1500.

[0096] As described above with respect to Figure 4 and the IVUS image correlation and visualization system 400, the disclosure provides for generating IVUS run frame mappings 420 for IVUS images 418a and 418b. In some embodiments, the IVUS run frame mappings 420 may be generated based on external images of the vessel. It should be noted that various techniques exist for colregistrating intravascular images (e.g., IVUS images 418a and / or 418b) with external images. The disclosure does not reproduce such techniques herein. However, for clarification, it should be noted that references may be identified on external images as well as intravascular images, and that references can be mapped to each other to colregist frames in intravascular images to points (e.g., x and y coordinates) on external images.

[0097] Therefore, in some examples, the IVUS image correlation and visualization system 1500 may be connected to an external imaging system 1506 (e.g., an angiography system, computed tomography (CT) system, magnetic resonance imaging (MRI) system, etc.) configured to capture external images of the vessels in which the IVUS images 418a and / or 418b are captured. Alternatively, the IVUS image correlation and visualization system 1500 may be connected to a memory device that stores the external images or frames of the external images.

[0098] The processor 406 may execute instruction set 416 to receive an external image 1502 (or multiple images) from an external imaging system 1506 (or memory storage device). The processor 406 may execute instruction set 416 to identify a reference in the external image 1502 and the IVUS image 418a (or IVUS image 418b). For example, the processor 406 may execute instruction set 416 to identify a vascular reference 702 corresponding to a reference in the IVUS image 418a and the external image 1502.

[0099] As described above, various techniques exist for identifying references in both internal and external imaging modalities. For example, side branch identification and matching are often used to colregist internal images with external images. This disclosure provides that a processor 406 may execute a set of instructions 416 to identify references and their locations, identify the angles of the references, and store a representation of the reference locations and angles in a vascular reference 702. In some embodiments, the processor 406 may use image processing techniques and / or ML inference to identify the angles of the references. For example, an ML model 702 may be trained, as described above, to identify references and their corresponding angles from an external image 1502. Once the angles of the references in the external image 1502 are identified, the processor 406 may execute a set of instructions 416 to identify an offset angle (e.g., an IVUS run frame mapping 420) used to rotate the frames of the IVUS images (e.g., IVUS images 418a and / or 418b) and align the viewing angle with the viewing angle of the external image 1502. Furthermore, the processor 406 can execute instruction set 416 to determine the offset relative to other frames in the IVUS image (for example, as described above with respect to Figures 10A and 10B, etc.) given the offset angle of the frame corresponding to the reference.

[0100] For example, Figure 16A shows an external image 1502 and two identified references (e.g., side branches) 1602a and 1602b. Processor 406 may execute instruction set 416 to determine the angles of references 1602a and 1602b. Note that the reference angles are derived based on a baseline, such as setting the Z direction from the two-dimensional (2D) image toward the observer as zero (0) degrees. Processor 406 may execute instruction set 416 to rotate (or derive an angle offset) frames from IVUS image 418a that match references 1602a and 1602b based on the angles of references 1602a and 1602b.

[0101] For example, Figures 16B and 16C show image frames 1604a and 1604b (e.g., frames from IVUS image 418a, etc.) that represent references 1602a and 1602b, respectively. The processor 406 executes instruction set 416 to rotate image frames 1604a and 1604b based on the vascular reference angles represented in the external image 1502 (e.g., the angles of the side branches, etc.) and the reference angles in each individual frame 1604a and 1604b, thereby generating rotated image frames 1606a and 1606b. The rotated image frames 1606a and 1606b are shown in Figures 16B and 16C, respectively.

[0102] In some examples, an image frame may be rotated based on a reference landmark. For example, a reference landmark 1610 is shown in Figure 16B. In some embodiments, the processor 406 may execute a set of instructions 416 to identify a reference landmark and rotate the image frame based on the angle of the reference landmark. For example, the reference landmark 1610 (e.g., a side branch) in image frame 1604a is shown at approximately 9 o'clock, or 270 degrees. This frame may be rotated by an angle based on the angle of a reference landmark in another image frame so that the reference landmark is aligned at a specific angle. For example, rotated image frame 1606a shows the reference landmark rotated by 180 degrees.

[0103] Accordingly, as described above, the processor 406 may execute instruction set 416 to angularly align frames in an IVUS run to the viewing perspective of an external image (e.g., external image 1502) so that the angle from which the reference is observed is aligned between both imaging modalities. Figure 16D shows a set of IVUS images 1608 aligned to an external image, where the field of view (or viewpoint) may correspond to frames from IVUS image 418a (or similar) aligned to the field of view of external image frame 1502. Note that this provides a significant improvement over the prior art. It should be understood that intravascular images are often independent of the field of view. For example, IVUS images are captured as the ultrasound transducer rotates within a blood vessel. Therefore, the actual field of view between frames may change. Furthermore, the field of view of the external image may also change (e.g., based on the patient's position relative to the image acquisition system). Therefore, the viewpoints between intravascular and extravascular images are not usually aligned. This disclosure addresses this problem.

[0104] Furthermore, as described above, a GUI may be generated to present a graphical representation of the aligned IVUS run. For example, the GUI may be generated to present a visual representation of a frame from an IVUS run aligned to a vessel observed in an external image. Figure 17 shows a GUI 1700 that may be generated according to some embodiments of the present disclosure. In some embodiments, the GUI 1700 may be the GUI 424 of Figure 4, Figure 7, or Figure 15. For example, the processor 406 may execute a set of instructions 416 to generate a GUI 424 having graphical components and arrangements as shown in the GUI 1700 of Figure 17. In such an example, the processor 406 may execute the set of instructions 416 to display the GUI 1700 on the display 404.

[0105] GUI1700 may include a graphical display of external image 1502 and IVUS image 1608 aligned with the IVUS external image. Thus, when a physician (or user) examines a frame of IVUS image 418a, IVUS image 1608 aligned with the external image is presented so that the lumen and references observed in the IVUS image frame match the angles of the vessels and references (e.g., references 1602a and 1602b) observed in the external image frame.

[0106] Figure 18 shows a computer-readable storage medium 1800. The computer-readable storage medium 1800 may comprise any non-temporary computer-readable or machine-readable storage medium, such as an optical storage medium, a magnetic storage medium, or a semiconductor storage medium. In various embodiments, the computer-readable storage medium 1800 may constitute a manufactured product. In some embodiments, the computer-readable storage medium 1800 may store computer-executable instructions 1802 that can be executed by circuits (e.g., processor 106, processor 4064, processor circuit 1406, etc.). For example, the computer-executable instructions 1802 may include instructions for performing operations described with respect to instruction set 416 and / or logic flow 1200. Examples of computer-readable storage medium 1800 or machine-readable storage medium may include any tangible medium capable of storing electronic data, such as volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. Examples of computer executable instruction 1802 may include any appropriate type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, and visual code.

[0107] Figure 19 shows a schematic diagram of machine 1900 in the form of a computer system in which a set of instructions can be executed to cause the machine to perform any one or more of the methods described herein. More specifically, Figure 19 shows a schematic diagram of machine 1900 in an exemplary form of a computer system in which instructions 1908 (e.g., software, programs, applications, applets, apps, or other executable code) can be executed to cause machine 1900 to perform any one or more of the methods described herein. For example, instruction 1908 can cause machine 1900 to perform logic flow 1200 in Figure 12, etc. More schematicly, instruction 1908 can cause machine 1900 to automatically determine the mapping (e.g., time, distance, angle, etc.) between frames of different IVUS runs through the same vessel (e.g., IVUS run before PCI, IVUS run during or before / after PCI (peri-PCI), and / or IVUS run after PCI), and / or determine the mapping between IVUS runs and external images.

[0108] Instruction 1908 translates a general, unprogrammed machine 1900 into a specific machine 1900 programmed to perform the functions described and illustrated in a specific manner. In alternative examples, machine 1900 may operate as a standalone device or be combined with other machines (e.g., networked). In a networked configuration, machine 1900 may operate as a server machine or a client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 1900 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a mobile phone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), another smart device, a web appliance, a network router, a network switch, a network bridge, or any machine capable of sequentially or otherwise executing Instruction 1908, which specifies the actions performed by machine 1900. Furthermore, although only a single machine 1900 is illustrated, the term “machine” shall also be interpreted to include a set of multiple machines 1900 that individually or collectively perform instruction 1908 to carry out any one or more of the methods described herein.

[0109] Machine 1900 may include a processor 1902, memory 1904, and I / O components 1942, which may be configured to communicate with each other via a bus 1944, etc. In one example, the processor 1902 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a composite instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processors 1906 and 1910 capable of executing instruction 1908. The term “processor” is intended to include multicore processors which may have two or more independent processors (sometimes called “cores”) capable of executing instructions simultaneously. Figure 19 shows multiple processors 1902, but machine 1900 may include a single processor with a single core, a single processor with multiple cores (e.g., a multicore processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0110] Memory 1904 may include main memory 1912, static memory 1914, and storage unit 1916, both of which are accessible to processor 1902 via bus 1944, etc. Main memory 1904, static memory 1914, and storage unit 1916 store instructions 1908 that embody any one or more of the methods or functions described herein. Instructions 1908 may also be present, all or in part, in main memory 1912, static memory 1914, machine-readable media 1918 in storage unit 1916, in at least one of processor 1902 (e.g., in the processor's cache memory), or any suitable combination thereof, while being executed by machine 1900.

[0111] The I / O component 1942 may include a wide variety of components for receiving inputs, providing outputs, generating outputs, transmitting information, exchanging information, capturing measurements, etc. The specific I / O component 1942 included in a particular machine depends on the type of machine. For example, portable devices such as mobile phones are likely to include touch input devices or other such input mechanisms, while headless server machines are unlikely to include such touch input devices. It will be understood that the I / O component 1942 may include many other components not shown in Figure 19. The I / O component 1942 is grouped according to function simply to simplify the following description, and this grouping is by no means limiting. In various exemplary embodiments, the I / O component 1942 may include an output component 1928 and an input component 1930. Output component 1928 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), auditory components (e.g., speakers), tactile components (e.g., vibration motors, resistance mechanisms), and other signal generators. Input component 1930 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photo-optical keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing devices), tactile input components (e.g., physical buttons, touchscreens that provide the position and / or force of touch or touch gestures, or other tactile input components), and voice input components (e.g., microphones).

[0112] In further exemplary embodiments, the I / O component 1942 may include, among a wide variety of other components, a biometric component 1932, a motion component 1934, an environmental component 1936, or a position component 1938. For example, the biometric component 1932 may include components that detect facial expressions (e.g., hand expressions, facial expressions, voice expressions, gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or electroencephalography), and identify people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or electroencephalography-based recognition). The motion component 1934 may include acceleration sensor components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), and the like. Environmental components 1936 may include, for example, lighting sensor components (e.g., photometers), temperature sensor components (e.g., one or more thermometers that detect ambient temperature), humidity sensor components, pressure sensor components (e.g., barometers), acoustic sensor components (e.g., one or more microphones that detect background noise), proximity sensor components (e.g., infrared sensors that detect nearby objects), gas sensors (e.g., gas detection sensors that detect the concentration of harmful gases for safety or measure airborne pollutants), or other components that can provide displays, measurements, or signals corresponding to the surrounding physical environment. Position components 1938 may include position sensor components (e.g., GPS receiver components), altitude sensor components (e.g., altimeters or barometers that detect air pressure from which altitude can be derived), direction sensor components (e.g., magnetometers), and the like.

[0113] Communication can be implemented using a wide variety of technologies. The I / O component 1942 may include a communication component 1940 capable of operating to connect machine 1900 to network 1920 or device 1922 via couplings 1924 and 1926, respectively. For example, the communication component 1940 may include a network interface component or another suitable device for interface connection with network 1920. In further examples, the communication component 1940 may include a wired communication component, a wireless communication component, a cellular communication component, a near-field communication (NFC) component, a Bluetooth® component (e.g., Bluetooth® Low Energy), a Wi-Fi® component, and other communication components that provide communication via other modalities. Device 1922 may be either another machine or a wide variety of peripheral devices (e.g., peripheral devices connected via USB).

[0114] Furthermore, the communication component 1940 may include components that can detect identifiers or are capable of operating to detect identifiers. For example, the communication component 1940 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying tagged audio signals). In addition, various types of information can be derived through the communication component 1940, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, and location by detection of NFC beacon signals that may indicate a specific location.

[0115] Various memories (i.e., memory 1904, main memory 1912, static memory 1914, and / or the memory of processor 1902) and / or storage unit 1916 may store one or more sets of instructions and data structures (e.g., software) that embody or are utilized by any one or more of the methods or functions described herein. When these instructions (e.g., instruction 1908) are executed by processor 1902, they trigger various actions for carrying out the exemplary embodiments disclosed.

[0116] As used herein, the terms “machine storage medium,” “device storage medium,” and “computer storage medium” mean the same thing and may be used interchangeably in this disclosure. These terms refer to one or more storage devices and / or media (e.g., centralized or distributed databases, and / or associated caches and servers) that store executable instructions and / or data. Accordingly, these terms shall be construed to include, but are not limited to, solid-state memory, including internal or external memory of a processor, as well as optical and magnetic media. Specific examples of machine storage medium, computer storage medium, and / or device storage medium include, for example, non-volatile memory, including semiconductor memory devices such as EPROM (erasable programmable read-only memory), EEPROM (electrically erasable programmable read-only memory), FPGAs, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms “machine storage medium,” “computer storage medium,” and “device storage medium” specifically exclude carrier waves, modulated data signals, and other such mediums, although at least some of these are covered under the term “signaling medium,” as described below.

[0117] In various examples, one or more parts of network 1920 could be an ad-hoc network, intranet, extranet, VPN, LAN, WLAN, WAN, WWAN, MAN, Internet, part of the Internet, part of the PSTN, POTS (plain old telephone service) network, cellular telephone network, wireless network, Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, network 1920 or part of network 1920 could include a wireless or cellular network, and combination 1924 could be a CDMA (Code Division Multiple Access) connection, a GSM (Global System for Mobile) connection, or another type of cellular or wireless connection. In this example, coupling 1924 can implement any of the various types of data transfer technologies, including 1xRTT (Single Carrier Radio Transmission Technology), EVDO (Evolution-Data Optimized) technology, GPRS (General Packet Radio Service), EDGE (Enhanced Data rates for GSM Evolution) technology, 3GPP (third Generation Partnership Project) including 3G, 4G (fourth generation wireless) networks, UMTS (Universal Mobile Telecommunications System), HSPA (High Speed ​​Packet Access), WiMAX (Worldwide Interoperability for Microwave Access), LTE (Long Term Evolution) standards, others defined by various standardization bodies, other long-range protocols, or other data transfer technologies.

[0118] Instruction 1908 may be transmitted or received on network 1920 using a transmission medium via a network interface device (e.g., a network interface component included in communication component 1940) and utilizing one of several well-known transport protocols (e.g., Hypertext Transport Protocol (HTTP)). Similarly, Instruction 1908 may be transmitted or received using a transmission medium via coupling 1926 to device 1922 (e.g., peer-to-peer coupling). The terms “transmission medium” and “signaling medium” mean the same thing and may be used interchangeably in this disclosure. The terms “transmission medium” and “signaling medium” shall be construed to include any intangible medium that can store, encode, or carry Instruction 1908 for execution by machine 1900 and that includes digital or analog communication signals or other intangible medium to enable communication of such software. Accordingly, the terms “transmission medium” and “signaling medium” shall be construed to include any form such as modulated data signals, carrier waves, etc. The term “modulated data signal” means a signal in which one or more of its properties are set or modified in such a way as to encode information in the signal.

[0119] Terms used herein should follow their common meanings in the relevant technical field or the meanings indicated by their use in the context; however, if an explicit definition is provided, that meaning shall prevail.

[0120] In this specification, references to “one embodiment” or “a certain embodiment” may refer to the same embodiment, but not necessarily to the same embodiment. Unless the context clearly indicates otherwise, throughout this specification and the claims, words such as “comprise” and “comprising” should be interpreted in a comprehensive sense, i.e., “including, but not limited to,” as opposed to an exclusive or exhaustive sense. Words used in singular or plural form also include plural or singular, respectively, unless expressly limited to one or more. Furthermore, when used in this application, “in this specification,” “above,” “below,” and words with similar meanings refer to the entire application, not to any part thereof. Where the claims use the word “or” in relation to a list of two or more items, the word encompasses all interpretations of the word, including any of the items in the list, all of the items in the list, and any combination of the items in the list, unless expressly limited to one or the other. Any term not expressly defined in this specification has the conventional meaning generally understood by those skilled in the art.

Claims

1. A method for computing devices, The processor receives a first series of intravascular ultrasound (IVUS) images of a patient's blood vessel, wherein the first series of IVUS images comprises a first plurality of frames. The processor receives a second series of intravascular ultrasound (IVUS) images of the patient's blood vessels, wherein the second series of IVUS images comprises a second plurality of frames. The processor determines an offset to the first plurality of frames based at least partially on the second plurality of frames. The processor applies the offset to the first plurality of frames to generate a series of offset IVUS images, and The processor generates a graphical user interface (GUI), the GUI including the display of the offset series of IVUS images and the second series of IVUS images. Methods that include...

2. Determining the offset for the first plurality of frames is: Identifying one of the first plurality of frames, including vascular criteria, Identifying one of the second plurality of frames, including the vascular criteria, and When applied, this determines the offset relative to the first plurality of frames, which aligns one frame of the first plurality of frames, which includes the vascular reference, with one frame of the second plurality of frames, which includes the vascular reference. The method according to claim 1, including the method described in claim 1.

3. The offset includes a first offset and a second offset, and determining the offset for the first plurality of frames is: Identifying a first frame among a plurality of first frames, including a first vascular reference, Identifying a second frame among a plurality of second frames, including the first vascular criterion, When applied to a first segment of the first plurality of frames, the first offset relative to the first plurality of frames is determined, which aligns the first frame of the first plurality of frames with the first frame of the second plurality of frames. Identifying a second frame among the first plurality of frames, including a second vascular criterion, Identifying a second frame among a second plurality of frames including the second vascular criterion, When applied to a second segment of the first plurality of frames, which is different from the first segment, the second offset relative to the first plurality of frames is determined, which aligns the second frame of the first plurality of frames with the second frame of the second plurality of frames. Includes, The method according to claim 2, wherein the second offset is different from the first offset.

4. The method according to claim 3, wherein the first offset includes an offset distance and the second offset includes an offset angle, or the first offset includes an offset distance or an offset angle and the second offset includes an offset distance and an offset angle.

5. Identifying one frame from the first plurality of frames including the vascular reference, and identifying one frame from the second plurality of frames including the vascular reference, Execute a machine learning (ML) model to infer one of the first multiple frames that include the vascular criteria, Execute the ML model to infer one of the second set of frames, including the vascular criteria. The method according to claim 2, including the method described in claim 2.

6. The method according to any one of claims 2 to 5, wherein the vascular criterion is one of the following: lumen shape, vascular shape, side branch position, calcium morphology, plaque distribution, or guide catheter position.

7. Determining the offset for the first plurality of frames is: Based on the frame-by-frame correlation with the second set of frames, a correlation score is calculated for each frame of the first set of frames. Identifying one frame from the first set of multiple frames having the highest correlation score, and one frame from the second set of multiple frames associated with the highest correlation score, and When applied, this determines the offset to the first set of frames, which aligns one of the first set of frames having the highest correlation score with one of the second set of frames associated with the highest correlation score. The method according to any one of claims 1 to 6, including the method described in any one of claims 1 to 6.

8. The offset is the offset distance, and the method is Based on the frame-by-frame correlation with the angular offset with the second set of frames, a correlation score is calculated for each frame of the first set of frames. Identifying one of the first plurality of frames having the highest correlation score, and one of the second plurality of frames associated with the highest correlation score, or a rotated version of the frame, and Based on the one frame of the second plurality of frames associated with the highest correlation score, or the rotated version of the frame, to determine an offset angle relative to the first plurality of frames that, when applied, aligns the one frame of the first plurality of frames having the highest correlation score with the one frame of the second plurality of frames associated with the highest correlation score. It further includes, The method according to claim 7, wherein the offset series of IVUS images are generated by applying the offset distance and the offset angle to the first plurality of frames.

9. Determining the offset for the first plurality of frames is: Based on the frame-by-frame correlation with the angular offset with the second set of frames, a correlation score is calculated for each frame of the first set of frames. Identifying one of the first plurality of frames having the highest correlation score, and one of the second plurality of frames associated with the highest correlation score, or a rotated version of the frame, and Based on the one frame of the second plurality of frames associated with the highest correlation score, or the rotated version of the frame, the offset to the first plurality of frames, which, when applied, aligns the one frame of the first plurality of frames having the highest correlation score to the one frame of the second plurality of frames associated with the highest correlation score. The method according to claim 1, including the method described in claim 1.

10. The method according to any one of claims 1 to 3 or 5 to 9, wherein the offset of the first plurality of frames is a distance offset, an angle offset, or a distance and angle offset.

11. Receiving the second series of IVUS images from the intravascular imaging device, and Receiving the first series of IVUS images from a memory storage device. The method according to any one of claims 1 to 10, including the method described in any one of claims 1 to 10.

12. The method according to any one of claims 1 to 11, wherein the first series of IVUS images are captured during a procedure prior to performing percutaneous coronary intervention (PCI).

13. The method according to claim 12, wherein the second series of IVUS images are captured during or before / after PCI, or during post-PCI procedures.

14. The method according to any one of claims 1 to 13, wherein the GUI includes longitudinal views of the first series of IVUS images and the second series of IVUS images, and the longitudinal views are set to a common scale.

15. A device for an intravascular imaging system, Processor and A memory device coupled to the aforementioned processor and An apparatus comprising, wherein the memory device includes a set of instructions executable by the processor, and when the set of instructions is executed by the processor, causes the intravascular imaging system to perform the method according to any one of claims 1 to 14.

Images