Alignment for multiple series of intravascular images

Abstract

The present invention provides for processing intravascular ultrasound (IVUS) images through different operations of a blood vessel to generate a mapping between frames of each IVUS operation and generating a graphical user interface (GUI) to graphically represent the IVUS operations in relation to each other. In some examples, blood vessel fiducials are identified in one frame of each IVUS operation, and one or two operations are offset in time, distance, and / or angle to align the frame with the identified blood vessel fiducials. In addition, the present invention provides for angularly aligning the viewing angles of the intravascular image and the extravascular image.

Description

Cross-reference to related applications

[0001] This application claims the benefit of U.S. Provisional Patent Application Serial No. 63 / 648,483, filed May 16, 2024, and U.S. Provisional Patent Application Serial No. 63 / 502,859, filed May 17, 2023, the disclosures of which are incorporated herein by reference. Technical Field

[0002] This invention generally relates to intravascular ultrasound (IVUS) imaging systems. In particular, but not exclusively, this invention relates to correlating frames in a first series of IVUS images with frames in a second series of IVUS images. Background Technology

[0003] Insertable ultrasound devices have proven their diagnostic capabilities for a wide range of diseases and conditions. For example, intravascular ultrasound (IVUS) imaging systems have been used as imaging modalities to diagnose blocked blood vessels and provide information to help healthcare practitioners select and place stents and other devices to restore or increase blood flow.

[0004] An IVUS imaging system includes a control module (containing a pulse generator, image acquisition and processing components, and a monitor), a catheter, and a transducer disposed within the catheter. The catheter, containing the transducer, is positioned in or near a lumen or cavity within the area to be imaged, such as the vessel wall or patient tissue adjacent to the vessel wall. The pulse generator in the control module generates electrical pulses, which are delivered to the transducer and converted into acoustic pulses that are transmitted through the patient tissue. The patient tissue (or other structure) reflects the acoustic pulses, and the reflected pulses are absorbed by the transducer and converted back into electrical pulses. The converted electrical pulses are delivered to the image acquisition and processing components and converted into an image that can be displayed on the monitor.

[0005] Typically, physicians capture a series of IVUS images at different stages of treatment. However, conventional tools and systems do not allow physicians to compare these different series of IVUS images beyond providing selected sets of measurements obtained from the images. Therefore, there is a need to correlate IVUS images of the same vessel acquired at different times and to provide a graphical interface to display these images in a way that is relevant to each other.

[0006] Machine learning (ML) is the study of improving computer algorithms through experience. Typically, ML algorithms build models based on sample data (called training data). This model can be used for inference (e.g., making predictions or decisions without being explicitly programmed to do so). It should be understood that the quality of the model's inferences depends on the training data. Therefore, a larger and more complete knowledge base is needed to train these ML models. Summary of the Invention

[0007] This invention provides a simplified overview of a set of concepts, which will be further described in the detailed description below. This invention is not intended to necessarily identify key or essential features of the claimed subject matter, nor is it intended to help determine the scope of the claimed subject matter.

[0008] Typically, the present invention is provided to process raw IVUS images, automatically detected lumen and vessel boundaries, and identify regions of interest in a series of IVUS images, or more particularly, to include the start and end points of frames of interest between them.

[0009] In some embodiments, the present invention may be implemented as a method for a computing device. The method may include receiving, by a processor, a first series of intravascular ultrasound (IVUS) images of a patient's blood vessels, the first series of IVUS images comprising a first plurality of frames; receiving, by the processor, a second series of intravascular ultrasound (IVUS) images of the patient's blood vessels, the second series of IVUS images comprising a second plurality of frames; determining, by the processor, an offset of the first plurality of frames based at least in part on the second plurality of frames; applying the offset to the first plurality of frames by the processor to generate an offset IVUS image series; and generating, by the processor, a graphical user interface (GUI) including indications of the offset IVUS image series and the second series of IVUS images.

[0010] In another embodiment of the method, determining the offset of the first plurality of frames includes: identifying frames in the first plurality of frames that include vascular reference points; identifying frames in the second plurality of frames that include vascular reference points; and determining the offset of the first plurality of frames, which, when applied, aligns the frames in the first plurality of frames that include vascular reference points with the frames in the second plurality of frames that include vascular reference points.

[0011] In another embodiment of the method, the offset includes a first offset and a second offset, and wherein determining the offset of the first plurality of frames includes: identifying a first frame in the first plurality of frames that includes a first vascular reference point; identifying a second frame in the second plurality of frames that includes the first vascular reference point; determining a first offset of the first plurality of frames, wherein when the first offset is applied to a first segment in the first plurality of frames, the first frame in the first plurality of frames is aligned with the first frame in the second plurality of frames; identifying a second frame in the first plurality of frames that includes a second vascular reference point; identifying a second frame in the second plurality of frames that includes a second vascular reference point; and determining a second offset of the first plurality of frames, wherein when the second offset is applied to a second segment in the first plurality of frames that is different from the first segment, the second frame in the first plurality of frames is aligned with the second frame in the second plurality of frames, wherein the second offset is different from the first offset.

[0012] In another embodiment of the method, the first offset includes an offset distance, and the second offset includes an offset angle, or the first offset includes an offset distance or an offset angle, and the second offset includes both an offset distance and an offset angle.

[0013] In another embodiment of the method, identifying frames that include vascular reference points in the first plurality of frames and identifying frames that include vascular reference points in the second plurality of frames includes: executing a machine learning (ML) model to infer frames that include vascular reference points in the first plurality of frames; and executing an ML model to infer frames that include vascular reference points in the second plurality of frames.

[0014] In another embodiment of the method, the vascular reference point is one of the following: lumen geometry, vascular geometry, collateral location, calcium morphology, plaque distribution, or guiding catheter location.

[0015] In another embodiment of the method, determining the offset of the first plurality of frames includes: calculating a correlation score for each frame in the first plurality of frames based on frame-by-frame correlation with the second plurality of frames; identifying the frame with the highest correlation score in the first plurality of frames and the frame associated with the highest correlation score in the second plurality of frames; and determining the offset of the first plurality of frames, which, when applied, aligns the frame with the highest correlation score in the first plurality of frames with the frame associated with the highest correlation score in the second plurality of frames.

[0016] In another embodiment of the method, the offset is an offset distance, and the method further includes: calculating a correlation score for each frame in the first plurality of frames based on frame-by-frame correlation with an angular offset of the second plurality of frames; identifying the frame with the highest correlation score in the first plurality of frames, and the frame or a rotated version of the frame associated with the highest correlation score in the second plurality of frames; and determining an offset angle for the first plurality of frames based on the frame or the rotated version of the frame associated with the highest correlation score in the second plurality of frames, wherein the offset angle, when applied, aligns the frame with the highest correlation score in the first plurality of frames with the frame associated with the highest correlation score in the second plurality of frames, wherein the offset IVUS image series is generated by applying the offset distance and the offset angle to the first plurality of frames.

[0017] In another embodiment of the method, determining the offset of the first plurality of frames includes: calculating a correlation score for each frame in the first plurality of frames based on frame-by-frame correlation with angular offsets of the second plurality of frames; identifying the frame with the highest correlation score in the first plurality of frames, and the frame or a rotated version of the frame associated with the highest correlation score in the second plurality of frames; and determining the offset of the first plurality of frames based on the frame or the rotated version of the frame associated with the highest correlation score in the second plurality of frames, wherein the offset, when applied, aligns the frame with the highest correlation score in the first plurality of frames with the frame associated with the highest correlation score in the second plurality of frames.

[0018] In another embodiment of the method, the offset of the first plurality of frames is a distance offset, an angle offset, or a combination of distance and angle offset.

[0019] In another embodiment, the method may include receiving a second series of IVUS images from an intravascular imaging device; and receiving a first series of IVUS images from a memory storage device.

[0020] In another embodiment of the method, the first series of IVUS images is captured prior to percutaneous coronary intervention (PCI) procedure.

[0021] In another embodiment of the method, the second series of IVUS images is captured during or after a PCI procedure.

[0022] In another embodiment of the method, the GUI includes a longitudinal view of a first series of IVUS images and a second series of IVUS images, wherein the longitudinal views are set at a common scale.

[0023] In some embodiments, the present invention may be implemented as an apparatus for an intravascular imaging system. The apparatus may include a processor; and a memory device coupled to the processor, the memory device including instructions executable by the processor, which, when executed by the processor, cause the intravascular imaging system to perform any of the methods described herein.

[0024] In some embodiments, the present invention may be implemented as at least one machine-readable storage device. The at least one machine-readable storage device may include a plurality of instructions, responsive to execution by a processor of an intravascular ultrasound (IVUS) imaging system, which cause the processor to perform any of the methods described herein.

[0025] In some embodiments, the present invention can be implemented as an apparatus for an intravascular imaging system. The apparatus may include an apparatus for an intravascular imaging system, comprising a display; a processor coupled to the display; and a memory device coupled to the processor, the memory device including instructions executable by the processor, the instructions, when executed by the processor, causing the intravascular imaging system to receive a first series of intravascular ultrasound (IVUS) images of a patient's blood vessels, the first series of IVUS images comprising a first plurality of frames; receive a second series of intravascular ultrasound (IVUS) images of the patient's blood vessels, the second series of IVUS images comprising a second plurality of frames; determine an offset of the first plurality of frames based at least in part on the second plurality of frames; apply the offset to the first plurality of frames to generate an offset IVUS image series; generate a graphical user interface (GUI) including indications of the offset IVUS image series and the second series of IVUS images; and display the GUI on the display.

[0026] In another embodiment of the device, the instructions also cause the intravascular imaging system to identify frames in a first plurality of frames that include vascular reference points; identify frames in a second plurality of frames that include vascular reference points; and determine an offset of the first plurality of frames, which, when applied, aligns the frames in the first plurality of frames that include vascular reference points with the frames in the second plurality of frames that include vascular reference points.

[0027] In another embodiment of the device, the offset includes a first offset and a second offset, and wherein the instructions further cause the intravascular imaging system to: identify a first frame including a first vascular reference point in a first plurality of frames; identify a second frame including the first vascular reference point in a second plurality of frames; determine a first offset of the first plurality of frames, which, when applied to a first segment in the first plurality of frames, aligns the first frame in the first plurality of frames with the first frame in the second plurality of frames; identify a second frame including a second vascular reference point in the first plurality of frames; identify a second frame including a second vascular reference point in the second plurality of frames; and determine a second offset of the first plurality of frames, which, when applied to a second segment in the first plurality of frames that is different from the first segment, aligns the second frame in the first plurality of frames with the second frame in the second plurality of frames, wherein the second offset is different from the first offset.

[0028] In another embodiment of the device, the first offset includes an offset distance, and the second offset includes an offset angle, or the first offset includes an offset distance or an offset angle, and the second offset includes both an offset distance and an offset angle.

[0029] In another embodiment of the device, the instructions also cause the intravascular imaging system to: execute a machine learning (ML) model to infer frames in a first plurality of frames that include vascular reference points; and execute an ML model to infer frames in a second plurality of frames that include vascular reference points.

[0030] In another embodiment of the device, the vascular reference point is one of the following: lumen geometry, vascular geometry, collateral location, calcium morphology, plaque distribution, or guide tube location.

[0031] In some embodiments, the present invention may be implemented as at least one machine-readable storage device. The at least one machine-readable storage device may include a plurality of instructions responsive to execution by a processor of an intravascular ultrasound (IVUS) imaging system, the plurality of instructions causing the processor to receive a first series of intravascular ultrasound (IVUS) images of a patient's blood vessels, the first series of IVUS images comprising a first plurality of frames; receive a second series of intravascular ultrasound (IVUS) images of the patient's blood vessels, the second series of IVUS images comprising a second plurality of frames; determine an offset of the first plurality of frames based at least in part on the second plurality of frames; apply the offset to the first plurality of frames to generate an offset IVUS image series; generate a graphical user interface (GUI) including indications of the offset IVUS image series and the second series of IVUS images; and send the GUI to a display coupled to the IVUS imaging system.

[0032] In another embodiment of at least one machine-readable storage device, the execution instructions further cause the IVUS imaging system to calculate a correlation score for each frame in the first plurality of frames based on frame-by-frame correlation with the second plurality of frames; identify the frame with the highest correlation score in the first plurality of frames and the frame associated with the highest correlation score in the second plurality of frames; and determine an offset for the first plurality of frames, which, when applied, aligns the frame with the highest correlation score in the first plurality of frames with the frame associated with the highest correlation score in the second plurality of frames.

[0033] In another embodiment of at least one machine-readable storage device, the offset is an offset distance, and the execution instructions further cause the IVUS imaging system to calculate a correlation score for each frame in the first plurality of frames based on frame-by-frame correlation with the angular offset of the second plurality of frames; identify the frame with the highest correlation score in the first plurality of frames, and the frame or a rotated version of the frame associated with the highest correlation score in the second plurality of frames; and determine an offset angle of the first plurality of frames based on the frame or the rotated version of the frame associated with the highest correlation score in the second plurality of frames, the offset angle aligning the frame with the highest correlation score in the first plurality of frames with the frame associated with the highest correlation score in the second plurality of frames, wherein the offset IVUS image series is generated by applying the offset distance and the offset angle to the first plurality of frames.

[0034] In another embodiment of at least one machine-readable storage device, the execution instructions further cause the IVUS imaging system to calculate a correlation score for each frame in the first plurality of frames based on frame-by-frame correlation with angular offsets of the second plurality of frames; identify the frame with the highest correlation score in the first plurality of frames, and the frame or a rotated version of the frame associated with the highest correlation score in the second plurality of frames; and determine an offset of the first plurality of frames based on the frame or the rotated version of the frame associated with the highest correlation score in the second plurality of frames, the offset aligning the frame with the highest correlation score in the first plurality of frames with the frame associated with the highest correlation score in the second plurality of frames when applied.

[0035] In another embodiment of at least one machine-readable storage device, the offset of the first plurality of frames is a distance offset, an angle offset, or a distance and angle offset. Attached Figure Description

[0036] For ease of identification of any discussion of a component or behavior, the highest digit or more digits in the reference numerals refer to the drawing number in which the component is first introduced.

[0037] Figure 1 An IVUS imaging system according to an embodiment of the present invention is shown.

[0038] Figure 2 Example angiographic images of blood vessels are shown.

[0039] Figure 3A and Figure 3B An IVUS image of a blood vessel is shown.

[0040] Figure 4 An IVUS image association and visualization system according to at least one embodiment of the present invention is shown.

[0041] Figure 5A An example frame-by-frame correlation between frames in one set of IVUS images and frames in another set of IVUS images is shown according to at least one embodiment of the present invention.

[0042] Figure 5B Showing can be based on Figure 5A The example in the image shows a correlation score graph generated by frame-by-frame correlation.

[0043] Figure 6A This illustrates another example of frame-by-frame correlation between frames in one set of IVUS images and frames in another set of IVUS images, as well as angularly offset versions of those frames, according to at least one embodiment of the invention.

[0044] Figure 6B Showing can be based on Figure 6A The example in the image shows a correlation score graph generated by frame-by-frame correlation.

[0045] Figure 7 Another IVUS image association and visualization system is shown according to at least one embodiment of the present invention.

[0046] Figure 8A and Figure 8B Several series of IVUS images of blood vessels aligned according to at least one embodiment of the present invention are shown.

[0047] Figure 9A and Figure 9B Examples of segment-by-segment alignment and offset identification are shown according to at least one embodiment of the present invention.

[0048] Figure 10A and Figure 10B Another example of segment-by-segment alignment and offset identification is shown according to at least one embodiment of the present invention.

[0049] Figure 11 A graphical user interface (GUI) according to at least one embodiment of the present invention is shown.

[0050] Figure 12 The present invention illustrates a logical flow for determining the mapping between different IVUS operations of a blood vessel according to at least one embodiment of the present invention.

[0051] Figure 13 The invention illustrates extracted and vectorized features based on a series of IVUS images, and temporal distortion along the longitudinal offset of two series of IVUS images, according to at least one embodiment of the invention.

[0052] Figure 14 An exemplary machine learning (ML) system suitable for use with exemplary embodiments of the present invention is shown.

[0053] Figure 15 Another IVUS image association and visualization system is shown according to at least one embodiment of the present invention.

[0054] Figure 16A Example extravascular images and identification reference points are shown.

[0055] Figure 16B , Figure 16C and Figure 16D Show rotation to, as in Figure 16A An example of a frame in which the reference point observed in the external image is aligned with the angle of the IVUS run.

[0056] Figure 17 A GUI is shown according to at least one embodiment of the present invention.

[0057] Figure 18 This shows a computer-readable storage medium.

[0058] Figure 19 A diagram of the machine is shown. Detailed Implementation

[0059] The features and technical advantages of the invention have been summarized above, which will facilitate a better understanding of the following detailed description. Those skilled in the art will understand that the disclosed embodiments can be readily used as the basis for modifications or the design of other structures to achieve the same objectives as the invention. The novel features of the invention with respect to its organization and operation, as well as further objects and advantages, will be better understood from the following description when considered in conjunction with the accompanying drawings. However, it should be clearly understood that each of the figures provided is for illustrative and descriptive purposes and is not intended to be a limiting definition of the invention.

[0060] As indicated, the present invention relates to IVUS images of a patient and lumens (e.g., blood vessels) and to processing IVUS recordings, or in other words, processing a series of IVUS images. Therefore, an example IVUS imaging system, a patient's blood vessels, and a series of IVUS images are described.

[0061] Suitable IVUS imaging systems include, but are not limited to, one or more transducers located on the distal end of a catheter configured and positioned for percutaneous insertion into a patient.

[0062] Figure 1 An embodiment of an IVUS imaging system 100 is schematically illustrated. The IVUS imaging system 100 includes a catheter 102 that can be coupled to a control system 104. The control system 104 may include, for example, a processor 106, a pulse generator 108, and a drive unit 110. In at least some embodiments, the pulse generator 108 generates electrical pulses that can be input to one or more transducers (not shown) disposed in the catheter 102.

[0063] In some embodiments, mechanical energy from the drive unit 110 can be used to drive an imaging core (also not shown) disposed in the conduit 102. In at least some embodiments, electrical signals transmitted from one or more transducers can be input to the processor 106 for processing. In at least some embodiments, the processed electrical signals from one or more transducers can be used to form a series of images, as described in more detail below. For example, a scan converter can be used to map scan line samples (e.g., radial scan line samples, etc.) onto a two-dimensional Cartesian grid, which can be used as the basis for a series of IVUS images displayed to the user.

[0064] In at least some embodiments, processor 106 may also be used to control the operation of one or more other components of control system 104. For example, processor 106 may be used to control at least one of the frequency or duration of electrical pulses transmitted from pulse generator 108, and the rotational rate of the imaging core driven by drive unit 110. Additionally, when IVUS imaging system 100 is configured for automatic pull-back, drive unit 110 may control the speed and / or length of pull-back.

[0065] Figure 2 An extravascular image 200 of a patient's vessel 202 is shown. As described, an IVUS imaging system (e.g., IVUS imaging system 100, etc.) is used to capture a series of intraluminal images or "records" of a vessel, such as vessel 202. For example, an IVUS catheter (e.g., catheter 102) is inserted into vessel 202, and a record or series of IVUS images is captured as catheter 102 is pulled back from distal end 204 to proximal end 206. Catheter 102 can be pulled back manually or automatically (e.g., under the control of a drive unit 110, etc.). The series of IVUS images captured between distal end 204 and proximal end 206 are generally referred to as images from an IVUS run.

[0066] Figure 3A and Figure 3B A two-dimensional (2D) representation of an IVUS image of vessel 202 is shown. For example, Figure 3A IVUS image 300a shows a longitudinal view of an IVUS record depicting blood vessel 202 between proximal 206 and distal 204.

[0067] Figure 3B Image frame 300b is shown, depicting a coaxial (or short-axis) view of vessel 202 at point 302. In other words, image frame 300b is a single frame or image from a series of IVUS images that may be captured between distal 204 and proximal 206, as described herein. As described above, physicians typically capture IVUS runs (e.g., series of IVUS images) at different stages of treatment. For example, IVUS images may be captured before percutaneous coronary intervention (PCI) and after performing PCI procedures (e.g., stent placement, balloon dilation, rotational atherectomy, etc.).

[0068] This invention provides IVUS runs from different time frames that can be aligned on a frame-by-frame basis and provides a graphical user interface associated with the IVUS runs, allowing physicians to view the associated IVUS runs and thus more directly understand their treatment of blood vessels, for example, by observing differences in the nature of lesions through side-by-side comparison.

[0069] Figure 4An IVUS image association and visualization system 400 according to some embodiments of the present invention is illustrated. Typically, the IVUS image association and visualization system 400 is a system for processing, association, and presentation of multiple series of IVUS images of the same vessel. The IVUS image association and visualization system 400 can be implemented in a commercial IVUS guidance or navigation system, such as, for example, the AVVIGO® guidance system available from Boston Scientific®. The present invention provides advantages over existing or conventional IVUS navigation systems because no conventional system can provide association of IVUS runs performed at different times.

[0070] In some embodiments, the IVUS image association and visualization system 400 may be implemented as part of the control system 104. Alternatively, the control system 104 may be implemented as part of the IVUS image association and visualization system 400. As depicted, the IVUS image association and visualization system 400 includes a computing device 402. Optionally, the IVUS image association and visualization system 400 includes an IVUS imaging system 100 and a display 404.

[0071] It should be noted that although the present invention frequently uses IVUS as an exemplary intravascular imaging modality, the present invention can be provided for longitudinal and / or angular alignment of frames from different runs captured using any of a variety of other intravascular imaging modalities, such as optical coherence tomography (OCT).

[0072] The computing device 402 can be any of a variety of computing devices. In some embodiments, the computing device 402 may be incorporated into and / or implemented by the console of the display 404. In some embodiments, the computing device 402 may be a workstation or server communicatively coupled to the IVUS imaging system 100 and / or the display 404. In other embodiments, the computing device 402 may be provided by a cloud-based computing device, such as a compute-as-a-service system accessible via a network (e.g., the Internet, an intranet, a wide area network, etc.). The computing device 402 may include a processor 406, a memory 408, input and / or output (I / O) devices 410, a network interface 412, and IVUS imaging system acquisition circuitry 414.

[0073] Processor 406 may include circuitry or processor logic, such as, for example, any of a variety of commercial processors. In some examples, processor 406 may include multiple processors, multi-threaded processors, multi-core processors (whether multiple cores coexist on the same die or on separate dies), and / or some other type of multiprocessor architecture, where multiple physically separate processors are linked together in some way by the aforementioned multiprocessor architecture. Additionally, in some examples, processor 406 may include a graphics processing section and may include dedicated memory, multi-threaded processing, and / or some other parallel processing capabilities. In some examples, processor 406 may be an application-specific integrated circuit (ASIC) or a field-programmable integrated circuit (FPGA).

[0074] Memory 408 may include logic, a portion of which includes an array of integrated circuits to form non-volatile memory for persistent storage of data or a combination of non-volatile and volatile memory. It should be understood that memory 408 may be based on any of a variety of technologies. In particular, the array of integrated circuits included in memory 120 may be arranged to form one or more types of memory, such as, for example, dynamic random access memory (DRAM), NAND memory, NOR memory, etc.

[0075] I / O device 410 can be any of a variety of devices for receiving input and / or providing output. For example, I / O device 410 may include a keyboard, mouse, joystick, foot pedal, display, touch-enabled display, haptic feedback device, LED, etc.

[0076] Network interface 412 may include logical and / or features for supporting communication interfaces. For example, network interface 412 may include one or more interfaces operating according to various communication protocols or standards for communication via direct or network communication links. Direct communication may occur via communication protocols or standards described in one or more industry standards (including their successors and variations). For example, network interface 412 may facilitate communication via buses such as, for example, Peripheral Component Interconnect High Speed ​​(PCIe), Non-Volatile Memory High Speed ​​(NVMe), Universal Serial Bus (USB), System Management Bus (SMBus), SAS (e.g., Serial Attached Small Computer System Interface (SCSI)) interfaces, Serial AT Attachment (SATA) interfaces, etc. Additionally, network interface 412 may include logical and / or features for enabling communication via various wired or wireless network standards (e.g., the 902.11 communication standard). For example, network interface 412 may be configured to support wired communication protocols or standards such as Ethernet, etc. As another example, network interface 412 may be configured to support wireless communication protocols or standards, such as, for example, Wi-Fi, Bluetooth, ZigBee, LTE, 5G, etc.

[0077] IVUS imaging system acquisition circuit 414 may include circuitry, including custom-made or specially programmed circuitry, configured to receive or receive and transmit signals between IVUS imaging systems 100, including indications of IVUS operation, a series of IVUS images, or frames or multiple frames of IVUS images.

[0078] Memory 408 may include instructions 416. During operation, processor 406 may execute instructions 416 to cause computing device 402 to receive (e.g., from IVUS imaging system 100, etc.) a series of IVUS images from multiple IVUS runs of a blood vessel and store the records in memory 408 as IVUS images 418a, IVUS images 418b, etc. For example, processor 406 may execute instructions 416 to receive information elements from IVUS imaging system 100, including indications of IVUS images captured by catheter 102 as it is pulled back from distal 204 to proximal 206, the images including indications of the anatomy and / or structures (including vessel walls and plaques) of blood vessel 202. Furthermore, it should be understood that processor 406 may execute instructions 416 to receive IVUS images from multiple runs through the blood vessel (e.g., pre-PCI, post-PCI, at different times, etc.). It should be understood that IVUS images 418a and 418b may be stored in various image formats or even non-image formats or data structures indicating the blood vessel 202. In addition, IVUS images 418a and 418b include images that can be used to form blood vessel 202 when collinearly represented, such as, for example, several “frames” or individual images of the image represented by IVUS image 300a.

[0079] This invention provides a frame-by-frame association of IVUS images 418a and 418b, and presents an associated view of the images in a graphical user interface. In some examples, processor 406 may execute instructions 416 to infer mappings using a machine learning (ML) model (e.g., see...). Figure 4 The processor 406 can identify the IVUS run frame mapping 420 from IVUS images 418a and 418b. In some embodiments, the processor 406 may execute instructions 416 to use frame-by-frame correlation or segment-by-segment correlation (e.g., see...). Figure 4 The IVUS run frame map 420 is identified from IVUS images 418a and 418b. In some embodiments, this can be facilitated using standard image processing techniques and / or ML inference. In other examples, memory 408 is executable with instructions 416 to determine one or more reference points (e.g., via machine learning, via image processing algorithms, etc.) and to determine the IVUS run frame map 420 based on the identified reference points (e.g., see [link to documentation]). Figure 7Although each method used to determine the IVUS run frame mapping 420 differs slightly and is discussed separately, these methods are similar once the IVUS run frame mapping 420 is identified. Furthermore, in some embodiments, the IVUS run may be mapped and / or aligned with angiographic images of the vessel (e.g., see Figure 6). It is important to note that... Figure 4 , Figure 7 and Figure 15 IVUS image association and visualization systems 400, 700, and 1500 are described respectively. This is done to clearly describe the various alignment techniques disclosed herein. However, it is important to note that an alignment technique described with respect to one system (e.g., 400) can be used in conjunction with an alignment technique disclosed with respect to another system (e.g., 700 and / or 1500). For example, it should be understood that one alignment technique can be used for aligning frames longitudinally, while another technique can be used for aligning frames angularly.

[0080] Turn now Figure 4 Once the IVUS runtime frame map 420 is generated, frame-by-frame correlations between IVUS images 418a and 418b can be generated based on the IVUS runtime frame map 420. For example, memory 408 can execute instructions 416 to associate each frame of IVUS image 418a with a corresponding frame of IVUS image 418b. Furthermore, memory 408 can execute instructions 416 to generate a graphical user interface (GUI) 424 that, based on the IVUS runtime frame map 420, depicts indications of frames of IVUS image 418a associated with and / or related to corresponding frames of IVUS image 418b.

[0081] In the example of using ML to generate IVUS running frame maps 420, processor 406 can execute instructions 416 to execute or "run" ML model 422 using IVUS images 418a and 418b as input to generate IVUS running frame maps 420. ML model 422 can infer IVUS running frame maps 420 from IVUS images 418a and 418b. Memory 408 can store a copy of ML model 422, and processor 406 can execute ML model 422 to generate IVUS running frame maps 420. Typically, ML model 422 can be any of a variety of ML models. Examples of ML models as envisioned herein, and even examples of training ML models, are provided below.

[0082] In some embodiments, the present invention may be provided to align IVUS runs based on the correlation between each frame of one IVUS run and all frames of another IVUS run. Processor 406 may execute instructions 416 to determine frame-by-frame correlations 426 for the IVUS runs. For example, processor 406 may execute instructions 416 to iterate through each frame of IVUS image 418a and compute (e.g., using a reference point, using ML, using background subtraction, using cross-correlation, etc.) the correlations between all frames of IVUS image 418b. Subsequently, processor 406 may execute instructions 416 to identify, for each frame in IVUS image 418a, the most closely related frames in IVUS image 418b. For example, Figure 5A Image frame 502 (e.g., from IVUS image 418a, etc.) and image frames 504a, 504b, 504c, etc. (e.g., from IVUS image 418b, etc.) are depicted. Processor 406 can execute instructions 416 to calculate the correlation (e.g., correlation value, score, etc.) between image frame 502 and image frames 504a, 504b, 504c, etc. Figure 5B Figure 506 illustrates the calculated correlation. Figure 506 plots the correlation scores between a specific frame from one set of IVUS images (e.g., image frame 502) and frames from another set of IVUS images (e.g., frames 504a, 504b, 504c, etc.). As shown, the correlation score values ​​are plotted on the y-axis 508, while the frame numbers from the second set of IVUS images are plotted on the x-axis 510.

[0083] In some embodiments, frame-by-frame correlation can be determined for each frame at different rotation angles. Processor 406 can execute instructions 416 to identify, for each frame in one set of IVUS images (e.g., IVUS image 418a, etc.), the correlation with frames in another set of IVUS images (e.g., IVUS image 418b, etc.) at several rotation angles. Figure 6A An example of this is shown. During operation of the IVUS image association and visualization system 400, processor 406 may execute instructions 416 to calculate a correlation score between frames from one set of IVUS images (e.g., image frame 502 from IVUS image 418a, etc.) and frames from another set of IVUS images (e.g., image frame 504a from IVUS image 418b, etc.) and rotated versions of the image frames (e.g., rotated image frames 602a and 602b). Similar to the frame-by-frame correlation described above, processor 406 may execute instructions 416 to calculate a correlation score between each frame from one set of IVUS images (e.g., IVUS image 418a) and each frame from another set of IVUS images (e.g., IVUS image 418b, etc.) and their rotated versions. Figure 6BFigure 604 illustrates the calculated correlation. Figure 604 plots the correlation scores between a specific image frame (e.g., image frame 502) from one set of IVUS images and image frames (e.g., image frame 504a) from another set of IVUS images, as well as rotated versions of those image frames (e.g., rotated image frames 602a and 602b). As shown, the correlation score values ​​are plotted on the y-axis 606, while the rotation angles are plotted on the x-axis 608.

[0084] In some examples, processor 406 may execute instruction 416 to generate rotated image frames at every possible rotation angle (e.g., rotated image frames 602a, 602b, etc.). In such an example, 359 rotated image frames will be generated. In other examples, processor 406 may execute instruction 416 to generate rotated image frames at subsets of all possible rotation angles (e.g., every 2 degrees, every 5 degrees, every 10 degrees, every 15 degrees, every 20 degrees, every 30 degrees, every 45 degrees, etc.).

[0085] Typically, the IVUS running frame map 420 may include an indication of offset (e.g., in time, distance, rotation, etc.) where one (or each) of the IVUS images 418a and 418b is to be adjusted to align them. As used herein, the term “alignment” refers to aligning the frames of an image in the longitudinal direction and / or by angle.

[0086] In some embodiments, processor 406 may execute instructions 416 to receive bookmarks (or multiple bookmarks) that identify frames of one of IVUS images 418a and / or IVUS images 418b. IVUS operation frame mapping 420 may be adjusted to align with the bookmarks or multiple bookmarks. In some embodiments, this mapping is not linear. For example, frames from IVUS image 418a may be linearly adjusted (e.g., by a first distance) and / or rotated (e.g., by a first angle) based on their relevance to frames from IVUS image 418b, while adjacent frames in IVUS image 418a may be linearly adjusted (e.g., by a second distance) and / or rotated (e.g., by a second angle) based on their relevance to the same or different frames from IVUS image 418b.

[0087] Figure 7 An IVUS image association and visualization system 700 according to some embodiments of the present invention is illustrated. Generally, the IVUS image association and visualization system 700 is a system for processing, associating, and presenting multiple series of IVUS images of the same blood vessel, similar to the IVUS image association and visualization system 400. For the sake of simplicity, many components of the IVUS image association and visualization system 400 are referenced and reused in the IVUS image association and visualization system 700.

[0088] As mentioned above Figure 4 As described in the IVUS image association and visualization system 400, the present invention provides an IVUS runtime frame mapping 420 for IVUS images 418a and 418b. In some embodiments, processor 406 may execute instructions 416 to identify vascular reference points 702 in IVUS images 418a and 418b. In some embodiments, vascular reference points 702 may be any one or more coronary artery anatomical reference points (e.g., lumen geometry, vessel geometry, collateral location, calcium morphology, plaque distribution, guide catheter location, etc.). In some embodiments, processor 406 executes instructions 416 to identify vascular reference points 702 from IVUS images 418a and 418b using image processing algorithms (e.g., geometric image recognition algorithms for identifying lumen contours, etc.). In other embodiments, memory 408 may include one or more ML models 704 configured to infer vascular reference points 702 from IVUS images (e.g., IVUS images 418a, 418b, etc.). For example, memory 408 may include ML model 704, which may include one or more ML models trained to infer reference points (e.g., collateral location, calcium morphology, guide catheter location, etc.). Thus, processor 406 may execute ML model 704 to identify vascular reference points 702 in frames of IVUS images 418a and 418b.

[0089] Processor 406 can execute instructions 416 to generate an IVUS running frame map 420 based on vascular reference point 702, for example by pairing frames from IVUS images 418a and 418b that identify the same anatomical reference point. Given the IVUS running frame map 420, processor 406 can execute instructions 416 to associate each frame of IVUS image 418a with a corresponding frame of IVUS image 418b. Furthermore, memory 406 can execute instructions 416 to generate a GUI 424 that, based on the IVUS running frame map 420, depicts indications of frames of IVUS image 418a associated with and / or related to corresponding frames of IVUS image 418b.

[0090] It should be understood that in some embodiments, processor 406 may execute instructions 416 and identify reference points in a single frame of each IVUS run (e.g., IVUS images 418a and 418b). For example, vascular reference point 702 may include a collateral branch identified in a frame of IVUS image 418a and the same collateral branch identified in a frame of IVUS image 418b. In other embodiments, processor 406 may execute instructions 416 and identify reference points in multiple frames. In such examples, the reference points need not be identical. For example, as described above, vascular reference point 702 may include a collateral branch location in a frame of IVUS image 418a and the same collateral branch location in a frame of IVUS image 418b, as well as a guiding catheter location in another frame of IVUS image 418a and another guiding catheter location in another frame of IVUS image 418b. In this context, the examples are not limited.

[0091] Figure 8A Several IVUS runs are shown at scale 802. The figure depicts an IVUS run or multiple sets of IVUS images 804a, 804b, and 804c. It should be understood that each set of IVUS images (e.g., multiple sets of IVUS images 804a, 804b, and 804c) comprises several frames. As summarized above, in some embodiments, a reference point is identified in one or more frames of each set of IVUS images. The figure depicts the reference point 806 identified in frames of each set of IVUS images 804a, 804b, and 804c. An IVUS run frame map 420 may be generated based on frames identified as indicating (representing, corresponding to, depicting, etc.) the reference point 806. For example, a processor 406 may execute instructions 416 to identify frames (e.g., frames from images 804a, 804b, and 804c, etc.) that include the vascular reference point 806 (or the vascular reference point itself) from each set of IVUS images. Processor 406 executes instructions 416 to identify offsets of frames in one or more sets of IVUS images, which, when applied, will align the frames in each set of IVUS images on scale 802. In some embodiments, the offset can be a time offset, a distance offset, an angular offset, or any combination of time, distance, and / or angular offsets. Furthermore, it should be understood that scale 802 can be any scale used to represent or graphically present an IVUS run. For example, some IVUS runs are graphically presented on a pullback scale with distal and proximal points along the pullback. As an example, the pullback scale can be represented in distance units (e.g., millimeters, etc.). In the case of offsets, offsets can be generated such that when the offset is applied, frames identified as indicating the same reference point (e.g., vascular reference point 806) are shifted or adjusted so that the frames are aligned on the scale.

[0092] For example, Figure 8BShowing again set at scale 802 from Figure 8A The IVUS images. However, frames from multiple sets of IVUS images 804a and 804c have been adjusted based on identified offsets (e.g., according to IVUS running frame mapping 420, etc.) to align the frames indicating reference points on scale 802. For example, IVUS image 804a is adjusted by offset 808a to shift IVUS image 804a relative to scale 802, while IVUS image 804c is adjusted by offset 808b to shift IVUS image 804c relative to scale 802. Applying offsets 808a and 808b to IVUS images 804a and 804c, respectively, aligns the IVUS images relative to scale 802, and specifically aligns the frames indicating vascular reference point 806 in IVUS images 804a, 804b, and 804c relative to scale 802. For example, as depicted in the figure, the reference point 806 identified in each IVUS run is aligned when IVUS images 804a, 804b, and 804c are adjusted based on offsets 808a and 808b. It should be noted that offsets 808a and 808b are depicted as longitudinal offsets, or more precisely, as the distance by which the frame is offset along scale 802. However, offsets 808a and 808b could alternatively be offset angles (e.g., the angle by which the frame is rotated) or both offset distances and offset angles. Furthermore, it should be noted that although only a single offset for each IVUS run is depicted (e.g., offset 808a for IVUS image 804a and offset 808b for IVUS image 804c), this document may also provide multiple offsets for each run (e.g., for frames in a segment, for each frame, for only some frames, etc.).

[0093] Several techniques and workflows are provided to identify the longitudinal and / or angular offsets of frames in a set of IVUS images (e.g., IVUS image 418a, etc.) to align frames with frames in another set of IVUS images (e.g., IVUS image 418b, etc.). It should be noted that, although... Figure 8A and Figure 8B Only longitudinal alignment is described, but the invention can also be implemented to align IVUS longitudinally, angularly, and / or longitudinally and angularly.

[0094] In some embodiments, processor 406 may execute instructions 416 to longitudinally align frames from IVUS image 418a with frames from IVUS image 418b on a segment-by-segment basis. For example, in some embodiments, processor 406 may execute instructions 416 to identify segments based on vascular reference point 702. Figure 9AAn IVUS image 418a is shown, along with identified reference points 902a and 902b. As summarized above, these reference points can be collateral vessels, lumen geometry, vascular geometry, calcium morphology, plaque distribution, etc. Processor 406 can execute instructions 416 to group frames from IVUS image 418a into segments based on the identified reference points 902a and 902b. For example, Figure 9A The frames from IVUS image 418a are shown as segments 904a, 904b, and 904c. Therefore, the offsets of frames in a set of IVUS images (e.g., IVUS image 418a, etc.) can be generated for different segments using the identified vascular reference point 702.

[0095] Figure 9B Points are shown representing each longitudinal offset of a frame corresponding to reference points 902a and 902b. Based on these points, Figure 906 can be generated, representing the longitudinal offset (plotted on the x-axis 910) for each frame in IVUS run 418a. In some embodiments, Figure 906 can be generated linearly between the points (e.g., as shown in Figure 908). Figure 9B (As drawn). In other embodiments, processor 406 may execute instructions 416 to generate Figure 906 based on one or more line fitting algorithms (e.g., raster-based line fitting, etc.). The longitudinal offset of the frame in each segment 904a, 904b, and 904c may be determined based on Figure 906.

[0096] In some embodiments, processor 406 may execute instructions 416 to align frames from IVUS image 418a and frames from IVUS image 418b in a rotational manner based on vascular reference point 702. For example, in some embodiments, IVUS running frame mapping 420 may include an offset angle (e.g., an angle that rotates the frames). Figure 10A An IVUS image 418a is shown, along with identified reference points 1002a, 1002b, and 1002c. As summarized above, these reference points can be collateral vessels, lumen geometry, vascular geometry, calcium morphology, plaque distribution, etc. As summarized above, frames in IVUS image 418a corresponding to reference points 1002a, 1002b, and 1002c can be mapped to specific frames in IVUS image 418b (e.g., based on vascular reference point 702, etc.), and offset angles between frames can be determined. In another embodiment, the offset angles can be determined based on calculating the correlation between each frame and the rotated version of each frame (e.g., as described above, related to...). Figure 6A and Figure 6B (as described).

[0097] Figure 10BThe diagram shows points representing each offset angle corresponding to reference points 1002a, 1002b, and 1002c. Based on these points, Figure 1004 can be generated, representing the offset angle (plotted on the y-axis 1006) for each frame in IVUS run 418a (plotted on the x-axis 1008). As described above, Figure 1004 can be generated linearly and / or based on one or more line fitting or line smoothing algorithms.

[0098] It should be understood that various techniques and workflows can be combined to identify alignment offsets. As used herein, "alignment offset" is intended to represent an offset distance (e.g., aligning a frame longitudinally) or an offset angle (e.g., aligning a frame angularly), or both. For example, IVUS running frame mapping 420 may include either an offset distance and / or an offset angle, or both. In some examples, the various offset derivation methods outlined herein can be combined on a segment-by-segment basis. For example, the first segment (e.g., Figure 9A The alignment offset of frames in segment 904a, etc., can be determined based on a first selection of the alignment method disclosed herein, while the alignment offset of another segment (e.g., Figure 9A The alignment offset of frames in segments 904b, 904c, etc., may be determined based on a second choice of the alignment method disclosed herein. As a specific example, frames in segment 904a may be aligned using frame-by-frame correlation, while frames in segment 904b may be aligned using inference based on an ML model. However, the claims are not limited to this example, but may include any combination of techniques implemented on a segment-by-segment basis.

[0099] As discussed above, a GUI can be generated to present graphical indications of different IVUS operations in a way that is related to each other, such as, for example, the case of frame alignment as described herein. Figure 11 A GUI 1100 that can be generated according to some embodiments of the present invention is shown. In some embodiments, the GUI 1100 may be... Figure 4 , Figure 7 or Figure 15 The GUI 424. For example, processor 406 can execute instructions 416 to generate a GUI with, for example, GUI 424. Figure 11 The GUI 1100 shows the graphical components and the GUI 424 is arranged. In such an example, the processor 406 can execute instructions 416 to cause the GUI 1100 to be displayed on the display 404.

[0100] GUI 1100 may include graphical representations of IVUS images 418a and 418b. As shown in this example, the graphical representations of IVUS images 418a and 418b include on-axis views (e.g., on-axis views 1102a and 1102b) and portrait views (e.g., portrait views 1104a and 1104b). As shown, GUI 1100 may arrange on-axis views 1102a and 1102b and portrait views 1104a and 1104b in a horizontal (e.g., side-by-side) visualization manner. In other embodiments, processor 406 may execute instructions 416 to generate GUI 1100 to visualize on-axis views 1102a and 1102b in a vertical arrangement.

[0101] Furthermore, the GUI 1100 may include a dual-view slider 1106 and a dual-view slider 1108. The dual-view slider 1108 is manipulatory (e.g., via a touchscreen, mouse, joystick, etc.) to slide (or move) through frames of the IVUS image. As the dual-view slider 1108 moves, the processor 406 may execute instructions 416 to regenerate the GUI 1100, causing frame indicators 1110a and 1110b positioned on the portrait views 1104a and 1104b to move along with the position of the dual-view slider 1108. Additionally, the on-axis views 1102a and 1102b may be modified to correspond to frames from each respective IVUS run that match the positions of the frame indicators 1110a and 1110b.

[0102] Therefore, as provided herein, one or two IVUS runs can be adjusted (e.g., based on offset distance and / or offset angle) to align the IVUS runs with each other. Thus, a user (e.g., a physician) can view different IVUS runs (e.g., before and after a PCI run), where the location of the vessels and the corresponding reference points are aligned in the visualization, such as, for example, as depicted in GUI1100.

[0103] In some embodiments, more than two (2) IVUS runs can be presented in the GUI. For example, Figure 8A and Figure 8B Three (3) IVUS runs are shown, which are shifted to align the IVUS runs with each other. Thus, a GUI 1100 can be generated to present a graphical indication of each of these three (3) IVUS runs.

[0104] Figure 12A logical flow 1200 for aligning different IVUS operations is illustrated according to some embodiments of the present invention. The logical flow 1200 may be implemented by the IVUS image association and visualization systems described herein, such as, for example, IVUS image association and visualization systems 400, 700, etc. For clarity and not limitation, the logical flow 1200 is described with reference to IVUS image association and visualization system 400.

[0105] Logic flow 1200 may begin at block 1202. At block 1202, “Receiving First Series of IVUS Images of a Patient’s Vessel,” a first series of IVUS images captured via an IVUS catheter percutaneously inserted into a patient’s vessel may be received. For example, information elements including indications of IVUS image 418a may be received from IVUS imaging system 100, wherein catheter 102 is percutaneously inserted into vessel 202. IVUS image 418a may include image frames representing images captured as catheter 102 is pulled back from distal end 204 to proximal end 206. Processor 406 may execute instructions 416 to receive information elements including indications of IVUS image 418a from IVUS imaging system 100 or directly from catheter 102 (as applicable).

[0106] Continuing to box 1204, “Receiving a Second Series of IVUS Images of a Patient’s Vessel,” a second series of IVUS images captured via an IVUS catheter percutaneously inserted into a patient’s vessel can be received. For example, information elements including indications of IVUS image 418b can be received from IVUS imaging system 100, wherein catheter 102 is percutaneously inserted into vessel 202. Similar to IVUS image 418a, IVUS image 418b may include image frames representing images captured as catheter 102 is pulled back from distal end 204 to proximal end 206. However, as described above and as contemplated herein, the distal end 204 and proximal end 206 of IVUS image 418a may be located at different locations than the distal end 204 and proximal end 206 of IVUS image 418b. Processor 406 can execute instruction 416 to receive information elements including indications of IVUS image 418b from IVUS imaging system 100 or directly from catheter 102 (as applicable).

[0107] Continuing to box 1206, “Identifying the mapping between frames in the first series of IVUS images and frames in the second series of IVUS images,” the mapping between frames in the first series of IVUS images and frames in the second series of IVUS images can be identified. For example, processor 406 can execute instruction 416 to generate an IVUS running frame map 420 based on ML model 422. In another embodiment, processor 406 can execute ML model 704 to identify vascular reference point 702, and then identify the IVUS running frame map 420 based on vascular reference point 702. In another example, processor 406 can execute instruction 416 to generate the IVUS running frame map 420 based on correlations as outlined above (e.g., frame-by-frame correlation, angle offset frame-by-frame correlation, etc.). In yet another example, processor 406 can execute instruction 416 to generate the IVUS running frame map 420 on a per-segment basis as outlined above.

[0108] In any of the above embodiments, the IVUS running frame mapping 420 may include indications of one or two series of IVUS image offsets (e.g., in time, distance, angle, etc.), which, when applied, will shift the longitudinal direction (e.g., as shown in the image). Figure 8B (The depicted) and / or angled alignment of the IVUS image. As described herein, IVUS runs frame mapping 420 to indicate offset distance and / or offset angle. In this context, the examples are not limited.

[0109] In some examples, processor 406 may execute instruction 416 to map frames based on a longitudinal offset as outlined herein. In such examples, processor 406 may execute instruction 416 to map frames based on partial overlap and temporal warp. It should be understood that one set of IVUS images (e.g., IVUS image 418a, etc.) may be captured at a first pull-back speed, while another set of IVUS images (e.g., IVUS image 418b, etc.) may be captured at a second pull-back speed different from the first pull-back speed. In yet another example, one set of IVUS images (e.g., IVUS image 418a, etc.) may be captured along a first pull-back path through the blood vessel, while another set of IVUS images (e.g., IVUS image 418b, etc.) may be captured along a slightly different pull-back path, or motion artifacts may appear in the captured IVUS images.

[0110] Therefore, although many examples discuss aligning (or co-registering) different IVUS images based on offset distance and / or angle, some embodiments provide that alignment (or co-registering) operations can also be based on motion overlap and / or temporal warp.

[0111] For example, Figure 13Figure 1300 illustrates the alignment of extracted and vectorized features from two IVUS runs through the vessel. Extracted and vectorized features 1302a are generated from IVUS image 418a, while extracted and vectorized features 1302b are generated from IVUS image 418b. These features can be aligned along a longitudinal offset based on temporal warp, as discussed herein. That is, as depicted in the figure, different amounts of longitudinal shift can be made between the frames of the IVUS run to account for different pull-back velocities and paths through the vessel.

[0112] Continuing to box 1208, “Generate a graphical user interface including indications of a first series of IVUS images and a second series of IVUS images, wherein at least one of the first series of IVUS images or the second series of IVUS images is offset (e.g., in time, distance, angle, etc.) based on the mapping to align the first series of IVUS images with the second series of IVUS images in the longitudinal direction and / or by angle,” a GUI can be generated, wherein the GUI includes graphical indications of the first series of IVUS images and the second series of IVUS images, and wherein any number of frames from the first and / or second series of IVUS images are offset (e.g., in time, distance, angle, etc.) to align the first and second series of IVUS images in the longitudinal direction and / or by angle. For example, processor 406 can execute instruction 416 to generate GUI 424, as discussed above. As a concrete example, processor 406 can execute instruction 416 to generate GUI 1100 as GUI 424 and cause GUI 1100 to be displayed on display 404.

[0113] As noted, in some embodiments, the processor 406 of the computing device 402 may execute instructions 416 to generate an IVUS run-frame map 420 using the ML model, or to generate a vascular reference point 702 from the ML model, and subsequently generate the IVUS run-frame map 420 from the vascular reference point 702. In such an example, the ML model may be stored in the memory 408 of the computing device 402. It should be understood that the ML model must be trained prior to deployment. Figure 14 An ML environment 1400 is shown, which can be used to train an ML model that can later be used to generate (or infer) maps or vascular reference points, as outlined herein. The ML environment 1400 may include an ML system 1402, such as a computing device that applies an ML algorithm to learn a relationship between inputs and inferred outputs. In this example, the ML algorithm may learn a relationship between inputs (e.g., IVUS images) and outputs (e.g., frame maps or vascular reference points, depending on the embodiment).

[0114] The ML system 1402 may utilize experimental data 1408 collected during several previous surgeries. Experimental data 1408 may include IVUS images from several IVUS runs from several patients. Experimental data 1408 may be juxtaposed with the ML system 1402 (e.g., stored in storage device 1410 of the ML system 1402), may be remote from the ML system 1402 and accessed via network interface 1504, or may be a combination of local and remote data.

[0115] Experimental data 1408 can be used to form training data 1412. As described above, the ML system 1402 may include a storage device 1410, which may include a hard disk drive, a solid-state storage device, and / or random access memory. The storage device 1410 can hold the training data 1412. Typically, the training data 1412 may include information elements or data structures that include indications of multiple series of IVUS images and corresponding desired outputs (e.g., mappings or vascular reference points). It should be understood that when the desired output is an IVUS frame mapping, the input may be two (or more, depending on the case) series of IVUS images. For reference Figure 4 In a concrete example, where ML model 1424 is trained and deployed as ML model 422, the input can be multiple pairs of first-series and second-series IVUS images (e.g., more than one IVUS run), and the output can be a mapping associated with each pair of first-series and second-series IVUS images (e.g., a mapping between IVUS runs). In another example, refer to... Figure 7 In the case where ML model 1424 is to be trained and deployed as ML model 704, the input can be a single series of IVUS images (e.g., a single IVUS run), and the output can be frames in the IVUS images in which vascular reference points (or multiple reference points) are identified.

[0116] Training data 1412 can be used to train ML model 1424. Depending on the application, different types of models can be used to form the basis of ML model 1424. For example, in this example, an artificial neural network (ANN) may be particularly well-suited for learning the correlation between IVUS images (e.g., IVUS images 418a, IVUS images 418b, etc.) and reference points or frame maps (e.g., IVUS running frame map 420, vascular reference point 702, etc.). Convolutional neural networks are also well-suited for this task. In another example, ML model 1424 may be based on a spatial transformer (e.g., a spatial transformation network, etc.). As yet another example, ML model 1424 can be multiple networks, such as, for example, a Siamese network, etc.

[0117] Any suitable training algorithm 1420 can be used to train the ML model 1424. For example, the examples described herein may be suitable for supervised training algorithms or reinforcement learning training algorithms. For supervised training algorithms, the ML system 1402 may apply an IVUS image 1414 as input 1430, to which the ML model 1424 can generate a desired output (e.g., a mapping or benchmark). In a reinforcement learning scenario, the training algorithm 1420 may attempt to maximize some or all (or a weighted combination) of the model inputs 1430 mapped to the output 1426 to produce an ML model 1424 with minimum error. In some embodiments, the training data 1412 may be divided into “training” data and “test” data, wherein a subset of the training data 1412 may be used to adjust the ML model 1424 (e.g., the model’s internal weights, etc.), while another non-overlapping subset of the training data 1412 may be used to measure the accuracy of the ML model 1424 to infer (or generalize) the output 1426 from the “invisible” training data 1430.

[0118] The ML model 1424 can be applied using processor circuitry 1406, which may include suitable hardware processing resources for operating on the logic and structure in storage device 1410. The development of training algorithm 1420 and / or the trained ML model 1424 may depend at least in part on hyperparameters 1422. In an exemplary embodiment, model hyperparameters 1422 may be automatically selected based on logic 1428, which may include any known hyperparameter optimization techniques to suit the selected ML model 1424 and the training algorithm 1420 to be used. In alternative embodiments, the ML model 1424 may be retrained over time to incorporate new knowledge and / or updated experimental data 1424.

[0119] Once the ML model 1424 is trained, it can be applied (e.g., by processor 406, etc.) to new input data (e.g., IVUS images 418a, IVUS images 418b, etc.). This input to the ML model (e.g., ML model 422, ML model 702, etc.) can be formatted according to a predefined model input 1430, which is a mirror image of the way the training data 1412 is provided to the ML model 1424. The ML model 1424 can generate an output 1426, which can be, for example, a generalization or IVUS run frame map 420 or vascular reference point 702 as discussed above.

[0120] The above description relates to a specific type of ML system 1402 that applies supervised learning techniques based on available training data with input / output pairs. However, the invention is not limited to use with a particular ML paradigm, and other types of ML techniques can be used. For example, in some embodiments, ML system 1402 may apply, for example, evolutionary algorithms or other types of ML algorithms and models to IVUS running frame mapping 420 (or vascular reference point 702, as appropriate) based on IVUS images 418a and / or IVUS images 418b.

[0121] In some embodiments, ML model 1424 may be a conventional ML model, such as, for example, a neural network, a convolutional neural network, an evolutionary artificial neural network, etc. However, in some embodiments, ML model 1424 may not be an ML model in the conventional sense. For example, ML model 1424 may be a dynamically programmed algorithm, wherein the parameters of the dynamically programmed algorithm are adjusted using training data 1412.

[0122] In some embodiments, the invention may be provided for angularly aligning IVUS operation with views of blood vessels from an extravascular imaging modality. For example, Figure 15 An IVUS image association and visualization system 1500 according to some embodiments of the present invention is illustrated. Generally, the IVUS image association and visualization system 1500 is a system for processing, associating, and presenting IVUS images with external images of the same blood vessel. For the sake of simplicity, many components of the IVUS image association and visualization system 400 are referenced and reused in the description of the IVUS image association and visualization system 1500.

[0123] As mentioned above Figure 4 As described in the IVUS image association and visualization system 400, the present invention provides an IVUS running frame mapping 420 for IVUS images 418a and 418b. In some embodiments, the IVUS running frame mapping 420 may be generated based on an external image of the blood vessel. It should be noted that various techniques exist for co-registering intravascular images (e.g., IVUS images 418a and / or 418b) with external images. Such techniques are not described again herein. However, for clarity, it should be noted that reference points can be identified on the external image as well as on the intravascular image, and reference points are mapped to each other to co-register frames in the intravascular image to points on the external image (e.g., in x and y coordinates).

[0124] Therefore, in some examples, the IVUS image association and visualization system 1500 may be coupled to an external imaging system 1506 (e.g., an angiography machine, a computed tomography (CT) machine, a magnetic resonance imaging (MRI) machine, etc.) configured to capture external images of a blood vessel, with which IVUS images 418a and / or 418b are captured. Alternatively, the IVUS image association and visualization system 1500 may be coupled to a memory device storing external images or frames of external images.

[0125] Processor 406 can execute instructions 416 to receive external image 1502 (or multiple images) from external imaging system 1506 (or memory storage device). Processor 406 can execute instructions 416 to identify reference points in external image 1502 and IVUS image 418a (or IVUS image 418b). For example, processor 406 can execute instructions 416 to identify vascular reference point 702 corresponding to the reference point in IVUS image 418a and external image 1502.

[0126] As summarized above, various techniques exist for identifying reference points in both internal and external imaging modalities. For example, collateral identification and matching are frequently used to co-register internal images to external images. This invention provides processor 406 executable instructions 416 to identify reference points and their locations, and to identify the angles of the reference points, and to store indications of the reference point locations and angles in vascular reference point 702. In some embodiments, processor 406 may use image processing techniques and / or ML inference to identify the angles of the reference points. For example, ML model 702 may be trained as described above to identify reference points and their corresponding angles from external image 1502. Once the angles of the reference points in external image 1502 are identified, processor 406 may execute instructions 416 to identify an offset angle (e.g., IVUS running frame mapping 420, etc.) to rotate frames of IVUS images (e.g., IVUS images 418a and / or 418b) to align the observation angle with the observation angle of external image 1502. Furthermore, processor 406 may also execute instruction 416 to identify the offset of other frames in the IVUS image based on the offset angle of the frame corresponding to the reference point (as mentioned above). Figure 10A and Figure 10B (As outlined above).

[0127] For example, Figure 16AAn external image 1502 and two identified reference points (e.g., lateral branches) 1602a and 1602b are shown. Processor 406 can execute instructions 416 to identify the angles of reference points 1602a and 1602b. It should be noted that the angles of the reference points are derived based on a baseline, such as setting zero (0) degrees from the Z-direction of the two-dimensional (2D) image toward the observer. Processor 406 can execute instructions 416 to rotate the IVUS image 418a to a frame (or derive the angular offset) that matches reference points 1602a and 1602b, based on the angles of reference points 1602a and 1602b.

[0128] For example, Figure 16B and Figure 16C Image frames 1604a and 1604b (e.g., frames from IVUS image 418a, etc.) depicting reference points 1602a and 1602b, respectively. Processor 406 is executable with instructions 416 to rotate image frames 1604a and 1604b based on the angles of the vascular reference points represented in the external image 1502 (e.g., collateral angles, etc.) and the angles of the reference points in each of the respective frames 1604a and 1604b, resulting in rotated image frames 1606a and 1606b. Figure 16B and Figure 16C The image frames 1606a and 1606b are depicted in rotation, respectively.

[0129] In some examples, image frames can be rotated based on reference point markers. For example, in Figure 16B A reference point marker 1610 is depicted in the image. In some embodiments, the processor 406 may execute instructions 416 to identify the reference point marker and rotate the image frame based on the angle of the reference point marker. For example, the reference point marker 1610 (e.g., a lateral branch) in image frame 1604a is depicted as being located approximately at 9 o'clock or 270 degrees. A frame may be rotated by an angle based on the angle of a reference point marker in another image frame, such that the reference point marker is aligned at a specific angle. For example, rotated image frame 1606a shows a reference point marker rotated to 180 degrees.

[0130] Therefore, as outlined above, processor 406 can execute instruction 416 to align the viewing angles of the frame and the external image (e.g., external image 1502, etc.) within IVUS operation by angle, such that the angle of the viewing reference point is aligned between the two imaging modalities. Figure 16DA set of IVUS images 1608 aligned with external images is shown, which may correspond to frames in IVUS image 418a (etc.) whose viewing angle (or perspective) has been aligned with external image frame 1502. It should be noted that this provides a significant improvement over conventional techniques. It should be understood that intravascular images are generally independent of the viewing angle. For example, IVUS images are captured as the ultrasound transducer rotates within the blood vessel. Therefore, the actual viewing angle between frames can vary. Furthermore, the viewing angle of the external image can also vary (e.g., based on the patient's position relative to the image acquisition system, etc.). Therefore, the perspective of view between intravascular and extravascular images is generally not aligned. This invention solves this problem.

[0131] Furthermore, as discussed above, a GUI can be generated to present graphical indications of aligned IVUS operations. For example, a GUI can be generated to present a visual representation of frames of IVUS operations aligned with vessels observed in external images. Figure 17 A GUI 1700 that can be generated according to some embodiments of the present invention is shown. In some embodiments, the GUI 1700 may be... Figure 4 , Figure 7 or Figure 15 The GUI 424. For example, processor 406 can execute instructions 416 to generate a GUI with, for example, GUI 424. Figure 17 The GUI 1700 shows the graphical components and the GUI 424 are arranged. In such an example, the processor 406 can execute instructions 416 to cause the GUI 1700 to be displayed on the display 404.

[0132] GUI1700 may include an external image 1502 and a graphical representation of an IVUS image 1608 aligned with the external IVUS image. Thus, when a physician (or user) examines a frame of IVUS image 418a, the IVUS image 1608 aligned with the external image is presented, such that lumens and reference points as observed in the IVUS image frame are matched at angles with vessels and reference points (e.g., reference points 1602a and 1602b) as observed in the external image frame.

[0133] Figure 18A computer-readable storage medium 1800 is shown. The computer-readable storage medium 1800 may include any non-transitory computer-readable or machine-readable storage medium, such as optical, magnetic, or semiconductor storage media. In various embodiments, the computer-readable storage medium 1800 may include an article of manufacture. In some embodiments, the computer-readable storage medium 1800 may store computer-executable instructions 1802 executable by circuitry (e.g., processor 106, processor 406, processor circuitry 1406, etc.). For example, the computer-executable instructions 1802 may include instructions for performing operations described with respect to instructions 416 and / or logic flow 1200. Examples of computer-readable or machine-readable storage media 1800 may include any tangible medium capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, and so on. Examples of computer-executable instructions 1802 may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, etc.

[0134] Figure 19 A diagram is shown of a machine 1900 in the form of a computer system, within which a set of instructions can be executed to cause the machine to perform any or more of the methods discussed herein. More specifically, Figure 19 The illustration shows a machine 1900 in the form of an example computer system, within which executable instructions 1908 (e.g., software, programs, applications, applets, applications, or other executable code) cause the machine 1900 to perform any or more of the methods discussed herein. For example, instruction 1908 may cause the machine 1900 to execute... Figure 12 The logical flow 1200, etc. More generally, instruction 1908 enables machine 1900 to automatically determine the mapping (e.g., in terms of time, distance, angle, etc.) between frames of different IVUS runs through the same blood vessel (e.g., from IVUS runs before PCI, IVUS runs during PCI, and / or IVUS runs after PCI) and / or between IVUS runs and external images.

[0135] Instruction 1908 transforms a general, unprogrammed machine 1900 into a specific machine 1900 programmed to perform the described and illustrated functions in a particular manner. In alternative embodiments, machine 1900 operates as a standalone device or is connectable (e.g., networked) to other machines. In a networked deployment, machine 1900 may operate as a server machine or client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 1900 may include, but is not limited to, server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, set-top boxes (STBs), PDAs, entertainment media systems, cellular phones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart appliances), other smart devices, network devices, network routers, network switches, bridges, or any machine capable of sequentially or otherwise executing instructions 1908 specifying actions to be taken by machine 1900. Furthermore, although only a single machine 1900 is shown, the term "machine" should also be considered as a collection of machines 1900 that individually or jointly execute instructions 1908 to perform any one or more of the methods discussed herein.

[0136] Machine 1900 may include processor 1902, memory 1904, and I / O components 1942, which may be configured to communicate with each other, for example, via bus 1944. In one example embodiment, processor 1902 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processor 1906 and processor 1910 that can execute instructions 1908. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously. Although Figure 19 Multiple processors 1902 are shown, but machine 1900 may also include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0137] Memory 1904 may include main memory 1912, static memory 1914, and memory cell 1916, all of which may be accessed by processor 1902, for example, via bus 1944. Main memory 1904, static memory 1914, and memory cell 1916 store instructions 1908 that embody one or more of the methods or functions described herein. During execution of instructions 1908 by machine 1900, instructions 1908 may also reside wholly or partially in main memory 1912, static memory 1914, machine-readable medium 1918 within memory cell 1916, at least one of processor 1902 (e.g., within the processor's cache), or any suitable combination thereof.

[0138] I / O component 1942 may include a wide variety of components to receive input, provide output, generate output, transmit information, exchange information, capture measurement results, and so on. The specific I / O component 1942 included in a particular machine will depend on the type of machine. For example, a portable machine, such as a mobile phone, may include a touch input device or other such input mechanism, while a headless server machine may not include such a touch input device. It should be understood that I / O component 1942 may include... Figure 19 Many other components are not shown. The I / O components 1942 are grouped according to function only for the sake of simplicity in the discussion below, and this grouping is by no means limiting. In various example embodiments, the I / O components 1942 may include output components 1928 and input components 1930. Output components 1928 may include visual components (e.g., displays, such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), acoustic components (e.g., speakers), haptic components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. Input components 1930 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other orientation devices), haptic input components (e.g., physical buttons, touchscreens providing position and / or force for touch or touch gestures, or other haptic input components), audio input components (e.g., microphones), etc.

[0139] In another example embodiment, I / O component 1942 may include biometric component 1932, motion component 1934, environmental component 1936 or position component 1938, and various other components. For example, biometric component 1932 may include components for detecting expressions (e.g., hand gestures, facial expressions, vocal expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and recognizing a person (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 1934 may include accelerometer components (e.g., accelerometer), gravity sensor components, rotation sensor components (e.g., gyroscope), etc. Environmental component 1936 may include, for example, an illuminance sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers for detecting ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones for detecting background noise), a proximity sensor component (e.g., an infrared sensor for detecting nearby objects), a gas sensor (e.g., a gas detection sensor for detecting hazardous gas concentrations to ensure safety or measuring pollutants in the atmosphere), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. Position component 1938 may include a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer for detecting air pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), etc.

[0140] Communication can be implemented using various technologies. I / O component 1942 may include communication component 1940 operable to connect machine 1900 to network 1920 or device 1922 via connectors 1924 and 1926, respectively. For example, communication component 1940 may include a network interface component or another suitable device that interfaces with network 1920. In further examples, communication component 1940 may include wired communication components, wireless communication components, cellular communication components, near field communication (NFC) components, Bluetooth® components (e.g., Bluetooth® Low Energy), Wi-Fi® components, and other communication components that provide communication via other means. Device 1922 may be another machine or any of a variety of peripheral devices (e.g., a peripheral device connected via USB).

[0141] Furthermore, the communication component 1940 may detect identifiers or include components operable to detect identifiers. For example, the communication component 1940 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, multi-dimensional barcodes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D barcodes, and other optical codes) or an acoustic detection component (e.g., a microphone for identifying audio signals from the tag). Additionally, various information can be derived via the communication component 1940, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detecting NFC beacon signals that indicate a specific location, and so on.

[0142] Various memories (i.e., memory 1904, main memory 1912, static memory 1914, and / or the memory of processor 1902) and / or storage units 1916 may store one or more sets of instructions and data structures (e.g., software) that embody or are utilized by any one or more of the methods or functions described herein. These instructions (e.g., instruction 1908) cause various operations to implement the disclosed embodiments when executed by processor 1902.

[0143] As used herein, the terms “machine storage medium,” “device storage medium,” and “computer storage medium” refer to the same thing and are used interchangeably in this invention. The terms refer to one or more storage devices and / or media (e.g., centralized or distributed databases and / or associated caches and servers) that store executable instructions and / or data. Therefore, the terms should be considered to include, but are not limited to, solid-state memory and optical and magnetic media, including memory internal or external to the processor. Specific examples of machine storage media, computer storage media, and / or device storage media include non-volatile memory, such as semiconductor memory devices, e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices; disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms “machine storage medium,” “computer storage medium,” and “device storage medium” explicitly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered under the term “signal medium” discussed below.

[0144] In various example embodiments, one or more portions of network 1920 may be an ad hoc network, intranet, extranet, VPN, LAN, WLAN, WAN, WWAN, MAN, Internet, a portion of PSTN, a common old-style telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, network 1920 or a portion thereof may include a wireless or cellular network, and coupler 1924 may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or another type of cellular or wireless coupler. In this example, the coupler 1924 can implement any of a variety of data transmission technologies, such as single-carrier radio transmission technology (1xRTT), evolved data optimization (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate GSM evolution (EDGE) technology, 3rd generation partnership program (3GPP) (including 3G), fourth generation wireless (4G) network, universal mobile telecommunications system (UMTS), high-speed packet access (HSPA), global microwave access interoperability (WiMAX), long-term evolution (LTE) standard, other standards defined by various standards-setting organizations, other remote protocols or other data transmission technologies.

[0145] Instruction 1908 may be transmitted or received via network 1920 using a transmission medium via a network interface device (e.g., a network interface device included in communication component 1940) and utilizing any of several well-known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instruction 1908 may be transmitted or received via a transmission medium via a coupler to device 1922 (e.g., a peer-to-peer coupler). The terms "transmission medium" and "signal medium" refer to the same thing and are used interchangeably in this invention. The terms "transmission medium" and "signal medium" should be considered to include any intangible medium that can store, encode, or carry instructions for execution by machine 1900; and include digital or analog communication signals or other intangible media to facilitate communication of such software. Therefore, the terms "transmission medium" and "signal medium" should be considered to include any form of modulated data signal, carrier wave, etc. The term "modulated data signal" refers to a signal whose characteristics are set or altered in a manner to encode information in the signal.

[0146] The terms used in this document should conform to their ordinary meaning in the relevant field, or the meaning indicated by their use in the context, unless a clear definition is provided.

[0147] In this document, references to “an embodiment” or “one embodiment” do not necessarily refer to the same embodiment, although they may refer to the same embodiment. Unless the context explicitly requires otherwise, throughout the specification and claims, the words “comprising,” “including,” etc., should be interpreted in an inclusive sense, rather than an exclusive or exhaustive sense; that is, meaning “including but not limited to.” Use of singular or plural terms also includes both singular and plural, unless explicitly limited to one or more. Additionally, the words “this document,” “above,” “below,” and similar terms, when used in this application, refer to the entire application and not any part thereof. When a claim uses the word “or” to refer to a list of two or more items, the word covers all of the following interpretations: any one of the items in the list, all the items in the list, and any combination of the items in the list, unless explicitly limited to one or the other. Any term not explicitly defined herein has its conventional meaning as commonly understood by one of ordinary skill in the art.

Claims

1. A method for a computing device, comprising: The processor receives a first series of intravascular ultrasound (IVUS) images of the patient's blood vessels, the first series of IVUS images comprising a first plurality of frames; The processor receives a second series of intravascular ultrasound (IVUS) images of the patient's blood vessels, the second series of IVUS images comprising a second plurality of frames; The offset of the first plurality of frames is determined by the processor based at least in part on the second plurality of frames; The processor applies the offset to the first plurality of frames to generate an offset IVUS image series; as well as The processor generates a graphical user interface (GUI) that includes an offset IVUS image series and an indication of the second series of IVUS images.

2. The method of claim 1, wherein determining the offset of the first plurality of frames comprises: Identify frames in the first plurality of frames that include vascular reference points; Identify frames in the second plurality of frames that include the vascular reference point; as well as The offset of the first plurality of frames is determined, and when applied, the offset is used to align the frame in the first plurality of frames that includes the vascular reference point with the frame in the second plurality of frames that includes the vascular reference point.

3. The method of claim 2, wherein the offset includes a first offset and a second offset, and wherein determining the offset of the first plurality of frames includes: Identify the first frame among the plurality of frames that includes the first blood vessel reference point; Identify the second frame in the second plurality of frames that includes the first blood vessel reference point; Determine the first offset of the first plurality of frames, and when the first offset is applied to the first segment of the first plurality of frames, align the first frame of the first plurality of frames with the first frame of the second plurality of frames; Identify the second frame among the first plurality of frames that includes the second blood vessel reference point; Identify the second frame in the plurality of frames that includes the second blood vessel reference point; as well as Determine the second offset of the first plurality of frames, and when the second offset is applied to a second segment in the first plurality of frames that is different from the first segment, align the second frame in the first plurality of frames with the second frame in the second plurality of frames. The second offset is different from the first offset.

4. The method of claim 3, wherein the first offset includes an offset distance and the second offset includes an offset angle, or wherein the first offset includes an offset distance or an offset angle and the second offset includes an offset distance and an offset angle.

5. The method of claim 2, wherein identifying the frame including the vascular reference point in the first plurality of frames and wherein identifying the frame including the vascular reference point in the second plurality of frames comprises: Execute a machine learning (ML) model to infer the frames in the first plurality of frames that include the vascular reference point; as well as The ML model is executed to infer the frames in the second plurality of frames that include the vascular reference point.

6. The method according to any one of claims 2 to 5, wherein the vascular reference point is one of the following: lumen geometry, vascular geometry, collateral location, calcium morphology, plaque distribution, or guiding catheter location.

7. The method according to any one of claims 1 to 6, wherein determining the offset of the first plurality of frames comprises: Based on the frame-by-frame correlation with the second plurality of frames, a correlation score is calculated for each frame in the first plurality of frames; Identify the frame with the highest relevance score among the first plurality of frames, and the frame associated with the highest relevance score among the second plurality of frames; as well as The offset of the first plurality of frames is determined, and when applied, the offset aligns the frame with the highest correlation score among the first plurality of frames with the frame associated with the highest correlation score among the second plurality of frames.

8. The method of claim 7, wherein the offset is an offset distance, and wherein the method further comprises: Based on the frame-by-frame correlation with the angle offset of the second plurality of frames, the correlation score of each frame in the first plurality of frames is calculated; Identify the frame with the highest correlation score among the first plurality of frames, and the frame or its rotated version associated with the highest correlation score among the second plurality of frames; as well as Based on the frame or its rotated version associated with the highest correlation score in the second plurality of frames, an offset angle is determined for the first plurality of frames. This offset angle, when applied, aligns the frame with the highest correlation score in the first plurality of frames with the frame associated with the highest correlation score in the second plurality of frames. The offset IVUS image series is generated by applying the offset distance and the offset angle to the first plurality of frames.

9. The method of claim 1, wherein determining the offset of the first plurality of frames comprises: Based on the frame-by-frame correlation with the angle offset of the second plurality of frames, the correlation score of each frame in the first plurality of frames is calculated; Identify the frame with the highest correlation score among the first plurality of frames, and the frame or its rotated version associated with the highest correlation score among the second plurality of frames; as well as The offset of the first plurality of frames is determined based on the frame or a rotated version of the frame associated with the highest correlation score in the second plurality of frames, and the offset, when applied, aligns the frame with the highest correlation score in the first plurality of frames with the frame associated with the highest correlation score in the second plurality of frames.

10. The method according to any one of claims 1 to 3 or 5 to 9, wherein the offset of the first plurality of frames is a distance offset, an angle offset, or a distance and angle offset.

11. The method according to any one of claims 1 to 10, comprising: Receive the second series of IVUS images from the intravascular imaging device; as well as Receive the first series of IVUS images from the memory storage device.

12. The method according to any one of claims 1 to 11, wherein the first series of IVUS images is captured prior to percutaneous coronary intervention (PCI) procedure.

13. The method of claim 12, wherein the second series of IVUS images is captured during or after a PCI procedure.

14. The method according to any one of claims 1 to 13, wherein the GUI comprises a longitudinal view of the first series of IVUS images and the second series of IVUS images, and wherein the longitudinal views are set at a common scale.

15. An apparatus for an intravascular imaging system, comprising: processor; as well as A memory device coupled to the processor, the memory device including instructions executable by the processor, the instructions, when executed by the processor, causing the intravascular imaging system to perform the method according to any one of claims 1 to 14.

Images