Three-dimensional vascular reconstruction based on intravascular ultrasound images

Abstract

The present disclosure provides for generating a 3D visualization of a blood vessel from intravascular ultrasound (IVUS) images. In particular, the present disclosure provides for reducing jitter between frames of an IVUS record to smoother the appearance of a longitudinal view of the blood vessel from the IVUS image frames, and constructing a 3D visualization of the blood vessel from jitter compensated IVUS image frames.

Description

Cross-references to related applications

[0001] This application claims the benefit of U.S. Provisional Patent Application No. 63 / 519,380, filed August 14, 2023, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] This disclosure relates to the generation of three-dimensional (3D) reconstruction of blood vessels based on intravascular ultrasound (IVUS) images. Background Technology

[0003] Doctors use a variety of imaging modalities and / or physiological measurements to assess the severity of stenosis in blood vessels. For example, doctors typically analyze images outside the vessel (e.g., angiography) as well as images inside the vessel (e.g., intravascular ultrasound, optical coherence tomography). Furthermore, when analyzing the severity of stenosis, doctors frequently refer to physiological measurements such as fractional flow reserve.

[0004] Given the diversity and complexity of the information physicians review for both pre-treatment planning and post-treatment analysis, graphical interfaces displaying this information often appear cluttered. Furthermore, while anatomical structures are three-dimensional (3D), information is typically displayed in two-dimensional (2D). Therefore, untrained personnel (e.g., patients, caregivers, treatment decision-makers, etc.) may struggle to fully grasp the necessity and / or benefits of treatment.

[0005] Therefore, there is a need to provide images or models of vascular anatomy that are easier for untrained users to interpret. Summary of the Invention

[0006] This disclosure provides for generating 3D visualizations of blood vessels from intravascular ultrasound (IVUS) images. In particular, this disclosure provides for reducing jitter between frames of IVUS recording to make the longitudinal view of blood vessels from IVUS image frames appear smoother, and constructing 3D visualizations of blood vessels based on jitter-compensated IVUS image frames.

[0007] Therefore, this disclosure provides a system for generating 3D reconstructions of blood vessels to provide a virtual physiological profile of the vessels. Physicians can use this virtual physiological profile to gain a more comprehensive understanding of percutaneous coronary intervention (PCI) both before and after the procedure. This virtual physiological profile can be shared with untrained users to help them understand the treatment plan and outcomes.

[0008] [To be completed after the claims are finalized] Attached Figure Description

[0009] To facilitate identification of any discussion of an element or action, the most prominent number or numbers in the figure labels indicate the figure number in which the element first appears.

[0010] Figure 1 An endovascular treatment system according to one embodiment is shown.

[0011] Figure 2 A process 200 for predicting vascular compliance according to one embodiment is shown.

[0012] Figure 3 A process 300 for predicting vascular compliance according to one embodiment is shown.

[0013] Figure 4 A process 400 for predicting vascular compliance according to one embodiment is shown.

[0014] Figure 5A One aspect of the subject matter according to one embodiment is shown.

[0015] Figure 5B One aspect of the subject matter according to one embodiment is shown.

[0016] Figure 6A One aspect of the subject matter according to one embodiment is shown.

[0017] Figure 6B One aspect of the subject matter according to one embodiment is shown.

[0018] Figure 7A An exemplary artificial intelligence / machine learning (AI / ML) system suitable for exemplary embodiments is shown.

[0019] Figure 7B An exemplary artificial intelligence / machine learning (AI / ML) system suitable for exemplary embodiments is shown.

[0020] Figure 7C An exemplary artificial intelligence / machine learning (AI / ML) system suitable for exemplary embodiments is shown.

[0021] Figure 8 A computer-readable storage medium 800 according to one embodiment is shown.

[0022] Figure 9A Another endovascular treatment system according to another embodiment is shown.

[0023] Figure 9B It shows Figure 9A It is part of an endovascular treatment system.

[0024] Figure 9C It shows Figure 9A It is part of an endovascular treatment system.

[0025] Figure 10 A schematic representation of a machine 1000 according to an example embodiment is shown, which is in the form of a computer system within which a set of instructions can be executed to cause the machine to perform any or more methods discussed herein. Detailed Implementation

[0026] As described above, in an exemplary embodiment, a system is provided that generates 3D visualizations of blood vessels from a series of intravascular ultrasound (IVUS) images. Although this disclosure uses the aorta and coronary arteries as examples, the disclosed system and method can also be used to generate 3D visualizations of other types of blood vessels.

[0027] Figure 1 A vascular visualization system 100 according to an embodiment of the present disclosure is illustrated. Typically, the vascular visualization system 100 is a system for generating 3D visualizations of blood vessels based on intravascular images. For this purpose, the vascular visualization system 100 includes an intravascular imager 102 and a computing device 104. The intravascular imager 102 can be any of various intravascular imagers (e.g., IVUS, OCT, OCE, etc.). In a specific example, the intravascular imager 102 can be as described below. Figure 9A , Figure 9B and Figure 9C The endovascular treatment system 900 is described.

[0028] The computing device 104 can be any of a variety of computing devices. In some embodiments, the computing device 104 may be integrated into and / or implemented therein in the console of the intravascular imager 102. In some embodiments, the computing device 104 may be a workstation or server communicatively connected to the intravascular imager 102. In other embodiments, the computing device 104 may be provided by a cloud-based computing device, for example, through a compute-as-a-service system accessible via a network (e.g., the Internet, intranet, wide area network, etc.). The computing device 104 may include a processor 110, a memory 112, input and / or output (I / O) devices 114, and a network interface 118.

[0029] Processor 110 may include circuitry or processor logic, such as, for example, any of a variety of commercially available processors. In some examples, processor 110 may include multiple processors, a multi-threaded processor, a multi-core processor (whether the multiple cores coexist on the same die or different dies), and / or some other type of multiprocessor architecture that links multiple physically separate processors in some way. Furthermore, in some examples, processor 110 may include a graphics processing section and may include dedicated memory, multi-threaded processing, and / or some other parallel processing capability. In some examples, processor 110 may be an application-specific integrated circuit (ASIC) or a field-programmable integrated circuit (FPGA).

[0030] Memory 112 may include logic circuitry, a portion of which includes an array of integrated circuits forming a non-volatile memory for persistent data storage, or a combination of non-volatile and volatile memory. It should be understood that memory 112 may be based on any of a variety of technologies. In particular, the array of integrated circuits contained in memory 112 may be arranged to form one or more types of memory, such as, for example, dynamic random access memory (DRAM), NAND memory, NOR memory, etc.

[0031] I / O device 114 can be any of a variety of devices for receiving input and / or providing output. For example, I / O device 114 may include a keyboard, mouse, joystick, foot pedal, haptic feedback device, LED, etc. Display 116 can be a conventional display or a touch display. In addition, display 116 can utilize various display technologies, such as liquid crystal display (LCD), light-emitting diode (LED), or organic light-emitting diode (OLED).

[0032] Network interface 118 may include logical and / or features for supporting communication interfaces. For example, network interface 118 may include one or more interfaces that operate according to various communication protocols or standards for direct communication or communication via a network communication link. Direct communication can be achieved using communication protocols or standards described in one or more industry standards, including their successors and variations. For example, network interface 118 may facilitate communication via buses such as, for example, Peripheral Component Interconnect High Speed ​​(PCIe), Non-Volatile Memory High Speed ​​(NVMe), Universal Serial Bus (USB), System Management Bus (SMBus), SAS (e.g., Serial Attached Small Computer System Interface (SCSI)) interfaces, Serial ATA (SATA) interfaces, etc. Furthermore, network interface 118 may include logical and / or features for implementing communication via various wired or wireless network standards (e.g., the 802.11 communication standard). For example, network interface 118 may be configured to support wired communication protocols or standards such as Ethernet. As another example, network interface 118 can be configured to support wireless communication protocols or standards, such as Wi-Fi, Bluetooth, ZigBee, LTE, 5G, etc.

[0033] The memory 112 may include instructions 120, IVUS image frames 122, key features 124, IVUS image frame masks 126, frame alignment parameters 128, aligned IVUS image frames 130, blood vessel volume 132, voxel color 134, graphic information elements 136, and machine learning models 138.

[0034] During operation, processor 110 may execute instructions 120 to cause computing device 104 to receive IVUS image frames 122 from intravascular imager 102. Typically, IVUS image frame 122 is a multidimensional, multivariable image that includes indications of vessel type, lesion in the vessel, lesion type, stent detection, lumen boundary, lumen size, minimum lumen area (MLA), media boundary (e.g., media boundary of the intima-media), media size, calcification angle / radian, calcification coverage, and combinations thereof.

[0035] Processor 110 may further execute instructions 120 to cause computing device 104 to determine key features 124 from IVUS image frame 122. For example, processor 110 may execute instructions 120 to automatically determine the lumen area at various points along the blood vessel from IVUS image frame 122. As another example, processor 110 may execute instructions 120 to automatically determine the vessel boundary at various points along the blood vessel from IVUS image frame 122. As another example, processor 110 may execute instructions 120 to automatically determine the vascular plaque load at various points along the blood vessel from IVUS image frame 122. These are just a few examples of evaluations that can be represented in key features 124. Other examples may include calcification load, collaterals, etc. In some embodiments, key features 124 may be inferred from machine learning model 138. For example, processor 110 may execute machine learning model 138 to infer key features 124 (e.g., boundaries, plaque load, etc.) from IVUS image frame 122.

[0036] Processor 110 may further execute instructions 120 to cause computing device 104 to generate an IVUS image frame mask 126 from IVUS image frames 122 and key features 124. For example, processor 110 may execute instructions 120 to generate a mask for each IVUS image frame 122, the mask including indications of key features 124 (e.g., luminal boundaries, vessel boundaries, plaque burden, etc.). In some embodiments, processor 110 may execute a machine learning model 138 to infer the IVUS image frame mask 126 from IVUS image frames 122 and key features 124. In other embodiments, processor 110 may execute a machine learning model 138 to infer the IVUS image frame mask 126 from IVUS image frames 122 only.

[0037] Processor 110 may further execute instructions 120 to cause computing device 104 to generate frame alignment parameters 128 from IVUS image frame 122 and IVUS image frame mask 126. For example, processor 110 may execute instructions 120 to cause computing device 104 to implement a self-registration algorithm that takes IVUS image frame mask 126 and derives frame alignment parameters 128 for each frame to produce a best-fit 3D model when the frames are stacked together. These adjustments may include, but are not limited to, affine transformations (e.g., translation, rotation, and area-preserving distortion) and B-spline transformations. These adjustments filter out transients (e.g., catheter sloshing, vascular deformation between heartbeats, etc.) while preserving the integrity of key metrics (e.g., lumen / vessel area, mean diameter, stenosis, and plaque load).

[0038] The processor 110 may further execute instructions 120 to cause the computing device 104 to resample and / or align the IVUS image frame 122 using frame alignment parameters 128, thereby generating an aligned IVUS image frame 130. For example, the position of each frame of the IVUS image frame 122 (e.g., relative to adjacent frames, relative to a fixed reference point, etc.) may be adjusted based on the frame alignment parameters 128 to generate the aligned IVUS image frame 130.

[0039] The processor 110 may further execute instructions 120 to cause the computing device 104 to generate a blood vessel volume 132 from the aligned IVUS image frames 130. For example, the frames of the aligned IVUS image frames 130 may be stacked to form a 3D volume of blood vessels, which may be represented by the blood vessel volume 132.

[0040] The processor 110 may further execute instructions 120 to cause the computing device 104 to determine a color for each voxel (e.g., pixel) of the blood vessel volume 132 based on key features 124, wherein the color can indicate the key features. For example, healthy blood vessel boundaries may be assigned a first color (e.g., brown, etc.), calcified plaques may be assigned a second color (e.g., gray, etc.), non-calcified plaques may be assigned a third color (e.g., yellow, etc.), luminal boundaries may be assigned a fourth color (e.g., transparent, etc.), and stents may be assigned a fifth color (e.g., white, etc.). In this way, the blood vessel volume 132 can be rendered in a manner that indicates the key features 124.

[0041] Furthermore, in some embodiments, the processor 110 may execute instructions 120 to generate a graphical information element 136 including an indication of the blood vessel volume 132, and display the graphical information element 136 on the display 116 to a user. For example, the processor 110 may execute instructions 120 to render the blood vessel volume 132 as a 3D volume using voxel colors 134, and generate a graphical user interface (GUI) including an indication of the rendered 3D volume.

[0042] In some embodiments, processor 110 may execute instructions 120 to provide a GUI in which a user can interact with the 3D model in real time, thereby providing a comprehensive view of the blood vessels. Processor 110 may execute instructions 120 to provide various interactions, including rotation, zooming in / out, panning, through-motion animation, creating cross-sectional views, showing / hiding certain detected features, performing area or distance measurements, and marking / labeling locations on the 3D volume.

[0043] Figure 2 , Figure 3 and Figure 4Processes 200, 300, and 400 according to some embodiments of the present disclosure are illustrated. Processes 200, 400, and 300 can be implemented by a vascular visualization system 100 or other computing devices as outlined herein to provide 3D visualization of blood vessels from a series of intravascular images (e.g., IVUS images, etc.). Process 200 can be used to align image frames from a series of vascular image frames (e.g., a single IVUS run, etc.), while process 400 can be used to generate a visualization of blood vessels from the aligned image frames. Process 300 can be used to generate a mask including indications of key features represented in each IVUS image frame.

[0044] Process 200 can begin at block 202: “Receive, at the computing device, multiple image frames associated with the patient’s blood vessels from the intravascular imaging device, the multiple image frames including multidimensional and multivariable images,” whereby the computing device 104 of the vascular visualization system 100 receives IVUS image frame 122 from the intravascular imager 102, wherein IVUS image frame 122 is a multidimensional and multivariable image of the blood vessels. For example, processor 110 can execute instruction 120 to receive data including the indication of IVUS image frame 122 from the intravascular imager 102 via network interface 118.

[0045] Continuing to box 204, “Generating a mask by a computing device, the mask including indications of key features of blood vessels,” from the plurality of image frames received in box 202, a mask including indications of key features of blood vessels can be generated for each of the plurality of image frames. For example, processor 110 can execute instruction 120 to generate IVUS image frame mask 126 from IVUS image frame 122. As described above, IVUS image frame mask 126 may include indications of vessel boundaries, lumen boundaries, plaque morphology, etc. Examples of frames in IVUS image frame 122 and their associated masks of IVUS image frame mask 126 (which can be generated as outlined herein) are shown in [the document]. Figure 5A and Figure 5B As given in the text, more details will follow below.

[0046] Continuing to box 206, “Aligning Multiple Image Frames Based on a Mask by a Computing Device,” multiple image frames received in box 202 can be aligned based on the mask generated in box 204. For example, processor 110 can execute instruction 120 to align (e.g., self-register, etc.) IVUS image frame 122 based on IVUS image frame mask 126. As a specific example, processor 110 can execute instruction 120 to determine frame alignment parameter 128 from IVUS image frame 122 and IVUS image frame mask 126, and generate aligned IVUS image frame 130 based on IVUS image frame 122 and frame alignment parameter 128. Examples of unaligned IVUS frames (e.g., IVUS image frame 122) and aligned IVUS frames (e.g., aligned IVUS image frame 130) (which can be generated as outlined herein) are shown in... Figure 6A and Figure 6B As given in the text, more details will follow below.

[0047] As mentioned above, Figure 3 A process 300 is illustrated, which can be used to generate a mask including indications of key features represented in each IVUS image frame. In some embodiments, process 300 can be implemented at block 204 of process 200. Process 300 can begin at block 302. In block 302, “Detection of luminal boundaries and / or vessel boundaries from multiple image frames by a computing device,” luminal boundaries and vessel boundaries can be detected from multiple image frames received at block 202. For example, processor 110 can execute instruction 120 to detect the boundaries of vessels and lumina. In a specific example, processor 110 can execute instruction 120 to segment image frames and detect boundaries based on the segmented image frames.

[0048] Continuing to box 304, “Inferring one or more key features of a blood vessel by a computing device using a machine learning model based at least in part on multiple image frames,” key features of a blood vessel can be inferred through a machine learning (ML) model. For example, processor 110 can execute instruction 120 to infer plaque load from IVUS image frame 122. The inferred plaque load indication can be stored as key feature 124.

[0049] Continuing to box 306, "Generating a vascular mask for each of a plurality of image frames by a computing device, the vascular mask including indications of key features, wherein the vascular mask can be used for motion compensation of the plurality of image frames," a mask for each image frame can be generated, the mask containing only indications of key features of the IVUS image frame. For example, processor 110 can execute instruction 120 to generate IVUS image frame mask 126 from IVUS image frame 122 and key features 124.

[0050] As mentioned above, Figure 4A process 400 is illustrated, which can be used to generate visualizations of blood vessels from aligned IVUS images. Process 400 can begin with process 200. From process 200, process 400 can continue to block 402. In block 402, “Resampling Multiple Image Frames by Computational Device Based on Frame Alignment Parameters,” multiple image frames (e.g., those received in block 202) can be resampled based on the frame alignment parameters generated in block 206. For example, processor 110 can execute instruction 120 to generate aligned IVUS image frame 130 by resampling IVUS image frame 122 using frame alignment parameters 128.

[0051] Continuing to box 404, "Generation of vessel volume from multiple resampled image frames by a computing device," the volume of a vessel can be generated based on resampled and aligned image frames. For example, processor 110 can execute instruction 120 to generate vessel volume 132 from aligned IVUS image frames 130. As a specific example, processor 110 can execute instruction 120 to generate vessel volume 132 by stacking aligned IVUS image frames 130.

[0052] Continuing to box 406, “Determining the color of each voxel of a blood vessel volume by a computing device based on key features of the blood vessel,” the color of each voxel (e.g., pixel) of a blood vessel volume can be determined based on key features of the blood vessel. For example, processor 110 can execute instruction 120 to determine the voxel color 134 of each pixel of blood vessel volume 132 according to key features 124. Typically, each type of key feature 124 can be assigned a specific color, and each pixel of blood vessel volume 132 can be assigned the color of the key feature type represented by that pixel.

[0053] Continuing to box 408, "Rendering a 3D visualization of blood vessel volume using the determined voxel color," a 3D visualization of blood vessel volume can be generated based on the blood vessel volume and the determined color. For example, processor 110 can execute instruction 120 to render a 3D visualization of blood vessel volume 132 using voxel color 134.

[0054] Proceeding to box 410, “Display the rendered 3D visualization on the display,” the rendered 3D visualization can be displayed on the display. For example, processor 110 can execute instruction 120 to generate a graphical information element (which includes an indication of the rendered 3D visualization) and display the graphical information element on a display (e.g., display 116, etc.).

[0055] Figure 5A and Figure 5BExamples of frames in a series of IVUS images and their corresponding masks are depicted. As described above, this disclosure provides for generating masks for each frame in a series of IVUS image frames and aligning IVUS image frames based on these masks. [Go to...] Figure 5A The image depicts IVUS image frame 500a. As described above, the vascular visualization system 100 can execute instruction 120 at block 202 to receive IVUS image frame 500a (or information elements and / or data structures including indications of IVUS image frame 500a). More specifically, it can be understood that several frames (such as IVUS image frame 500a) can be received as the probe retracts in the blood vessel and captures ultrasound images.

[0056] Figure 5B An image frame mask 500b associated with IVUS image frame 500a is depicted, showing only key features such as boundaries 502 (e.g., vessel boundaries and lumen boundaries) and plaques 504. In some embodiments (e.g., as described above), the image frame mask 500b can be generated based on a first detection of boundaries 502 and a second detection of plaques and / or other key features. In other embodiments, the image frame mask 500b can be inferred from IVUS image frame 500a using a machine learning model (e.g., machine learning model 138, etc.).

[0057] Figure 6A Various views of IVUS image frame 122 are shown, including an on-axis view 602a, a longitudinal view 604a, and a through-view 606a. As can be seen from the figures, views 602a, 604a, and 606a highlight the motion between frames within IVUS image frame 122. As mentioned above, this motion could be caused by patient movement, heartbeat, blood flow, catheter movement, etc. However, to provide a more realistic view of the blood vessels, this disclosure provides a method for aligning frames as described above.

[0058] Figure 6B Various views of the aligned IVUS image frame 130 are shown, including an aligned on-axis view 602b, an aligned portrait view 604b, and an aligned walk-through view 606b. It can be seen from this figure that, with... Figure 6A Compared to the views shown, views 602b, 604b, and 606b depict a smoother transition between frames. Therefore, a more realistic vessel volume 132 can be generated from the aligned IVUS image frames 130 and presented to the user as envisioned herein.

[0059] As previously described, in some embodiments, machine learning (ML) models can be used to infer key features and / or masks. For example, the processor 110 of computing device 104 can execute instructions 120 to infer key features 124 from IVUS image frame 122 using machine learning model 138. As another example, the processor 110 of computing device 104 can execute instructions 120 to infer IVUS image frame mask 126 from IVUS image frame 122 (or IVUS image frame 122 and key features 124) using machine learning model 138. In such examples, the ML model (e.g., machine learning model 138) can be stored in memory 112 of computing device 104. However, it should be understood that the ML model needs to be trained before deployment. Figure 7A An ML training environment 700a is shown, which can be used to train an ML model that can later be used to generate (or infer) key features 124 as described herein. The ML training environment 700a may include an ML system 702, such as a computational device that applies an ML algorithm to learn relationships. In this example, the ML algorithm can learn a relationship between a set of inputs (e.g., IVUS image frame 122) and outputs (e.g., key features 124).

[0060] The ML system 702 can utilize experimental data 708 collected in several prior processes. Experimental data 708 may include IVUS image frames 122 from multiple patients, or more precisely, multiple IVUS runs through different vessels. Experimental data 708 may be located in the same location as the ML system 702 (e.g., stored in the memory 710 of the ML system 702), may be located remotely from the ML system 702 and accessed via a network interface 704, or may be a combination of local and remote data.

[0061] Experimental data 708 can be used to form training data 712. As described above, ML system 702 may include memory 710, which may include hard disk drive, solid-state memory, and / or random access memory. Memory 710 may store training data 712. Typically, training data 712 may include information elements or data structures that include multiple patient IVUS image frames 122 and indications of associated desired key features. In some embodiments, experimental data 708 includes only patient IVUS image frames 122, and ML system 702 is configured (e.g., having a processor and instructions executable by the processor) to generate and / or receive desired key features 724 for each patient IVUS image frame 122 represented in experimental data 708.

[0062] Training data 712 can be used to train ML model 714. Depending on the application, different types of models can be used to form the basis of ML model 714. For example, in this example, an artificial neural network (ANN) might be particularly well-suited for learning the association between IVUS image frames (e.g., IVUS image frame 122) and key features (e.g., key feature 124). Convolutional neural networks might also be particularly well-suited for this task. Any suitable training algorithm 716 can be used to train ML model 714. Nevertheless, Figure 7A The examples depicted may be particularly well-suited for supervised training or reinforcement learning training algorithms. For supervised training, the ML system 702 may apply IVUS image frame 122 as model input 718 (expecting that key features 724 can be mapped to this input) to learn the association between IVUS image frame 122 and key features 124. In a reinforcement learning scenario, the training algorithm 716 may attempt to maximize some or all (or a weighted combination) of the mapping from model input 718 to key features 124 to produce an ML model 714 with minimum error. In some embodiments, training data 712 may be divided into “training” data and “test” data, where a subset of training data 712 can be used to adjust the ML model 714 (e.g., the model's internal weights, etc.), while another non-overlapping subset of training data 712 can be used to measure the accuracy of the ML model 714 inferring (or summarizing) key features 124 from “unseen” training data 712 (e.g., training data 712 not used to train the ML model 714).

[0063] The ML model 714 can be applied using processor circuitry 706, which may include suitable hardware processing resources for the logic circuitry and structure in the operating memory 710. The development of the training algorithm 716 and / or the trained ML model 714 may depend at least in part on hyperparameters 720. In an exemplary embodiment, model hyperparameters 720 may be automatically selected based on hyperparameter optimization logic 722, which may include any known hyperparameter optimization techniques suitable for the selected ML model 714 and the training algorithm 716 to be used. In an alternative embodiment, the ML model 714 may be retrained over time to adapt to new knowledge and / or updated experimental data 708.

[0064] Once the ML model 714 is trained, it can be applied (e.g., by processor circuitry 706, processor 110, etc.) to new input data (e.g., IVUS image frames 122 captured during pre-PCI intervention, post-PCI intervention, etc.). This input to the ML model 714 can be formatted according to a predefined model input 718, which mirrors the way the training data 712 is provided to the ML model 714. The ML model 714 can generate key features 124, which may include, for example, indications of luminal and vascular boundaries, plaque load, etc., represented in the IVUS image frames 122 provided as input to the ML model 714.

[0065] The above description relates to a specific type of ML system 702 that applies supervised learning techniques given available training data with input / outcome pairs. However, the invention is not limited to use with a specific ML paradigm, and other types of ML techniques can be used. For example, in some embodiments, the ML system 702 may apply, for example, evolutionary algorithms or other types of ML algorithms and models to generate key features 124 from IVUS image frames 122.

[0066] In some examples, ML models can be used to infer masks from IVUS image frames. Figure 7B An ML training environment 700b is shown, which is an example of an ML training environment 700a. The ML training environment 700a is configured to train an ML model 714 to infer an IVUS image frame mask 126 from an IVUS image frame 122. Therefore, training data 712 may include an IVUS image frame 122 and a desired image frame mask 726, and the ML model 714 may be "trained" as described above to infer the IVUS image frame mask 126 from the IVUS image frame 122.

[0067] In some examples, a mask can be inferred from IVUS image frames and key features using an ML model. In such examples, the key features can be generated from the ML model or from another algorithm (e.g., a segmentation algorithm). Figure 7C An ML training environment 700c is shown, which is an example of an ML training environment 700a. The ML training environment 700a is configured to train an ML model 714 to infer an IVUS image frame mask 126 from IVUS image frame 122 and key features 124. Therefore, training data 712 may include IVUS image frame 122 and key features 124, as well as a desired image frame mask 726, and the ML model 714 may be "trained" as described above to infer the IVUS image frame mask 126 from IVUS image frame 122 and key features 124.

[0068] Figure 8A computer-readable storage medium 800 is illustrated. The computer-readable storage medium 800 may include any non-transitory computer-readable or machine-readable storage medium, such as optical, magnetic, or semiconductor storage media. In various embodiments, the computer-readable storage medium 800 may include an article of manufacture. In some embodiments, the computer-readable storage medium 800 may store computer-executable instructions 802, which circuitry (e.g., processor 110, etc.) can execute. For example, the computer-executable instructions 802 may include instructions for implementing the operations described with respect to process 200, which may be specifically programmed to cause the vascular visualization system 100 to perform reference... Figure 2 Process 200 Figure 3 Process 300 or Figure 4 The operation described in process 400. As another example, computer-executable instructions 802 may include instructions 120, ML model 714, and / or training algorithm 716. Examples of computer-readable storage medium 800 or machine-readable storage medium may include any tangible medium capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. Examples of computer-executable instructions 802 may include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, etc.

[0069] Figure 9A , Figure 9B and Figure 9C An exemplary endovascular treatment system 900 is shown and described herein. Figure 9A It is a component-level view, and Figure 9B and Figure 9C They are Figure 9A Side and perspective views of a portion of the endovascular treatment system 900. The endovascular treatment system 900 employs an IVUS imaging system and can achieve [the following is unclear and likely incomplete: "... "] Figure 1 This is part of a vascular visualization system 100. The endovascular treatment system 900 includes a catheter 902 and a control subsystem 904. The control subsystem 904 includes a computing device 104, a drive unit 906, and a pulse generator 908. The catheter 902 and the control subsystem 904 are operatively coupled, or more specifically, the catheter 902 is electrically and / or mechanically coupled to the computing device 104, the drive unit 906, and the pulse generator 908, such that signals (e.g., control, measurement, image data, etc.) can communicate between the catheter 902 and the control subsystem 904.

[0070] Note that the computing device 104 includes a display 116. However, in some applications, the display 116 may be provided as a separate unit from the computing device 104, for example, in a different housing, etc. In some instances, a pulse generator 908 generates electrical pulses that can be input to one or more transducers 930 disposed in a conduit 902.

[0071] In some instances, mechanical energy from the drive unit 906 can be used to drive the imaging core 924 disposed in the catheter 902. In some instances, electrical signals transmitted from one or more transducers 930 can be input to the processor 110 of the computing device 104 for processing as outlined herein. For example, for generating vessel volume 132 and graphic information elements 136. In some instances, the processed electrical signals from one or more transducers 930 can also be displayed as one or more images on the display 116.

[0072] In some instances, processor 110 may also be used to control the functions of one or more other components of control subsystem 904. For example, processor 110 may be used to control at least one of the following: the frequency or duration of electrical pulses transmitted from pulse generator 908, the rotational rate of driving unit 906 relative to imaging core 924, the retraction speed or length of driving unit 906 relative to imaging core 924, or one or more attributes of one or more images formed on display 116, such as blood vessel volume 132 and graphic information element 136.

[0073] Figure 9B yes Figure 9A A side view of an embodiment of a catheter 902 in an endovascular therapy system 900. The catheter 902 includes an elongated member 910 and a connector 912. The elongated member 910 includes a proximal end 914 and a distal end 916. Figure 9B In this embodiment, the proximal end 914 of the elongated member 910 is coupled to the catheter connector 912, and the distal end 916 of the elongated member 910 is configured and arranged for percutaneous insertion into a patient. Optionally, the catheter 902 may define at least one irrigation port, such as irrigation port 918. Irrigation port 918 may be defined in the connector 912. The connector 912 may be configured and arranged to be coupled to the control subsystem 904 of the endovascular treatment system 900. In some instances, the elongated member 910 and the connector 912 are formed integrally. In other instances, the elongated member 910 and the catheter connector 912 are formed separately and subsequently assembled together.

[0074] Figure 9CThis is a perspective view of one embodiment of the distal end 916 of the elongated member 910 of catheter 902. The elongated member 910 includes a sheath 920 having a longitudinal axis (e.g., a central longitudinal axis extending axially through the center of sheath 920 and / or catheter 902) and a lumen 922. An imaging core 924 is disposed within the lumen 922. The imaging core 924 includes an imaging device 926 coupled to the distal end of a drive shaft 928, which is rotatable manually or using a computer-controlled drive mechanism. One or more transducers 930 may be mounted to the imaging device 926 and used for transmitting and receiving acoustic signals. The sheath 920 may be formed of any flexible, biocompatible material suitable for insertion into a patient. Examples of suitable materials include, for example, polyethylene, polyurethane, plastics, spiral-cut stainless steel, nitinol, etc., or combinations thereof.

[0075] In some instances, such as those shown in these figures, an array of transducers 930 is mounted on the imaging device 926. Alternatively, a single transducer may be used. Any suitable number of transducers 930 may be used. For example, there may be 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 15, 16, 20, 25, 50, 100, 500, 1000 or more transducers. It is understood that other numbers of transducers may also be used. When multiple transducers 930 are used, the transducers 930 can be configured in any suitable arrangement, including, for example, a ring arrangement, a rectangular arrangement, etc.

[0076] One or more transducers 930 may be formed of a material capable of converting an applied electrical pulse into pressure deformation on the surface of one or more transducers 930, and vice versa. Examples of suitable materials include piezoelectric ceramic materials, piezoelectric composite materials, piezoelectric plastics, barium titanate, lead zirconate titanate, lead metaniobate, polyvinylidene fluoride, etc. Other transducer technologies include composite materials, single-crystal composite materials, and semiconductor devices (e.g., capacitive micromechanical ultrasonic transducers (“cMUT”), piezoelectric micromechanical ultrasonic transducers (“pMUT”), etc.).

[0077] Pressure deformation on the surface of one or more transducers 930 generates acoustic pulses whose frequency is based on the resonant frequency of one or more transducers 930. The resonant frequency of one or more transducers 930 may be influenced by the size, shape, and material used to form one or more transducers 930. One or more transducers 930 can be formed into any suitable shape to be positioned within the conduit 902 and to propagate acoustic pulses of the desired frequency in one or more selected directions. For example, the transducers can be disc-shaped, block-shaped, rectangular, elliptical, etc. One or more transducers can be formed into the desired shape by any process, including, for example, cutting, dicing and filling, machining, micromachining, etc.

[0078] As an example, each of one or more transducers 930 may include a layer of piezoelectric material sandwiched between a matching layer and a conductive backing material formed of an acoustic-absorbing material (e.g., an epoxy resin substrate with tungsten particles). During operation, the piezoelectric layer may be electro-excited to induce the emission of acoustic pulses.

[0079] One or more transducers 930 can be used to form radial cross-sectional images of the surrounding space. Thus, for example, when one or more transducers 930 are disposed in catheter 902 and inserted into a patient's blood vessel, one or more transducers 930 can be used to form images of the vessel wall and the tissue surrounding the vessel.

[0080] The imaging core 924 rotates about the longitudinal axis of the conduit 902. As the imaging core 924 rotates, one or more transducers 930 emit acoustic signals in different radial directions (e.g., along different radial scan lines). For example, one or more transducers 930 may emit acoustic signals in regular (or irregular) increments, such as 256 radial scan lines per revolution. It is understood that other numbers of radial scan lines may also be emitted per revolution.

[0081] When a emitted acoustic pulse with sufficient energy encounters one or more media boundaries, such as one or more tissue boundaries, a portion of the emitted acoustic pulse is reflected back to the transmitting transducer as an echo pulse. Each echo pulse that reaches the transducer with sufficient energy and is detected is converted into an electrical signal in the receiving transducer. One or more converted electrical signals are transmitted to the processor 110 of the computing device 104, where they are processed to form an IVUS image frame 122, and subsequently generate a vessel volume 132 and graphic information elements 136 for display on the display 116. In some instances, the rotation of the imaging core 924 is driven by a drive unit 906, which may be located in the control subsystem 904. In alternative embodiments, one or more transducers 930 are fixed and do not rotate. In this case, the drive shaft 928 may instead cause a mirror to rotate, reflecting the acoustic signal to and from the fixed one or more transducers 930.

[0082] When one or more transducers 930 rotate about the longitudinal axis of catheter 902 to emit acoustic pulses, multiple images can be formed that collectively form a radial cross-sectional image (e.g., a tomographic image) of a portion of a region surrounding one or more transducers 930, such as the vessel wall of interest and the tissue surrounding the vessel. The radial cross-sectional image may form the basis of IVUS image frame 122 and may optionally be displayed on display 116. At least one of the imaging cores 924 can be rotated manually or using a computer-controlled mechanism.

[0083] The imaging core 924 can also move longitudinally along the blood vessel into which the catheter 902 is inserted, allowing multiple cross-sectional images to be formed along the longitudinal length of the vessel. During the imaging process, one or more transducers 930 can retract (e.g., pull back) along the longitudinal length of the catheter 902. The catheter 902 may include at least one telescopic portion that can retract during the retraction of one or more transducers 930. In some instances, a drive unit 906 drives the retraction of the imaging core 924 within the catheter 902. The drive unit 906 allows the imaging core to retract by any suitable distance, for example, including at least 5 cm, 10 cm, 15 cm, 20 cm, 25 cm, or longer. The entire catheter 902 can retract during the imaging process, regardless of whether the imaging core 924 moves longitudinally independently of the catheter 902.

[0084] Alternatively, a stepper motor can be used to retract the imaging core 924. The stepper motor can retract the imaging core 924 a short distance, then stop for a sufficient time to allow one or more transducers 930 to capture an image or a series of images, and then retract the imaging core 924 another short distance and capture another image or another series of images, and so on.

[0085] The image quality generated from one or more transducers 930 at different depths may be affected by one or more factors, including bandwidth, transducer focus, beam pattern, and the frequency of the acoustic pulse. The frequency of the acoustic pulse output from one or more transducers 930 may also affect the penetration depth of the acoustic pulse output from one or more transducers 930. Generally, as the frequency of the acoustic pulse decreases, the penetration depth of the acoustic pulse within the patient tissue increases. In some instances, the endovascular treatment system 900 operates in a frequency range of 5 MHz to 900 MHz.

[0086] One or more conductors 932 can electrically connect the transducer 930 to the control subsystem 904. In this case, one or more conductors 932 can extend along the longitudinal length of the rotatable drive shaft 928.

[0087] A catheter 902 having one or more transducers 930 mounted to the distal end 916 of an imaging core 924 can be percutaneously inserted into the patient at a site of the selected portion to be imaged, away from the selected area (e.g., a blood vessel), via an accessible blood vessel (e.g., the femoral artery, femoral vein, or jugular vein). The catheter 902 can then be advanced through the patient's blood vessels to the selected imaging site, such as a portion of the selected blood vessel.

[0088] An image or image frame (“frame”) can be generated each time one or more acoustic signals are output to surrounding tissue and one or more corresponding echo signals are received by the imaging device 926 and transmitted to the processor 110 of the computing device 104. Alternatively, the image or image frame can be a composite of scan lines generated by a complete or partial rotation of the imaging core or device. Multiple frames (e.g., sequences) can be acquired over time during any type of movement of the imaging device 926. For example, frames can be acquired during rotation and retraction of the imaging device 926 along the target imaging position. It is understood that frames can be acquired whether the imaging device 926 rotates or does not rotate and whether the imaging device 926 retracts or does not retract. Furthermore, it is understood that other types of movement procedures can be used to acquire frames in addition to at least one of rotation or retraction of the imaging device 926, or as an alternative.

[0089] In some instances, the retraction can be performed at a constant rate, providing a tool for potential applications that can calculate longitudinal vessel / plaque measurements. In some instances, the imaging device 926 retracts at a constant rate of about 0.3–0.9 mm / s or about 0.5–0.8 mm / s. In some instances, the imaging device 926 retracts at a constant rate of at least 0.3 mm / s. In some instances, the imaging device 926 retracts at a constant rate of at least 0.4 mm / s. In some instances, the imaging device 926 retracts at a constant rate of at least 0.5 mm / s. In some instances, the imaging device 926 retracts at a constant rate of at least 0.6 mm / s. In some instances, the imaging device 926 retracts at a constant rate of at least 0.7 mm / s. In some instances, the imaging device 926 retracts at a constant rate of at least 0.8 mm / s.

[0090] In some instances, one or more acoustic signals are output to surrounding tissue at constant time intervals. In some instances, one or more corresponding echo signals are received by imaging device 926 and transmitted to processor 110 of computing device 104 at constant time intervals. In some instances, the resulting frames are generated at constant time intervals.

[0091] Figure 10 A schematic representation of machine 1000 is shown, which is in the form of a computer system within which a set of instructions can be executed to cause the machine to perform any or more methods discussed herein. More specifically, Figure 10A schematic representation of machine 1000 is shown as an example of a computer system, within which instructions 1008 (e.g., software, programs, applications, applets, or other executable code) can be executed to cause machine 1000 to perform any or more methods discussed herein. For example, instruction 1008 can cause machine 1000 to execute instruction 120, Figure 2 The process 200, training algorithm 716, etc. More generally, instruction 1008 enables machine 1000 to generate a 3D model or vascular physiological condition from a single angiographic image, a series of IVUS images, and vascular pressure measurements, as described herein.

[0092] Instruction 1008 transforms a general, unprogrammed machine 1000 into a specific machine 1000 programmed to perform the described and illustrated functions in a specific manner. In alternative embodiments, machine 1000 operates as a standalone device or can be coupled (e.g., networked) to other machines. In network deployments, machine 1000 can operate as a server machine or client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 1000 can be, but is not limited to, server computers, client computers, personal computers (PCs), tablet computers, laptop computers, netbooks, set-top boxes (STBs), personal digital assistants (PDAs), entertainment media systems, cellular phones, smartphones, mobile devices, wearable devices (e.g., smartwatches), smart home devices (e.g., smart appliances), other smart devices, network devices, network routers, network switches, network bridges, or any machine capable of executing instruction 1008, which sequentially or otherwise specifies the actions to be taken by machine 1000. Furthermore, although only a single machine 1000 is shown, the term "machine" should also be considered as including a collection of machines 200, which individually or in combination execute instructions 1008 to perform any one or more methods discussed herein.

[0093] Machine 1000 may include processor 1002, memory 1004, and I / O components 1042, which may be configured to communicate with each other, such as via bus 1044. In one example embodiment, processor 1002 (e.g., a central processing unit (CPU), a reduced instruction set computing (RISC) processor, a complex instruction set computing (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processor 1006 and processor 1010, which can execute instructions 1008. The term "processor" is intended to include multi-core processors, which may include two or more independent processors (sometimes referred to as "cores") capable of executing instructions simultaneously. Although Figure 10 Multiple processors 1002 are shown, but machine 1000 may include a single processor with a single core, a single processor with multiple cores (e.g., a multi-core processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0094] Memory 1004 may include main memory 1012, static memory 1014, and memory cells 1016, which are accessible by processor 1002 via bus 1044. Main memory 1004, static memory 1014, and memory cells 1016 store instructions 1008 embodying any one or more methods or functions described herein. Instructions 1008 may also reside wholly or partially in main memory 1012, static memory 1014, machine-readable medium 1018 within memory cells 1016, at least one processor 1002 (e.g., within the processor's cache memory), or any suitable combination thereof during execution by machine 1000.

[0095] I / O component 1042 may include various components for receiving input, providing output, generating output, transmitting information, exchanging information, capturing measurement values, etc. The specific I / O component 1042 included in a particular machine will depend on the type of machine. For example, a portable machine such as a mobile phone is likely to include a touch input device or other such input mechanism, while a headless server machine is likely not to include such a touch input device. It should be understood that I / O component 1042 may include... Figure 10Many other components are not shown. The grouping of I / O components 1042 according to function is merely for the purpose of simplifying the following discussion and is by no means limiting. In various example embodiments, I / O components 1042 may include output components 1028 and input components 1030. Output components 1028 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tube (CRTs), acoustic components (e.g., speakers), haptic components (e.g., vibration motors, resistance mechanisms), other signal generators, etc. Input components 1030 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photoelectric keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other indicating tools), haptic input components (e.g., physical buttons, touchscreens providing position and / or force for touch or touch gestures, or other haptic input components), audio input components (e.g., microphones), etc.

[0096] In another example embodiment, I / O component 1042 may include biometric component 1032, motion component 1034, environmental component 1036, or position component 1038, as well as various other components. For example, biometric component 1032 may include components for detecting expressions (e.g., gestures, facial expressions, vocal expressions, body posture, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or brain waves), and identifying a person (e.g., voiceprint recognition, retinal recognition, facial recognition, fingerprint recognition, or EEG-based recognition). Motion component 1034 may include accelerometer components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), etc. Environmental component 1036 may include, for example, a lighting sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers for detecting ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones for detecting background noise), a proximity sensor component (e.g., an infrared sensor for detecting nearby objects), a gas sensor (e.g., a gas detection sensor for detecting hazardous gas concentrations to ensure safety or to measure atmospheric pollutants), or other components that can provide indications, measurements, or signals corresponding to the surrounding physical environment. Orientation component 1038 may include a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer for detecting air pressure from which altitude can be derived), an orientation sensor component (e.g., a magnetometer), etc.

[0097] Various technologies can be used to implement communication. I / O component 1042 may include communication component 1040, operable to connect machine 1000 to network 1020 or device 1022 via connectors 1024 and 1026, respectively. For example, communication component 1040 may include a network interface component or other suitable means for interfacing with network 1020. In further examples, communication component 1040 may include a wired communication component, a wireless communication component, a cellular communication component, a near field communication (NFC) component, or Bluetooth. ® Components (e.g., Bluetooth) ® Low power consumption, Wi-Fi ® Components and other communication components for providing communication in other ways. Device 1022 can be any of another machine or various peripheral devices (e.g., a peripheral device connected via USB).

[0098] Furthermore, the communication component 1040 can detect identifiers or include components operable to detect identifiers. For example, the communication component 1040 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes (such as Universal Product Code (UPC) barcodes), multi-dimensional barcodes (such as Quick Response (QR) codes, Aztec codes, DataMatrix, Dataglyph, MaxiCod, PDF417, UltraCode, UCCRSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying tag audio signals). Additionally, various information can be exported via the communication component 1040, such as location obtained through Internet Protocol (IP) geolocation, or information obtained via Wi-Fi. ® Locations obtained through signal triangulation or by detecting NFC beacon signals (which can indicate a specific location), etc.

[0099] Various memories (i.e., memory 1004, main memory 1012, static memory 1014, and / or the memory of processor 1002) and / or storage units 1016 may store one or more sets of instructions and data structures (e.g., software) embodying or utilizing any one or more methods or functions described herein. These instructions (e.g., instruction 1008), when executed by processor 1002, cause various operations to implement the disclosed embodiments.

[0100] As used herein, the terms “machine storage medium,” “device storage medium,” and “computer storage medium” have the same meaning and may be used interchangeably in this disclosure. These terms refer to one or more storage devices and / or media (e.g., centralized or distributed databases and / or associated caches and servers) that store executable instructions and / or data. These terms should therefore be considered to include, but are not limited to, solid-state memory and optical and magnetic media, including memory internal or external to a processor. Specific examples of machine storage media, computer storage media, and / or device storage media include non-volatile memory, such as semiconductor memory devices, such as erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices; magnetic disks, such as internal hard disks and removable disks; magneto-optical disks; and CD-ROMs and DVD-ROMs. The terms “machine storage medium,” “computer storage medium,” and “device storage medium” specifically exclude carrier waves, modulated data signals, and other media, at least some of which are included in the term “signal medium” discussed below.

[0101] In various example embodiments, one or more portions of network 1020 may be an ad hoc network, intranet, extranet, VPN, LAN, WLAN, WAN, WWAN, MAN, the Internet, a portion of the Internet, a portion of the PSTN, a common legacy telephone service (POTS) network, a cellular telephone network, a wireless network, a Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, network 1020 or a portion of network 1020 may include a wireless or cellular network, and connector 1024 may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or another type of cellular or wireless connection. In this example, connector 1024 can implement any of a variety of data transmission technologies, such as single-carrier radio transmission technology (1xRTT), evolved data optimization (EVDO) technology, general packet radio service (GPRS) technology, enhanced data rate GSM evolution (EDGE) technology, including 3G third generation partnership program (3GPP), fourth generation wireless (4G) networks, universal mobile telecommunications system (UMTS), high-speed packet access (HSPA), global microwave access interoperability (WiMAX), long-term evolution (LTE) standard, other standards defined by various standards-setting organizations, other long-distance protocols, or other data transmission technologies.

[0102] Instruction 1008 may be transmitted or received via network 1020 using a transmission medium, through a network interface device (e.g., a network interface component included in communication component 1040), and utilizing any of a variety of known transmission protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, instruction 1008 may be transmitted or received by device 1022 via a connection 1026 (e.g., a peer-to-peer connection) using a transmission medium. The terms “transmission medium” and “signal medium” have the same meaning and may be used interchangeably in this disclosure. The terms “transmission medium” and “signal medium” should be understood to include any intangible medium capable of storing, encoding, or carrying instruction 1008 executed by machine 1000, and include digital or analog communication signals or other intangible media that facilitate such software communication. Therefore, the terms “transmission medium” and “signal medium” should be understood to include any form of modulated data signal, carrier wave, etc. The term “modulated data signal” refers to a signal whose one or more characteristics are set or altered in a manner that encodes information.

[0103] The terms used herein should have their common meaning in the relevant field, or the meaning indicated by the context in which they are used, unless otherwise specified.

[0104] References to “one embodiment” or “an embodiment” herein do not necessarily refer to the same embodiment, although they may refer to the same embodiment. Unless the context explicitly requires otherwise, throughout the specification and claims, the words “comprising,” “including,” etc., should be interpreted in an inclusive sense, not an exclusive or exhaustive sense; that is, in the sense of “including, but not limited to.” Use of singular or plural terms also includes both the plural and singular, unless explicitly limited to one or more. Furthermore, the words “here,” “above,” “below,” and similar terms, when used in this application, refer to the entire application and not any particular part of it. When the claim uses the word “or” in a list of two or more items, the word covers all of the following interpretations: any item in the list, all items in the list, and any combination of items in the list, unless explicitly limited to one or the other. Any term not explicitly defined herein has its conventional meaning as commonly understood by one of ordinary skill in the art.

Claims

1. A method for self-registering a series of IVUS image frames, comprising: Multiple images associated with a patient's blood vessels are received at a computing device from an intravascular imaging device; these multiple images include multidimensional and multivariable images. The computing device generates a mask for each of the plurality of images, the mask including indications of key features of the blood vessels; as well as The computing device aligns the multiple images based on the multiple masks.

2. The method according to claim 1, wherein, The key features include at least one of the following: vascular boundary, lumen boundary, plaque, or lesion.

3. The method according to claim 2, further comprising: The computing device detects the blood vessel boundary and the lumen boundary; The computing device uses a machine learning (ML) model to infer the plaque or the lesion; and The mask is generated, which includes the detected vessel boundaries, the detected lumen boundaries, and indications of the inferred plaques or lesions.

4. The method of claim 2, further comprising inferring the plurality of masks from the plurality of images by the computing device using a machine learning (ML) model.

5. The method according to any one of claims 1 to 4, wherein, Aligning the multiple images based on the multiple masks includes: Derive the frame alignment parameters for each of the plurality of images; and The computing device resamples the plurality of image frames based on the frame alignment parameters.

6. The method according to any one of claims 1 to 5, further comprising generating blood vessel volumes from the aligned plurality of images by the computing device.

7. The method according to claim 6, further comprising: The computing device determines the color for each voxel of the blood vessel volume based on the key features; as well as The computing device renders a three-dimensional (3D) visualization of the blood vessel volume using the determined colors.

8. The method according to claim 7, wherein, The key features include at least the vessel boundary, calcified plaque, non-calcified plaque, luminal boundary, and stent, and each of the key features is associated with a different color.

9. The method according to claim 8, wherein, The vessel boundary is associated with brown, the calcified plaque with gray, the non-calcified plaque with yellow, the lumen boundary with transparent, and the stent with white.

10. The method according to any one of claims 7 to 9, further comprising displaying a 3D visualization of the rendered blood vessel volume on a display.

11. The method according to any one of claims 7 to 10, wherein, The 3D visualization includes a longitudinal view of the blood vessel and an on-axis view of the blood vessel.

12. The method according to any one of claims 7 to 11, wherein, The 3D visualization includes a view of the blood vessels passing through the vessel.

13. A computer-readable storage device comprising instructions executable by a processor of a computing device coupled to an intravascular imaging apparatus and a fluorescence fluoroscopy apparatus, wherein, When executed, the instructions cause the computing device to implement the method according to any one of claims 1 to 12.

14. An apparatus comprising a processor arranged to be coupled to an intravascular imaging apparatus and a fluorescence fluoroscopy apparatus, the apparatus further comprising a memory containing instructions, the processor being arranged to execute the instructions to implement the method according to any one of claims 1 to 12.

15. The device of claim 14, comprising an intravascular ultrasound (IVUS) probe.

Images