Generation of vascular physiological structures through co-registration of angiography and IVUS.

Abstract

This disclosure provides an apparatus and method for generating a three-dimensional (3D) model of the physiological structure of a blood vessel from other physical characteristics of the vessel, such as pressure, along with a single angiographic image and a series of intravascular images.

Description

Technical Field

[0001] The present disclosure relates to generating a virtual physiological structure of blood vessels. This application claims the benefit of U.S. Provisional Patent Application No. 63 / 456,335, filed Mar. 31, 2023, the disclosure of which is incorporated herein by reference.

Background Art

[0002] The currently accepted technique for assessing the severity of stenosis within a blood vessel, including ischemia that causes lesions, is intravascular imaging combined with angiography. Physicians can also evaluate stenosis using the fractional flow reserve (FFR). The FFR is calculated as the ratio of the distal pressure measurement (obtained distal to the stenosis) to the proximal pressure measurement (obtained proximal to the stenosis). The FFR provides an index of stenosis severity that enables determination of whether the occlusion is restricting blood flow within the blood vessel to the extent that treatment is required. The normal value of FFR in a healthy blood vessel is 1.00, but values less than about 0.80 are generally considered significant and require treatment. Common treatment options for stenosis include percutaneous coronary intervention (PCI or angioplasty), stent placement, or coronary artery bypass graft (CABG) surgery. As with all medical procedures, PCI, stent placement, and CABG procedures carry certain risks. Additional information regarding the risks and likelihood of success associated with treatment options is needed for surgeons to make more informed decisions regarding treatment options.

[0003] However, the location of stenosis within a blood vessel can be difficult to visualize in black-and-white angiography images and IVUS images. Therefore, improved devices, systems, and methods remain needed to assess the severity of intravascular occlusion and stenosis. In this regard, improved devices, systems, and methods remain needed to provide a visual representation of blood vessels that enables the assessment of vascular stenosis or lesions. Furthermore, there remains a need for improved devices, systems, and methods to objectively assess the risks associated with one or more available treatment options for blood vessels and the likelihood of success for those treatment options. [Overview of the project]

[0004] This disclosure provides a method for generating a physiological flow model of a patient's blood vessels based on the co-registration of a single angiographic image and a series of intravascular images, with one or more additional physiological measurement information optionally provided. In a particular example, this disclosure provides a method for generating a three-dimensional (3D) model representing at least a portion of a patient's heart based on a series of intravascular ultrasound (IVUS) images co-registered with a single angiographic image.

[0005] Therefore, this disclosure provides a system for generating 3D reconstructions of blood vessels to provide physicians with a virtual physiological structure of the blood vessels. In particular, this disclosure can be integrated with an intravascular ultrasound evaluation system, thus avoiding the use of guidewire-based evaluation systems solely for the purpose of evaluating vascular physiological structure. This improves physician efficiency and reduces the number of invasive procedures that patients undergo during vascular evaluation.

[0006] This disclosure offers several advantages over conventional methods for generating models of the physiological structure of blood vessels. For example, some conventional methods use multiple angiographic images to generate a physiological model of blood vessels. However, as mentioned above, this method requires multiple (e.g., two or more) angiographic projections, resulting in workflow overhead. Furthermore, boundary tracing on angiographic images often requires manual correction, further increasing workflow overhead. Another approach uses only a series of IVUS images. However, this method cannot adequately consider the vascular curvature and side branch sizes necessary for accurate FFR calculation and vascular physiological modeling.

[0007] This disclosure provides a method for generating a model of vascular physiological structure using a single angiographic image and a series of IVUS images. This method corrects for inaccuracies in conventional methods, such as shortening and inaccurate vascular dimensions, that occur when generating a physiological model using a single imaging modality. The method of this disclosure corrects for shortening, which frequently occurs when relying on a single angiographic view to model vessels, by performing accurate distance measurements using IVUS images. Furthermore, this disclosure provides more accurate lumen / vascular dimensions than those that can be provided by angiographic projection alone by utilizing IVUS cross-sectional area measurements. In addition, this disclosure provides mutual detection of the position and dimensions of side branches by utilizing co-registration information between a single angiographic image and a series of IVUS images, thereby providing a more accurate vascular physiological structure.

[0008] In some embodiments, the present disclosure may be implemented as a method for generating a 3D model of the physiological structure of a blood vessel. The method may include receiving angiographic images of a patient's blood vessels from a fluoroscopy device in a computing device, receiving a plurality of images associated with the patient's blood vessels from an intravascular imaging device in the computing device, the plurality of images including multidimensional and multivariate images, and generating a three-dimensional (3D) model of the physiological structure of the blood vessel from the angiographic images and the plurality of images.

[0009] In a further embodiment, the method may include generating graphic information elements, including the display of the 3D model, by the computing device, and displaying the graphic information elements on a display coupled to the computing device by the computing device.

[0010] In a further embodiment of the above method, generating a 3D model of the physiological structure of the blood vessels includes co-registering the angiographic images and the plurality of images. In a further embodiment, the method may include identifying a starting point for a pullback operation associated with the plurality of images of the vessel represented in the angiographic image, identifying an ending point for the pullback operation associated with the plurality of images of the vessel represented in the angiographic image, and identifying the centerline of the vessel between the starting point and the ending point.

[0011] In a further embodiment, the method may include identifying a plurality of collateral branches of the blood vessel on the angiographic image and within the plurality of images, and associating one of the plurality of collateral branches identified on the angiographic image with one of the plurality of collateral branches identified within the plurality of images.

[0012] In a further embodiment, the method may include mapping the frames of the plurality of images to positions along the centerline of the blood vessels on the angiographic image. In a further embodiment, the method may include generating evaluation information for the blood vessels.

[0013] In a further embodiment of the above method, the evaluation information includes the diameter of the blood vessel, the area of ​​the blood vessel, or the diameter and area of ​​the blood vessel. In a further embodiment of the above method, the evaluation information includes the diameter of the lumen, the area of ​​the lumen, or the diameter and area of ​​the lumen.

[0014] In a further embodiment, the method may include receiving an index of additional physiological features of the patient's blood vessels in the computing device, and generating a 3D model of the physiological structure of the blood vessels from the angiographic images, the plurality of images, and the additional physiological features of the blood vessels.

[0015] In a further embodiment of the above method, the additional physiological features of the blood vessel include pressure or flow rate. In a further embodiment, the method may include generating an inference of the 3D model of the physiological structure of the blood vessels from the machine learning (ML) model, based in part on applying the angiographic images and the plurality of images as input to the ML model.

[0016] In a further embodiment of the above method, the ML model is trained in part on a supervised learning training algorithm that uses the expected output of the ML model derived based on a computational fluid dynamics (CFD) model. The CFD model takes angiographic images and multiple images as input and generates a 3D vascular physiological model as output.

[0017] In some embodiments, the present disclosure may be implemented as an apparatus comprising a processor configured to be coupled to an intravascular imaging apparatus and a fluoroscopy apparatus. The apparatus further comprises a memory having a plurality of instructions, and the processor is configured to perform the methods of any embodiment described herein by executing the plurality of instructions.

[0018] In some embodiments, the present disclosure may be implemented as a computer-readable storage device. The computer-readable storage device comprises a plurality of instructions executable by a processor of an arithmetic unit coupled to an intravascular imaging device and a fluoroscopy device, and when the plurality of instructions are executed, the arithmetic unit performs a method of any embodiment described herein.

[0019] In some embodiments, the present disclosure may be implemented as an apparatus for angiography medical devices. The apparatus may include a processor configured to be coupled to an intravascular imaging device and a fluoroscopy device, and a memory device coupled to the processor. The memory device comprises a plurality of instructions, which, when executed by the processor, cause the apparatus to perform the following actions: receive angiographic images of a patient's blood vessels from the fluoroscopy device; receive a plurality of images associated with the patient's blood vessels from the intravascular imaging device, the plurality of images including multidimensional and multivariate images; and generate a three-dimensional (3D) model of the physiological structure of the blood vessels from the angiographic images and the plurality of images.

[0020] In a further embodiment of the above apparatus, the plurality of instructions, when executed by the processor, further cause the apparatus to generate graphic information elements, including a display of the 3D model, and to display the graphic information elements on a display coupled to the arithmetic unit.

[0021] In a further embodiment of the above apparatus, the plurality of instructions, when executed by the processor, further cause the apparatus to co-register the angiographic image and the plurality of images.

[0022] In a further embodiment of the apparatus described above, the plurality of instructions, when executed by the processor, further cause the apparatus to identify the starting point of a pullback operation associated with the plurality of images of the vessel represented in the angiographic image, identify the ending point of the pullback operation associated with the plurality of images of the vessel represented in the angiographic image, and identify the centerline of the vessel between the starting point and the ending point.

[0023] In a further embodiment of the above apparatus, the plurality of instructions, when executed by the processor, further cause the apparatus to identify a plurality of collateral branches of the blood vessel on the angiographic image and within the plurality of images, and to associate one of the plurality of collateral branches identified on the angiographic image with one of the plurality of collateral branches identified within the plurality of images.

[0024] In a further embodiment of the apparatus described above, the plurality of instructions, when executed by the processor, further cause the apparatus to map the plurality of image frames to positions along the centerlines of the blood vessels on the angiographic image.

[0025] In a further embodiment of the apparatus described above, the plurality of instructions, when executed by the processor, further cause the apparatus to generate evaluation information of the blood vessels. The evaluation information includes the diameter of the blood vessel, the area of ​​the blood vessel, or the diameter and area of ​​the blood vessel. The evaluation information also includes the diameter of the lumen, the area of ​​the lumen, or the diameter and area of ​​the lumen.

[0026] In a further embodiment of the apparatus described above, the plurality of instructions, when executed by the processor, cause the apparatus to receive indices of additional physiological features of the patient's blood vessels in the computing unit, and to generate the 3D model of the physiological structure of the blood vessels from the angiographic images, the plurality of images, and the additional physiological features of the blood vessels. The additional physiological features of the blood vessels include pressure or flow rate.

[0027] In a further embodiment of the above device, when the plurality of instructions are executed by the processor, the device is caused to further generate an inference of the 3D model of the physiological structure of the blood vessel from the ML model, based in part on applying the angiography image and the plurality of images as inputs to a machine learning (ML) model.

[0028] In a further embodiment of the above device, the ML model is trained based in part on a supervised learning training algorithm that uses an expected output of the ML model derived based on a computational fluid dynamics (CFD) model. The CFD model takes an angiography image and a plurality of images as inputs and generates a 3D vascular physiological model as an output.

[0029] In some embodiments, the present disclosure may be implemented as a computer-readable storage device. The storage device may include instructions executable by a processor of an arithmetic unit coupled to an intravascular imaging device and a fluoroscopy device. The plurality of instructions cause the arithmetic unit, when the plurality of instructions are executed, to receive an angiography image of a patient's blood vessel from the fluoroscopy device, receive a plurality of images associated with the patient's blood vessel from the intravascular imaging device, the plurality of images including multi-dimensional and multi-variable images, and generate a three-dimensional (3D) model of the physiological structure of the blood vessel from the angiography image and the plurality of images.

[0030] In a further embodiment of the above storage device, the plurality of instructions further cause the arithmetic unit, when executed by the processor, to generate a graphic information element including a display of the 3D model and display the graphic information element on a display coupled to the arithmetic unit.

[0031] In a further embodiment of the memory device described above, the plurality of instructions, when executed by the processor, cause the arithmetic unit to identify the starting point of a pullback operation associated with the plurality of images of the vessel represented in the angiographic image; identify the ending point of the pullback operation associated with the plurality of images of the vessel represented in the angiographic image; identify the centerline of the vessel between the starting point and the ending point; identify a plurality of side branches of the vessel on the angiographic image and within the plurality of images; associate one of the plurality of side branches identified on the angiographic image with one of the plurality of side branches identified within the plurality of images; and map the frames of the plurality of images to positions along the centerline of the vessel on the angiographic image.

[0032] In a further embodiment of the memory device described above, the plurality of instructions, when executed by the processor, further cause the arithmetic unit to generate evaluation information of the blood vessels. The evaluation information includes the diameter of the blood vessel, the area of ​​the blood vessel, or the diameter and area of ​​the blood vessel. The evaluation information also includes the diameter of the lumen, the area of ​​the lumen, or the diameter and area of ​​the lumen.

[0033] In a further embodiment of the memory device described above, the plurality of instructions, when executed by the processor, cause the arithmetic unit to receive in the arithmetic unit an index of additional physiological features of the patient's blood vessels, and to generate the 3D model of the physiological structure of the blood vessels from the angiographic images, the plurality of images, and the additional physiological features of the blood vessels. The additional physiological features of the blood vessels include pressure or flow rate.

[0034] In a further embodiment of the memory device described above, the plurality of instructions, when executed by the processor, further cause the arithmetic unit to generate an inference of the 3D model of the physiological structure of the blood vessels from the ML model, partly based on applying the angiographic image and the plurality of images as input to the ML model. The ML model is trained partly based on a supervised learning training algorithm using the expected output of the ML model derived based on a computational fluid dynamics (CFD) model. The CFD model takes the angiographic image and the plurality of images as input and generates a 3D vascular physiological model as output. [Brief explanation of the drawing]

[0035] [Figure 1] Figure 1 shows an intravascular treatment system according to at least one embodiment. [Figure 2A] Figure 2A shows another intravascular treatment system according to at least another embodiment. [Figure 2B] Figure 2B shows a portion of the endovascular treatment system shown in Figure 2A. [Figure 2C] Figure 2C shows a portion of the endovascular treatment system shown in Figure 2A. [Figure 3] Figure 3 shows a routine 300 for generating a three-dimensional (3D) model of blood vessels according to at least one embodiment. [Figure 4A] Figure 4A illustrates an image of an element or feature of the subject according to at least one embodiment. [Figure 4B] Figure 4B illustrates an image of an element or feature of the subject according to at least one embodiment. [Figure 4C] Figure 4C illustrates an image of an element or feature of the subject according to at least one embodiment. [Figure 4D] Figure 4D illustrates an image of an element or feature of the subject according to at least one embodiment. [Figure 4E] Figure 4E illustrates an image of an element or feature of the subject according to at least one embodiment. [Figure 4F] Figure 4F illustrates an image of an element or feature of the subject according to at least one embodiment. [Figure 4G] Figure 4G illustrates an image of an element or feature of the subject according to at least one embodiment. [Figure 4H] Figure 4H illustrates an image of an element or feature of the subject according to at least one embodiment. [Figure 5] Figure 5 shows an exemplary machine learning (ML) environment according to at least one embodiment. [Figure 6] Figure 6 shows a computer-readable storage medium according to at least one embodiment. [Figure 7] Figure 7 shows a schematic diagram of a machine in the form of a computer system in which a set of instructions can be executed to cause the machine to perform any one or more of the methods described herein. [Modes for carrying out the invention]

[0036] As described above, in exemplary embodiments, the system is configured to generate a 3D model of vascular physiological structure from a series of intravascular ultrasound (IVUS) images co-registered with a single angiographic image. In some embodiments, the system may be configured to further utilize additional vascular features (e.g., pressure measurements, flow measurements, etc.) to generate the model. As a specific example, this disclosure provides generating a 3D model of vascular physiological structure from a series of IVUS images co-registered with a single angiographic image and aortic pressure measurements (e.g., Pa, FFR, etc.). While this disclosure uses the aorta and coronary arteries as examples, the disclosed system and method may be implemented to generate 3D models of other types of vessels. To facilitate identification of elements or processes, the most significant one or more digits of the reference numeral refer to the figure number in which the element was first introduced.

[0037] Figure 1 shows a vascular physiological modeling system 100 according to one embodiment of the present disclosure. Generally, the vascular physiological modeling system 100 is a system for generating a virtual model of a blood vessel based on various images and features of the blood vessel. To this end, the vascular physiological modeling system 100 includes an intravascular imager 102, an angiography imager 104, and a computing unit 106, and optionally includes a pressure sensor 108. The intravascular imager 102 can be any of various intravascular imagers (e.g., IVUS, OCT, or OCE). In a particular example, the intravascular imager 102 may be an intravascular treatment system 200, which will be described below with reference to Figure 2A. Similarly, the angiography imager 104 can be any of various angiography imagers (e.g., a fluoroscopy device). The pressure sensor 108 can be any of various vascular pressure sensing devices (e.g., a pressure-sensing catheter). In some embodiments, the intravascular imager 102 and the pressure sensor 108 can be integrated within the same device.

[0038] The computing device 106 can be any of various computing devices. In some embodiments, the computing device 106 may be integrated into and / or implemented within the console of the intravascular imager 102. In some embodiments, the computing device 106 may be a workstation or server communicatively coupled to the intravascular imager 102. In yet other embodiments, the computing device 106 may be provided by a cloud-based computing device, such as computing means as a service system accessible over a network (e.g., the Internet, an intranet, a wide area network, etc.). The computing device 106 may include a processor 110, memory 112, input and / or output (I / O) devices 114, and a network interface 118.

[0039] The processor 110 may include, for example, a circuit or processor logic section such as one of various commercially available processors. In some examples, the processor 110 may include multiple processors, a multithreaded processor, a multicore processor (where multiple cores may coexist on the same die or be separate), and / or some other type of multiprocessor architecture in which multiple physically separate processors are linked in some way. In some examples, the processor 110 may also include a graphics processing section, as well as dedicated memory, multithreading, and / or some other parallel processing capability. In some examples, the processor 110 may be an application-specific integrated circuit (ASIC) or a field-programmable integrated circuit (FPGA).

[0040] The memory 112 may include a logic section, part of which includes an array of integrated circuits, forming a non-volatile memory for persistently storing data, or a combination of non-volatile and volatile memory. The memory 112 may be based on any of a variety of technologies. In particular, the array of integrated circuits included in the memory 112 may be arranged to form one or more types of memory, such as dynamic random access memory (DRAM), NAND memory, or NOR memory.

[0041] The I / O device 114 may be any of a variety of devices for receiving inputs and / or providing outputs. For example, the I / O device 114 may include a keyboard, mouse, joystick, foot pedal, haptic feedback device, LED, etc. The display 116 may be a conventional display or a touch-enabled display. Furthermore, the display 116 can utilize various display technologies such as liquid crystal displays (LCDs), light-emitting diodes (LEDs), or organic light-emitting diodes (OLEDs).

[0042] The network interface 118 may include logic sections and / or functions to support communication interfaces. For example, the network interface 118 may include one or more interfaces that operate according to various communication protocols or standards for communication directly or over a network communication link. Direct communication may be performed via the use of communication protocols or standards described in one or more industry standards (including derivatives and variations). For example, the network interface 118 may facilitate communication over buses such as Peripheral Component Interconnection Express (PCIe®), Non-Volatile Memory Express (NVMe®), Universal Serial Bus (USB), System Management Bus (SMBus®), SAS (e.g., Serial Attachment Small Computer System Interface (SCSI)), and Serial AT Attachment (SATA) interfaces. The network interface 118 may also include logic sections and / or functions that enable communication over various wired or wireless network standards (e.g., 802.11 communication standards). For example, the network interface 118 may be configured to support wired communication protocols or standards such as Ethernet®. As another example, the network interface 118 may be configured to support wireless communication protocols or standards such as Wi-Fi®, Bluetooth®, ZigBee®, LTE, and 5G.

[0043] Memory 112 may include multiple instructions 120, angiography images 122, IVUS images 124, vascular pressure 126, vascular lumen profile information 128, co-registration information 130, vascular physiological structure 132, and graphic information elements 134.

[0044] During operation, the processor 110 may cause the computing unit 106 to receive IVUS images 124 from the intravascular imager 102 by executing instructions 120. Generally, the IVUS images 124 are multidimensional and multivariate images that include indicators such as vessel type, intravascular lesion, lesion type, stent detection, lumen boundary, lumen dimensions, minimum lumen area (MLA), medial boundary (e.g., medial boundary of the intravascular media), medial dimensions, calcification angle / arc, calcification coverage, and combinations thereof.

[0045] The processor 110 can, by further executing instruction 120, cause the arithmetic unit 106 to receive angiographic images 122 from the angiography imager 104. Generally, angiographic images 122 are X-ray images of the blood vessels in the patient's heart. The blood vessels are visualized by X-rays by injecting a contrast agent into the blood vessels (for example, via a catheter) and capturing X-ray images while the contrast agent is acting.

[0046] Optionally, the processor 110 may further cause the arithmetic unit 106 to receive the vascular pressure 126 by executing an instruction 120. In some embodiments, the processor 110 may cause the arithmetic unit 106 to automatically receive the vascular pressure 126 (e.g., from a pressure sensor 108, etc.) by executing an instruction 120. In other embodiments, the processor 110 may give instructions to the arithmetic unit 106 from a user of the vascular physiological modeling system 100 by executing an instruction 120. For example, a physician may input the vascular pressure 126 using an I / O device 114.

[0047] The processor 110 can further cause the computing unit 106 to determine vascular lumen profile information 128 from the IVUS images 124 by executing instructions 120. For example, the processor 110 can automatically determine the lumen area at various points along the vessel from the IVUS images 124 by executing instructions 120. As another example, the processor 110 can automatically determine the vessel boundaries at various points along the vessel from the IVUS images 124 by executing instructions 120. As yet another example, the processor 110 can automatically determine the plaque load of the vessel at various points along the vessel from the IVUS images 124 by executing instructions 120. These are just a few examples of the evaluations that can be represented in the vascular lumen profile information 128.

[0048] The processor 110 can further cause the arithmetic unit 106 to co-register the angiographic image 122 and the IVUS image 124 by executing instruction 120. The IVUS image 124 is a series of cross-sectional views of a vessel acquired during the pullback of an intravascular ultrasound transducer (e.g., intravascular imager 102), depicting the lumen and arterial wall of the vessel. The angiographic image 122 is an image captured by an X-ray beam emitted in a vessel (e.g., angiographic imager 104) during the period when a contrast agent is injected into the vessel, creating the outline of the intraluminal silhouette of the vessel. Therefore, the angiographic image 122 and the IVUS image 124 are complementary. However, if they are captured by different instruments (e.g., intravascular imager 102 and angiographic imager 104, etc.), the position of the captured IVUS image 124 does not correlate with the position on the angiographic image 122. Therefore, a process is provided for registering or mapping the IVUS image 124 to a position on the angiography image 122, which is referred to herein as co-registration.

[0049] Since several co-registration processes are available, a complete description of the co-registration procedure is not provided herein. Generally, by executing instruction 120, the processor 110 may receive (or determine) the start and end positions of a "pull-back" operation that brings about an IVUS image 124 on an angiographic image 122. Furthermore, by executing instruction 120, the processor 110 may determine the positions of landmarks (e.g., side branches, etc.) in both the IVUS image 124 and the angiographic image 122 and map these positions to each other to obtain co-registration information 130.

[0050] The processor 110 can further cause the arithmetic unit 106 to generate a vascular physiological structure 132 from the angiographic image 122 and the IVUS image 124 by executing the instruction 120. In some embodiments, the processor 110 can cause the arithmetic unit 106 to generate a vascular physiological structure 132 from the angiographic image 122, the IVUS image 124, and the vascular pressure 126 by executing the instruction 120. Alternatively, the processor 110 can generate a vascular physiological structure 132 from the angiographic image 122, the IVUS image 124, the vascular lumen profile information 128, the co-registration information 130, and optionally the vascular pressure 126 by executing the instruction 120. In some embodiments, the vascular physiological structure 132 is a 3D model of the physiological structure of the blood vessels represented in the IVUS image 124 and captured on the angiographic image 122.

[0051] In some embodiments, the processor 110 may, by executing instructions 120, generate a vascular physiological structure 132 from angiography images 122, IVUS images 124, vascular lumen profile information 128, co-registration information 130, and optionally vascular pressure 126 using a machine learning model (e.g., a neural network (NN), a convolutional neural network (CNN), a random forest model, etc.). In other examples, the processor 110 may, by executing instructions 120, generate a vascular physiological structure 132 using a numerical analysis model such as a computational fluid dynamics (CFD) model. The processor 110 may, by executing instructions 120, generate a vascular physiological structure 132 from a machine learning model trained with the inputs described herein and the expected output of a vascular physiological model generated using a CFD model. This will be explained in more detail below, for example, with respect to Figure 5.

[0052] In some embodiments, the processor 110 may, by executing an instruction 120, generate a graphic information element 134 that includes an indicator of the vascular physiological structure 132, and display the graphic information element 134 on the display 116 for the user.

[0053] Figures 2A, 2B, and 2C illustrate an exemplary endovascular treatment system 200, which are described below. Figure 2A is a component-level diagram, and Figures 2B and 2C are side and perspective views of a portion of the endovascular treatment system 200 of Figure 2A, respectively. The endovascular treatment system 200 has the form of an IVUS imaging system and can be implemented as part of the vascular physiological modeling system 100 of Figure 1. The endovascular treatment system 200 includes a catheter 202 and a control subsystem 204. The control subsystem 204 includes a computing unit 106, a drive unit 206, and a pulse generator 208. The catheter 202 and the control subsystem 204 are operably coupled, more specifically, the catheter 202 is electrically and / or mechanically coupled to the computing unit 106, the drive unit 206, and the pulse generator 208 so that signals (e.g., control, measurement, image data, etc.) can be communicated between the catheter 202 and the control subsystem 204.

[0054] The computing unit 106 includes a display 116. However, in some applications, the display 116 may be provided as a separate unit from the computing unit 106, for example, inside a different housing. In some examples, a pulse generator 208 forms electrical pulses that can be input to one or more transducers 230 located inside the catheter 202.

[0055] In some examples, mechanical energy from a drive unit 206 may be used to drive an imaging core 224 located within the catheter 202. In some examples, electrical signals transmitted from one or more transducers 230 may be input to a processor 110 of the computing unit 106 for processing outlined herein. For example, electrical signals transmitted from one or more transducers 230 may be used to generate vascular lumen profile information 128 and graphic information elements 134. In some examples, processed electrical signals from one or more transducers 230 may be displayed on a display 116 as one or more images.

[0056] In some examples, the processor 110 may be used to control the function of one or more of the other components of the control subsystem 204. For example, the processor 110 may be used to control the frequency or duration of electrical pulses transmitted from the pulse generator 208, the rotational speed of the imaging core 224 by the drive unit 206, the speed or length of the pullback of the imaging core 224 by the drive unit 206, or at least one of the features of one or more images formed on the display 116, such as vascular lumen profile information 128 and graphic information elements 134.

[0057] Figure 2B is a side view of one embodiment of the catheter 202 of the intravascular treatment system 200 shown in Figure 2A. The catheter 202 includes a length member 210 and a hub 212. The length member 210 includes a proximal end 214 and a distal end 216. In Figure 2B, the proximal end 214 of the length member 210 is coupled to the catheter hub 212, and the distal end 216 of the length member 210 is configured and positioned to allow percutaneous insertion into the patient. Optionally, the catheter 202 may define at least one flush port, such as a flush port 218. The flush port 218 may be defined within the hub 212. The hub 212 may be configured and positioned to be coupled to a control subsystem 204 of the intravascular treatment system 200. In some examples, the length member 210 and the hub 212 are formed as a single unit. In other examples, the length member 210 and the catheter hub 212 are formed separately and then assembled.

[0058] Figure 2C is a perspective view of one embodiment of the distal end 216 of the elongated member 210 of a catheter 202. The elongated member 210 includes a sheath 220 having a longitudinal axis (e.g., a central longitudinal axis extending axially through the center of the sheath 220 and / or catheter 202) and a lumen 222. An imaging core 224 is provided within the lumen 222. The imaging core 224 includes an imaging device 226 coupled to the distal end of a drive shaft 228 which is rotatable manually or using a computer-controlled drive mechanism. One or more transducers 230 may be mounted on the imaging device 226. One or more transducers 230 may be used to transmit and receive acoustic signals. The sheath 220 may be formed from any flexible biocompatible material suitable for insertion into a patient. Examples of suitable materials include, for example, polyethylene, polyurethane, plastic, spiral-cut stainless steel, nitinol hypo tubing, or combinations thereof.

[0059] In some examples, an array of transducers 230 is mounted on the imaging device 226, for example, as shown in these figures. Alternatively, a single transducer may be used. Any appropriate number of transducers 230 may be used. For example, there may be 2, 3, 4, 5, 6, 7, 8, 9, 10, 12, 15, 16, 20, 25, 50, 100, 500, 1000 or more transducers. As can be understood, other numbers of transducers may also be used. When multiple transducers 230 are employed, the transducers 230 may be configured in any appropriate array, including, for example, a ring array, a rectangular array, etc.

[0060] One or more transducers 230 may be formed from a material capable of converting applied electrical pulses into pressure strains on the surface of one or more transducers 230, and vice versa. Examples of suitable materials include piezoelectric ceramic materials, piezoelectric composite materials, piezoelectric plastics, barium titanate, lead zirconate titanate, lead metaniobate, and polyvinylidene fluoride. Other transducer technologies include composite materials, single-crystal composite materials, and semiconductor devices (e.g., capacitive microfabrication ultrasonic transducers ("cMUT"), piezoelectric microfabrication ultrasonic transducers ("pMUT"), etc.).

[0061] Pressure strain on the surface of one or more transducers 230 forms acoustic pulses of a frequency based on the resonant frequencies of the one or more transducers 230. The resonant frequencies of the one or more transducers 230 may be influenced by the size, shape, and material used to form the one or more transducers 230. The one or more transducers 230 may be formed into any shape suitable for placement within the catheter 202 and for propagating acoustic pulses of a desired frequency in one or more selected directions. For example, the transducers may be disk-shaped, block-shaped, rectangular, elliptical, etc. The one or more transducers may be formed into a desired shape by any process including, for example, dicing, die-and-fill, machining, micro-machining, etc.

[0062] As an example, each of one or more transducers 230 may include a layer of piezoelectric material sandwiched between a matching layer and a conductive backing material formed from an acoustic absorbing material (e.g., an epoxy substrate having tungsten particles). During operation, the piezoelectric layer can be electrically excited to cause the emission of acoustic pulses.

[0063] One or more transducers 230 may be used to form radial cross-sectional images of the surrounding space. For example, when one or more transducers 230 are placed in a catheter 202 and inserted into a patient's blood vessel, one or more transducers 230 may be used to form images of the blood vessel wall and the surrounding tissue.

[0064] The imaging core 224 rotates around the longitudinal axis of the catheter 202. As the imaging core 224 rotates, one or more transducers 230 emit acoustic signals in different radial directions (e.g., along different radial scan lines). For example, one or more transducers 230 may emit acoustic signals in regular (or irregular) increments, such as 256 radial scan lines per rotation. Alternatively, it may be understood that other numbers of radial scan lines can be emitted per rotation.

[0065] When an emitted acoustic pulse with sufficient energy reaches one or more medium boundaries, such as one or more tissue boundaries, a portion of the emitted acoustic pulse is reflected back to the transducer that emitted it as an echo pulse. Each echo pulse that reaches the transducer with sufficient energy for detection is converted into an electrical signal in the receiving transducer. One or more converted electrical signals are transmitted to the processor 110 of the arithmetic unit 106, where they are processed to form an IVUS image 124 and further generate vascular lumen profile information 128 and graphic information elements 134 to be displayed on the display 116. In some examples, the rotation of the imaging core 224 is driven by a drive unit 206, which may be located within the control subsystem 204. In alternative embodiments, one or more transducers 230 are fixed in place and do not rotate. In that case, instead, the drive shaft 228 may rotate a mirror that reflects acoustic signals between the fixed one or more transducers 230 and the mirror.

[0066] When one or more transducers 230 are rotated around the longitudinal axis of a catheter 202 that emits acoustic pulses, multiple images can be formed that collectively form radial cross-sectional images (e.g., tomographic images) of a portion of the region surrounding one or more transducers 230, such as the wall of the blood vessel of interest and the surrounding tissue. These radial cross-sectional images can form the basis of the IVUS image 124 and can be optionally displayed on the display 116. At least one of the imaging cores 224 can be rotated manually or using a computer-controlled mechanism.

[0067] Furthermore, the imaging core 224 may move longitudinally along the vessel into which the catheter 202 is inserted, so that multiple cross-sectional images can be formed along the longitudinal length of the vessel. During the imaging process, one or more transducers 230 may be retracted (e.g., pulled back) along the longitudinal length of the catheter 202. The catheter 202 may include at least one expandable section that can be retracted during the pullback of one or more transducers 230. In some examples, a drive unit 206 drives the pullback of the imaging core 224 within the catheter 202. The pullback distance of the imaging core by the drive unit 206 may be any appropriate distance, for example, at least 5 cm, 10 cm, 15 cm, 20 cm, 25 cm, or more. The entire catheter 202 can be retracted during the imaging process, regardless of whether the imaging core 224 moves longitudinally independently of the catheter 202.

[0068] Optionally, a stepping motor may be used to pull back the imaging core 224. The stepping motor allows the imaging core 224 to be pulled back a short distance, stopping for a length of time sufficient for one or more transducers 230 to acquire an image or a series of images, and then the imaging core 224 to be pulled back another short distance to acquire another image or a series of images again, and so on.

[0069] The quality of images generated at different depths by one or more transducers 230 may be influenced by one or more factors, including, for example, bandwidth, transducer focus, beam pattern, and acoustic pulse frequency. The frequency of the acoustic pulses output from one or more transducers 230 may also affect the penetration depth of the acoustic pulses output from one or more transducers 230. Generally, as the frequency of the acoustic pulse decreases, the penetration depth of the acoustic pulse in patient tissue increases. In some examples, the endovascular treatment system 200 operates within a frequency range of 5 MHz to 200 MHz.

[0070] One or more conductors 232 can electrically couple the transducer 230 to the control subsystem 204. In this case, one or more conductors 232 may extend along the longitudinal length of the rotatable drive shaft 228.

[0071] A catheter 202, equipped with one or more transducers 230 attached to the distal end 216 of an imaging core 224, can be percutaneously inserted into a patient via an accessible blood vessel, such as the femoral artery, femoral vein, or jugular vein, at a site away from a selected portion of the selected area to be imaged, such as a blood vessel. The catheter 202 can then be advanced through the patient's blood vessel to a selected imaging site, such as a portion of a selected blood vessel.

[0072] Each time one or more acoustic signals are output to the surrounding tissue and one or more corresponding echo signals are received by the imaging device 226 and transmitted to the processor 110 of the arithmetic unit 106, an image or image frame ("frame") may be generated. Alternatively, the image or image frame may be a composite of scan lines based on a full or partial rotation of the imaging core or device. Multiple (e.g., a series of) frames may be acquired over time during any type of movement of the imaging device 226. For example, frames may be acquired during rotation and pullback of the imaging device 226 along the imaging position of the target. Frames may be acquired with or without rotation of the imaging device 226, and with or without pullback. Furthermore, frames may be acquired using other types of movement procedures in addition to, or instead of, at least one of rotation and pullback of the imaging device 226.

[0073] In some examples, when the pullback is performed, the pullback may be at a constant speed, in which case it provides a tool for potential applications that can calculate longitudinal vessel / plaque measurements. In some examples, the imaging device 226 is pulled back at a constant speed of approximately 0.3–0.9 mm / s or approximately 0.5–0.8 mm / s. In some examples, the imaging device 226 is pulled back at a constant speed of at least 0.3 mm / s. In some examples, the imaging device 226 is pulled back at a constant speed of at least 0.4 mm / s. In some examples, the imaging device 226 is pulled back at a constant speed of at least 0.5 mm / s. In some examples, the imaging device 226 is pulled back at a constant speed of at least 0.6 mm / s. In some examples, the imaging device 226 is pulled back at a constant speed of at least 0.7 mm / s. In some examples, the imaging device 226 is pulled back at a constant speed of at least 0.8 mm / s.

[0074] In some examples, one or more acoustic signals are output to the surrounding tissue at regular time intervals. In some examples, one or more corresponding echo signals are received by the imaging device 226 and transmitted to the processor 110 of the computing device 106 at regular time intervals. In some examples, the resulting frames are generated at regular time intervals.

[0075] Figure 3 shows routine 300 according to several embodiments of the present disclosure. Routine 300 can be implemented by the vascular physiological modeling system 100 described herein or another computing device to provide a 3D physiological representation or vessel from a single angiographic image, a series of intravascular images, and pressure or flow rate.

[0076] Routine 300 may begin in block 302. In block 302, “receive angiographic images associated with the patient’s blood vessels from the fluoroscopy device to the computing device,” the computing device 106 of the vascular physiological modeling system 100 receives an angiographic image 122 from the angiography scanner 104. This angiographic image 122 is an angiographic image captured by the X-ray machine while the blood vessels are exposed to the contrast agent. For example, the processor 110 may receive data including a display of the angiographic image 122 from the angiography scanner 104 via the network interface 118 by executing instruction 120.

[0077] Proceeding to Block 304, "The computing unit receives multiple images, including multidimensional and multivariate images associated with the patient's blood vessels, from the intravascular imaging device," the computing unit 106 of the vascular physiological modeling system 100 receives IVUS images 124 from the intravascular imager 102. These IVUS images 124 are multidimensional and multivariate images of the blood vessels. For example, the processor 110 may receive data including a display of the IVUS images 124 from the intravascular imager 102 via the network interface 118 by executing instruction 120.

[0078] Proceeding to "Generate information including a display of co-registered multiple images and angiographic images using a processing unit" in block 306, information may be generated that includes a display of mapping or registration of a portion of a blood vessel shown in the image received in block 302 and an image of a portion of a blood vessel received in block 304. For example, by executing instruction 120, processor 110 may identify the start and end points of a pullback operation through a blood vessel on the angiographic image 122. Furthermore, by executing instruction 120, processor 110 may identify one or more other landmarks (e.g., a blood vessel centerline, a side branch, etc.) in both the angiographic image 122 and the IVUS image 124, and may map or adjust the frames of the IVUS image 124 to positions on the angiographic image 122 based on the identified start and end points and other landmarks. As described above, co-registration is a complex process, and the specific details of actual co-registration are outside the scope of this specification. Furthermore, by executing instruction 120, processor 110 may store the display of co-registration as co-registration information 130.

[0079] Proceeding to "Generate information including a display of vascular lumen evaluation based on multiple images using a computing device" in block 308, information including a display of vascular lumen evaluation may be generated based on the images received in block 304. For example, by executing instruction 120, processor 110 may automatically identify the geometric features (e.g., diameter, area, etc.) of blood vessels and lumens represented in the images received in blocks 302 and 304, and store this display as vascular lumen profile information 128. In some embodiments, by executing instruction 120, processor 110 may generate lumen and blood vessel evaluations based on machine learning, image processing, geometric image analysis, etc.

[0080] Proceeding to Block 310, "Should additional physiological features be used?", the computing unit 106 of the vascular physiological modeling system 100 may determine whether to use additional physiological features in the 3D model generation process (for example, as described above). From Block 310, Method 300 may proceed to either Block 312 or Block 316. Based on the determination in Block 310 that additional physiological features are used when generating a 3D model of the vascular physiological structure, Method 300 may proceed from Block 310 to Block 312. On the other hand, based on the determination in Block 310 that additional physiological features are not used when generating a 3D model of the vascular physiological structure, Method 300 may proceed from Block 310 to Block 316.

[0081] In block 312, "Receiving indicators of vascular physiological characteristics with the computing device," the computing device 106 of the vascular physiological modeling system 100 receives the vascular pressure 126. For example, the processor 110 may automatically receive the vascular pressure 126 from the pressure sensor 108 by executing instruction 120. Alternatively, the processor 110 may receive the vascular pressure 126 from the user via an I / O device 114, etc., by executing instruction 120. In some embodiments, the vascular pressure 126 may include aortic pressure. In other embodiments, the vascular pressure 126 may include blood flow velocity measurements. In some embodiments, the vascular pressure 126 may include aortic pressure measurements (e.g., FFR, DFR, etc.) acquired during a pullback operation associated with the IVUS image 124.

[0082] Moving from block 312 to block 314, "Generating a 3D model of the physiological structure of blood vessels in a computing device based on multiple images, angiographic images, and physiological features," a 3D model of the physiological structure of blood vessels represented in the angiographic image 122 and IVUS image 124 may be generated. For example, by executing instruction 120, processor 110 may generate a 3D model or representation of the physiological structure of a blood vessel (or part of a blood vessel) represented in the IVUS image 124 and shown in 2D format on the angiographic image 122, and store the display of that 3D model as vascular physiological structure 132. By executing instruction 120, processor 110 may generate a 3D model using other physiological features (e.g., vascular pressure 126, etc.) along with vascular lumen profile information 128 and co-registration information 130. More specifically, the processor 110 can generate a 3D representation from the generated lumen diameter, generated centerline, and 2D representation of the blood vessel shown in the angiographic image 122 by executing the instruction 120. Specifically, when the mapping between the IVUS image 124 and the angiographic image 122 is provided from the co-registration information 130, the lumen diameter of the vascular lumen profile information 128 can be used to form a 3D model shown by the vascular physiological structure 132.

[0083] In block 316, "Generating a 3D model of the physiological structure of a blood vessel in a computing device based on multiple images and angiography images," a 3D model of the physiological structure of a blood vessel shown in angiography image 122 and IVUS image 124 can be generated. For example, by executing instruction 120, processor 110 can generate a 3D model or representation of the physiological structure of a blood vessel (or part of a blood vessel) shown in IVUS image 124 and in 2D format on angiography image 122, and store the display of that 3D model as vascular physiological structure 132. By executing instruction 120, processor 110 can generate a 3D model using vascular lumen profile information 128 and co-registration information 130. More specifically, by executing instruction 120, processor 110 can generate a 3D representation from the generated lumen diameter, generated centerline, and 2D representation of the blood vessel shown in angiography image 122. Specifically, when a mapping between IVUS images 124 and angiography images 122 is provided from co-registration information 130, the lumen diameter of the vascular lumen profile information 128 is used to form a 3D model represented by the vascular physiological structure 132.

[0084] Method 300 can proceed from blocks 314, 316 to block 318. In block 318, "Generating a graphic information element including a 3D model display using a computing device," a graphic information element including a display of a vascular physiological structure 132 (e.g., a 3D model) can be generated. For example, by executing instruction 120, processor 110 can generate a graphic information element 134 including a display of a 3D model of a blood vessel shown in the angiographic image 122 and IVUS image 124, which is shown in the vascular physiological structure 132.

[0085] Proceeding to "display the graphic information element on the display using the arithmetic unit" in block 320, the graphic information element generated in block 314 can be displayed on the display. For example, the processor 110 can display the graphic information element 134 on the display 116 by executing instruction 120.

[0086] Figures 4A to 4H illustrate examples of images and evaluations described herein. These figures are illustrated with reference to the operation or blocks of routine 300 in Figure 3. However, this is done for the purpose of clarifying the presentation and is not limiting. Referring to Figure 4A, an angiographic image 400a is shown. As described above, the vascular physiological modeling system 100 may receive the angiographic image 400a (or an information element and / or data structure containing a representation of the angiographic image 400a) in block 302 by executing instruction 120.

[0087] Figures 4B and 4C show the on-axial IVUS image view 400b and IVUS image 400c, respectively. For example, the on-axial IVUS image view 400b shows the on-axial (or short-axis) view of the vessel represented by the IVUS image 124 in one frame of a plurality of IVUS images 124, and the IVUS image 400c shows the longitudinal view of the vessel between the start and end of the pullback operation that generated the plurality of IVUS images 124. As described above, the vascular physiological modeling system 100 may receive the on-axial IVUS image view 400b and IVUS image 400c (or information elements and / or data structures including the display of the on-axial IVUS image view 400b and IVUS image 400c) in block 304 by executing the command 120.

[0088] Figure 4D shows an angiographic image 400a with a designated start point 402, end point 404, midpoint 406, and centerline 408 of a vessel (or vessel portion) represented in the IVUS image 124. As described above, routine 300 may include a block 306 for generating co-registration information 130 from the angiographic image 122 and the IVUS image 124. Identification of side branches (see Figure 4E) along with the start point 402, end point 404, and centerline 408 may be part of the co-registration process performed in block 306.

[0089] Figure 4E shows an angiographic image 400a with designated side branches 410 on the angiographic image 400a. Five side branches (A, B, C, D, E) are designated on the centerline 408 of the vessel, which is represented in the angiographic image 122 and the IVUS image 124. In some examples, the processor 110 may receive instructions for a start point 402, an end point 404, and an intermediate point 406 (e.g., via an I / O device 114, etc.) by executing instruction 120, and generate the centerline 408 from the designated start point 402, end point 404, intermediate point 406, and angiographic image 122. In a further example, the processor 110 may receive adjustment results for the generated centerline 408 (e.g., via an I / O device 114, etc.) by executing instruction 120. Furthermore, the processor 110 may generate the locations of the side branches 410 by executing instruction 120. As a further example, the processor 110 may, by executing the instruction 120, receive the result of the adjustment to the position of the generated side branch 410 (for example, via an I / O device 114, etc.).

[0090] Figure 4F shows a graphical user interface (GUI) 400f that displays the representation of angiographic images 122 and IVUS images 124 along with co-registration information 130 and vascular lumen profile information 128. For example, the angiographic images 122 and IVUS images 124 are shown along with the longitudinal vascular profile view 412 and vascular evaluation 414. It can be seen that the frames of the IVUS image 124 are mapped to positions along the vascular centerline 408 between the start point 402 and the end point 404. Thus, position 416a is co-registered with position 416b, i.e., they are at the same location. Furthermore, the vascular evaluation 414 (representing the vascular lumen profile information 128 generated in block 308) is shown for the frames of the IVUS image 124 corresponding to positions 416a and 416b.

[0091] Figure 4G shows a pressure curve 400g including a representation of vascular pressure 126. As described above, vascular pressure 126 can be received by the vascular physiological modeling system 100 in block 310 of routine 300 and can optionally be used when generating a 3D representation of the vessel. In this case, the graph shown in curve 400g shows the pressure along the IVUS pullback distance corresponding to a series of IVUS images 124.

[0092] Figure 4H shows a 3D model 400h of vessel 418 represented by angiographic image 122 and IVUS image 124. More specifically, the 3D model 400h shows a 3D representation of the 2D view of vessel 418 shown in angiographic image 400a (e.g., angiographic image 122). As can be seen from the figure, the physiological representation of the 3D model 400h shows both the geometric gradient and pressure gradient of vessel 418. As described above, routine 300 may include a block 312 for generating the 3D model 400h from angiographic image 122, IVUS image 124, vascular pressure 126, vascular lumen profile information 128, and co-registration information 130.

[0093] As described above, in some embodiments, the processor 110 of the arithmetic unit 106 may generate vascular physiological structures 132 using a machine learning (ML) model by executing instructions 120. In such examples, the ML model may be stored in the memory 112 of the arithmetic unit 106. The ML model may be trained before deployment. Figure 5 shows an ML environment 500 that can be used to train an ML model which may later be used to generate (or infer) the vascular physiological structures 132 described herein. The ML environment 500 may include an ML system 502, such as an arithmetic unit, which applies an ML algorithm to learn relationships. In this example, the ML algorithm can learn relationships between a set of inputs (e.g., angiographic images 122, IVUS images 124, vascular lumen profile information 128, co-registration information 130, and optionally vascular pressure 126) and outputs (e.g., vascular physiological structures 132).

[0094] The ML system 502 may utilize experience data 508 collected during several previous procedures. The experience data 508 may include angiographic images 122 and IVUS images 124 of several patients. The experience data 508 may be located with the ML system 502 (for example, stored in the storage device 510 of the ML system 502), or it may be located remotely from the ML system 502 and accessed via the network interface 504, or it may be a combination of local and remote data.

[0095] The experience data 508 may be used to form the training data 512. As described above, the ML system 502 may include a storage device 510. This storage device 510 may include a hard drive, solid-state storage, and / or random-access memory. The storage device 510 may hold the training data 512. Generally, the training data 512 may include information elements or data structures that include displays of angiographic images 122 and IVUS images 124 of several patients. The training data 512 may also optionally include the patients' vascular pressures 126. Furthermore, in some embodiments, the training data 512 may include vascular lumen profile information 128 and co-registration information 130 for each patient. In some embodiments, the experience data 508 includes only the patient's angiographic images 122 and IVUS images 124, and the ML system 502 is configured to generate vascular lumen profile information 128 and co-registration information 130 from the angiographic images 122 and IVUS images 124 of each patient represented in the experience data 508 (for example, using a processor and instructions executable by that processor).

[0096] The training data 512 can be applied to train the ML model 514. Depending on the application, different types of models may be used to form the basis of the ML model 514. In this example, an artificial neural network (ANN) may be particularly suitable for learning the associations between angiographic images (e.g., angiographic image 122), IVUS images (e.g., IVUS image 124), and a 3D model of vascular physiological structure (e.g., vascular physiological structure 132). A convolutional neural network is also well suited to this task. Any suitable training algorithm 516 can be used to train the ML model 514. The example shown in Figure 5 may be particularly suitable for supervised training algorithms or reinforcement learning training algorithms. In the case of a supervised training algorithm, the ML system 502 may apply the angiographic images 122 and IVUS images 124 (and optionally, vascular pressure 126, vascular lumen profile information 128, and / or co-registration information 130) as model inputs 518. For this model input 518, an expected output (e.g., vascular physiological structure 132) generated from the training data 512 using a CFD modeler 520 may be mapped to learn the association between the model input 518 and the vascular physiological structure 132. In the case of reinforcement learning, the training algorithm 516 may generate an ML model 514 with the minimum error by attempting to maximize the mapping of some or all (or weighted combinations) of the model input 518 to the vascular physiological structure 132. In some embodiments, the training data 512 may be split into “training” data and “test” data. In this case, several subsets of the training data 512 may be used to tune the ML model 514 (e.g., the model’s internal weightings), while another non-overlapping subset of the training data 512 may be used to measure the accuracy of the ML model 514 and to infer (or generalize) the vascular physiological structure 132 from “unseen” training data 512 (e.g., training data 512 not used to train the ML model 514).

[0097] The ML model 514 may be applied using a processor circuit 506 which may include appropriate hardware processing resources operating on the logic and structure within the storage device 510. The development of the training algorithm 516 and / or the trained ML model 514 may depend at least in part on the hyperparameters 522. In an exemplary embodiment, the model hyperparameters 522 may be automatically selected based on a hyperparameter optimization logic 524 which may include any known hyperparameter optimization techniques appropriate for the selected ML model 514 and the training algorithm 516 used. In an optional embodiment, the ML model 514 may be retrained over time to adapt to new knowledge and / or updated empirical data 508.

[0098] Once the ML model 514 is trained, it can be applied to new input data (e.g., angiographic images 122 and IVUS images 124 captured during pre-PCI intervention) (e.g., by a processor circuit 506 or processor 110, etc.). This input to the ML model 514 can be formatted according to a predetermined model input 518 that reflects how the training data 512 was provided to the ML model 514. The ML model 514 can generate a vascular physiological structure 132, which may be a generalization or inference of the 3D vascular physiological structure of the vessels represented in the angiographic images 122 and IVUS images 124 provided as input to the ML model 514.

[0099] The above description relates to a specific type of ML system 502 that applies a supervised learning technique when given available training data having input / result pairs. However, the present invention is not limited to use in a specific ML paradigm, and other types of ML techniques may also be used. For example, in some embodiments, the ML system 502 may generate vascular physiological structures 132 from angiographic images 122 and IVUS images 124 by applying, for example, evolutionary algorithms or other types of ML algorithms and models.

[0100] Figure 6 shows a computer-readable storage medium 600. The computer-readable storage medium 600 may include any non-temporary computer-readable or machine-readable storage medium, such as an optical storage medium, a magnetic storage medium, or a semiconductor storage medium. In various embodiments, the computer-readable storage medium 600 may include a manufactured product. In some embodiments, the computer-readable storage medium 600 may store computer-executable instructions 602 that a circuit (e.g., a processor 110) can execute. For example, the computer-executable instructions 602 may include instructions for implementing the operation described with respect to routine 300 and may be specifically programmed to cause the vascular physiological modeling system 100 to perform the operation described with reference to routine 300 and Figure 3. In another example, the computer-executable instructions 602 may include instructions 120, an ML model 514, and / or a training algorithm 516. Examples of computer-readable storage media 600 or machine-readable storage media may include any tangible medium capable of storing electronic data, including volatile or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writable or rewritable memory, etc. Examples of computer executable instructions 602 may include any appropriate type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, etc.

[0101] Figure 7 shows a schematic diagram of machine 700 in the form of a computer system in which a set of instructions can be executed to cause the machine to perform any one or more of the methods described herein. Specifically, Figure 7 shows a schematic diagram of machine 700 in the exemplary form of a computer system in which a set of instructions 708 (e.g., software, programs, applications, applets, apps, or other executable code) can be executed within machine 700 to cause machine 700 to perform any one or more of the methods described herein. For example, a set of instructions 708 could cause machine 700 to execute instruction 120, routine 300 in Figure 3, training algorithm 516, etc. More generally, a set of instructions 708 could cause machine 700 to generate a 3D model or vascular physiological information from a single angiographic image, a series of IVUS images, and vascular pressure measurements, as described herein.

[0102] Multiple instructions 708 translate a general, unprogrammed machine 700 into a specific machine 700 programmed to perform the functions described and illustrated in a particular manner. In alternative embodiments, machine 700 may operate as a standalone device or be coupled to other machines (e.g., networked). In a networked configuration, machine 700 may operate as a server or client machine in a server-client network environment, or as a peer machine in a peer-to-peer (or distributed) network environment. Machine 700 may include, but is not limited to, a server computer, a client computer, a personal computer (PC), a tablet computer, a laptop computer, a netbook, a set-top box (STB), a PDA, an entertainment media system, a mobile phone, a smartphone, a mobile device, a wearable device (e.g., a smartwatch), a smart home device (e.g., a smart appliance), another smart device, a web appliance, a network router, a network switch, a network bridge, or any machine capable of sequentially or otherwise executing multiple instructions 708 that specify the actions performed by machine 700. Furthermore, although only a single machine 700 is illustrated, the term “machine” can also be interpreted to include a set of machines 200 that individually or collaboratively execute multiple instructions 708 to perform any one or more of the methods described herein.

[0103] Machine 700 may include a processor 702, memory 704, and I / O components 742, which may be configured to communicate with each other via a bus 744, etc. In an exemplary embodiment, the processor 1902 (e.g., a central processing unit (CPU), a reduced instruction set arithmetic (RISC) processor, a composite instruction set arithmetic (CISC) processor, a graphics processing unit (GPU), a digital signal processor (DSP), an ASIC, a radio frequency integrated circuit (RFIC), another processor, or any suitable combination thereof) may include, for example, processors 706 and 710, which are capable of executing multiple instructions 708. The term “processor” is intended to include multicore processors, which may include two or more independent processors (sometimes called “cores”) capable of executing instructions simultaneously. Figure 7 shows multiple processors 702, but machine 700 may include a single processor with a single core, a single processor with multiple cores (e.g., a multicore processor), multiple processors with a single core, multiple processors with multiple cores, or any combination thereof.

[0104] Memory 704 may include main memory 712, static memory 714, and storage unit 716, which are accessible to processor 702 via bus 744, etc. Main memory 704, static memory 714, and storage unit 716 store a plurality of instructions 708 that embody any one or more of the plurality of methods or functions described herein. The plurality of instructions 708 may also be fully or partially present in main memory 712, static memory 714, machine-readable media 718 in storage unit 716, in at least one of processor 702 (e.g., in the processor's cache memory), or any suitable combination thereof, during their execution by machine 700.

[0105] The I / O component 742 may include a wide variety of components for receiving inputs, providing outputs, generating outputs, transmitting information, exchanging information, capturing measurements, and so on. The specific I / O component 742 included in a particular machine depends on the type of machine. For example, portable devices such as mobile phones are likely to include touch input devices or other such input mechanisms, while headless server machines are unlikely to include such touch input devices. The I / O component 742 may also include many other components not shown in Figure 7. The I / O component 742 is grouped according to function simply to simplify the following description, and this grouping is not limiting. In various exemplary embodiments, the I / O component 742 may include an output component 728 and an input component 730. The output component 728 may include visual components (e.g., displays such as plasma display panels (PDPs), light-emitting diode (LED) displays, liquid crystal displays (LCDs), projectors, or cathode ray tubes (CRTs)), acoustic components (e.g., speakers), tactile components (e.g., vibration motors, resistors), other signal generators, etc. The input component 730 may include alphanumeric input components (e.g., keyboards, touchscreens configured to receive alphanumeric input, photo-optical keyboards, or other alphanumeric input components), point-based input components (e.g., mice, touchpads, trackballs, joysticks, motion sensors, or other pointing devices), tactile input components (e.g., physical buttons, touchscreens that provide the position and / or force of touch or touch gestures, or other tactile input components), voice input components (e.g., microphones), etc.

[0106] In further exemplary embodiments, the I / O component 742 may include, among many other components, a biometric component 732, a motion component 734, an environmental component 736, or a position component 738. For example, the biometric component 732 may include components for detecting facial expressions (e.g., hand expressions, facial expressions, voice expressions, gestures, or eye tracking), measuring biosignals (e.g., blood pressure, heart rate, body temperature, sweating, or electroencephalography), and identifying people (e.g., voice recognition, retinal recognition, facial recognition, fingerprint recognition, or electroencephalography-based recognition). The motion component 734 may include acceleration sensor components (e.g., accelerometers), gravity sensor components, rotation sensor components (e.g., gyroscopes), and the like. The environmental component 736 may include, for example, an illuminance sensor component (e.g., a photometer), a temperature sensor component (e.g., one or more thermometers that detect ambient temperature), a humidity sensor component, a pressure sensor component (e.g., a barometer), an acoustic sensor component (e.g., one or more microphones that detect background noise), a proximity sensor component (e.g., an infrared sensor that detects nearby objects), a gas sensor (e.g., a gas detection sensor that detects the concentration of harmful gases for safety or measures airborne pollutants), or other components that can provide displays, measurements, or signals corresponding to the surrounding physical environment. The position component 738 may include a position sensor component (e.g., a GPS receiver component), an altitude sensor component (e.g., an altimeter or barometer that detects atmospheric pressure from which altitude can be derived), a compass sensor component (e.g., a magnetometer), and the like.

[0107] Communication can be implemented using a wide variety of technologies. The I / O component 742 may include a communication component 740 capable of connecting machine 700 to network 720 or device 722 via connections 724 and 726, respectively. For example, the communication component 740 may include a network interface component or another suitable device for interface connection with network 720. In further embodiments, the communication component 740 may include a wired communication component, a wireless communication component, a cellular communication component, a near-field communication (NFC) component, a Bluetooth® component (e.g., Bluetooth® Low Energy), a Wi-Fi® component, and other communication components for communicating via other modalities. Device 722 may be another machine or one of a wide variety of peripheral devices (e.g., a peripheral device connected via USB).

[0108] Furthermore, the communication component 740 may include components that can detect identifiers or are capable of operating to detect identifiers. For example, the communication component 740 may include a radio frequency identification (RFID) tag reader component, an NFC smart tag detection component, an optical reader component (e.g., an optical sensor for detecting one-dimensional barcodes such as Universal Product Code (UPC) barcodes, Quick Response (QR) Code®, Aztec Code, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D barcodes, and other optical codes), or an acoustic detection component (e.g., a microphone for identifying tagged audio signals). In addition, various types of information, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, and location by detection of NFC beacon signals that may indicate a specific location, can be derived via the communication component 740.

[0109] Various memories (i.e., memory 704, main memory 712, static memory 714, and / or the memory of processor 702) and / or storage unit 716 may store one or more sets of instructions and data structures (e.g., software) that embody or are utilized by any or more of the methods or functions described herein. When these instructions (e.g., a number of instructions 708) are executed by processor 702, they trigger a variety of operations for carrying out the disclosed embodiments.

[0110] As used herein, the terms “mechanical storage medium,” “device storage medium,” and “computer storage medium” mean the same thing and may be used interchangeably in this disclosure. These terms refer to one or more storage devices and / or media that store executable instructions and / or data (e.g., centralized or distributed databases, and / or associated caches and servers). Thus, these terms are to be interpreted as including, but are not limited to, solid-state memory and magneto-optical media, including memory inside or outside a processor. Specific examples of mechanical storage mediums, computer storage mediums, and / or device storage mediums include non-volatile memory, such as semiconductor memory devices like erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices, as well as magnetic disks, magneto-optical disks, such as internal hard disks and removable disks, and CD-ROM and DVD-ROM disks. The terms “mechanical storage medium,” “computer storage medium,” and “device storage medium” specifically exclude carrier waves, modulated data signals, and other such mediums, although at least some of them are included under the term “signaling medium,” as described below.

[0111] In various exemplary embodiments, one or more parts of network 720 could be an ad hoc network, intranet, extranet, VPN, LAN, WLAN, WAN, WWAN, MAN, Internet, part of the Internet, part of the PSTN, Basic Telephone Services (POTS) network, cellular telephone network, wireless network, Wi-Fi® network, another type of network, or a combination of two or more such networks. For example, network 720 or part of network 720 could include a wireless or cellular network. Connection 724 could be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM®) connection, or another type of cellular or wireless connection. In this example, connection 724 may implement any of the various types of data transfer technologies, including single-carrier radio transmission technology (1xRTT), Evolution Data Optimization (EVDO) technology, General-Purpose Packet Radio Service (GPRS) technology, Extended Data Rate (EDGE) technology for GSM® Evolution, 3G Partnership Projects (3GPP®) including 3G, 4G wireless networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Interoperability for Microwave Access (WiMAX®), Long-Term Evolution (LTE) standards, others defined by various standards-setting bodies, other long-range protocols, or other data transfer technologies.

[0112] Multiple instructions 708 may be sent and received over the network 720 using a transmission medium via a network interface device (e.g., a network interface component included in communication component 740) and utilizing one of several well-known transfer protocols (e.g., Hypertext Transfer Protocol (HTTP)). Similarly, multiple instructions 708 may be sent and received using a transmission medium via a connection 726 to device 722 (e.g., a peer-to-peer connection). The terms “transmission medium” and “signaling medium” mean the same thing and may be used interchangeably in this disclosure. The terms “transmission medium” and “signaling medium” are to be interpreted as including any intangible medium capable of storing, encoding, or carrying multiple instructions 708 for execution by machine 700, and including digital or analog communication signals or other intangible medium for facilitating communication of such software. Accordingly, the terms “transmission medium” and “signaling medium” are to be interpreted as including any form such as modulated data signals, carrier waves, etc. The term “modulated data signal” means a signal in which one or more of its properties are set or modified in order to encode information within the signal.

[0113] Terms used herein are given their usual meanings in the relevant technical field, or the meanings indicated by their use in the context; however, where a clear definition is provided, that meaning shall prevail.

[0114] In this specification, references to “one embodiment” or “a certain embodiment” do not necessarily refer to the same embodiment, nor do they necessarily refer to the same embodiment. Unless the context clearly indicates otherwise, throughout this specification and the claims, words such as “equipped with,” “equipped with,” etc., are interpreted comprehensively, as opposed to exclusive or exhaustive, meaning “including but not limited to.” Terms used singular or plural include both singular and plural, respectively, unless expressly limited to one or more. Also, when used in this application, words such as “in this specification,” “above,” “below,” and similar words refer to the entire application, not to any part thereof. When the claims use the word “or” in relation to a list of two or more items, the word includes any item in the list, all items in the list, and any combination of items in the list, unless expressly limited to one or the other. Any term not expressly defined in this specification has its conventional meaning as generally understood by those skilled in the art.

Claims

1. A method for generating a 3D model of the physiological structure of blood vessels, The computing unit receives angiographic images of the patient's blood vessels from a fluorescence fluoroscopy device. The calculation device receives a plurality of images associated with the patient's blood vessels from an intravascular imaging device, the plurality of images including multidimensional and multivariate images, and To generate a three-dimensional (3D) model of the physiological structure of the blood vessel from the angiographic image and the plurality of images, A method for providing this.

2. The arithmetic unit generates graphic information elements including the display of the 3D model, and The arithmetic unit displays the graphic information elements on a display connected to the arithmetic unit. The method according to claim 1, further comprising:

3. The method according to claim 1 or 2, wherein generating a 3D model of the physiological structure of the blood vessels includes co-registering the angiographic image and the plurality of images.

4. Identifying the starting point of the pullback operation associated with the plurality of images of the blood vessel represented in the angiographic image, Identifying the endpoint of the pullback operation associated with the plurality of images of the blood vessel shown in the angiographic image, and Identifying the center line of the blood vessel between the aforementioned starting point and the aforementioned ending point, The method according to claim 3, comprising:

5. Identifying multiple collateral branches of the blood vessel on the angiographic image and within the multiple images, To associate one of the multiple collateral branches identified on the angiographic image with one of the multiple collateral branches identified within the multiple images, The method according to claim 4, comprising:

6. The method according to claim 5, further comprising mapping the frames of the plurality of images to a position along the center line of the blood vessel on the angiographic image.

7. The method according to any one of claims 3 to 6, further comprising generating the evaluation results of the blood vessels.

8. The method according to claim 7, wherein the evaluation result includes the diameter of the blood vessel, the area of ​​the blood vessel, or the diameter and area of ​​the blood vessel.

9. The method according to claim 7, wherein the evaluation result includes the diameter of the lumen, the area of ​​the lumen, or the diameter and area of ​​the lumen.

10. The method according to any one of claims 1 to 9, comprising: receiving an index of additional physiological features of the patient's blood vessels in the computing device; and generating a 3D model of the physiological structure of the blood vessels from the angiographic image, the plurality of images, and the additional physiological features of the blood vessels.

11. The method according to claim 10, wherein the additional physiological features of the blood vessel include pressure or flow rate.

12. The method according to any one of claims 1 to 11, comprising generating an inference of a 3D model of the physiological structure of the blood vessel from the ML model, based in part on applying the angiographic image and the plurality of images as input to a machine learning (ML) model.

13. The method according to claim 12, wherein the ML model is trained in part on a supervised learning training algorithm that uses the expected output of the ML model derived on a computational fluid dynamics (CFD) model, and the CFD model takes an angiographic image and multiple images as input and generates a 3D vascular physiological model as output.

14. An apparatus comprising a processor configured to be coupled to an intravascular imaging device and a fluorescence fluoroscopy device, wherein the apparatus further comprises a memory having a plurality of instructions, and the processor is configured to perform the method according to any one of claims 1 to 13 by executing the plurality of instructions.

15. A computer-readable storage device comprising a plurality of instructions executable by a processor of a computing device coupled to an intravascular imaging device and a fluorescence fluoroscopy device, wherein when the plurality of instructions are executed, the computing device performs the method according to any one of claims 1 to 13.

Images