Vascular path identification based on extravascular images

Abstract

The present disclosure provides methods and systems for a medical imaging device to plan a vascular path on an extravascular image based on at least two points indicated on the image. An image velocity map can be generated from an extravascular image and a list of shortest paths from points on the image to selected points on the image can be identified from the image velocity map. Then, based on the identified shortest path, a path from another point on the image to the selected point on the image is determined.

Description

Cross Reference to Related Applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 459,865, filed April 17, 2023, the disclosure of which is incorporated herein by reference. TECHNICAL FIELD

[0002] The present disclosure relates to the field of medical imaging. More specifically, the present disclosure relates to identifying a vessel path in a medical image. BACKGROUND

[0003] Medical imaging is used for treatment and diagnosis of vascular diseases. There are currently a variety of imaging modalities available for generating medical images, including video, fluoroscopy, angiography, ultrasound, computed tomography (CT), magnetic resonance (MR), positron emission tomography (PET), positron emission tomography-computed tomography (PET-CT), computed tomography angiography (CTA), single photon emission computed tomography (SPECT), gamma camera imaging, optical coherence tomography (OCT), near-infrared spectroscopy (NIRS), vibrational response imaging (VRI), optical imaging, infrared imaging, electrical mapping imaging, other forms of functional imaging, focused acoustic computed tomography (FACT), optical frequency domain imaging (OFDI).

[0004] In addition, there are a variety of intraluminal medical devices used for treatment and diagnostic purposes. Devices such as intravascular ultrasound (IVUS) probes, fractional flow reserve (FFR) probes, and instantaneous wave-free ratio (iFR) probes typically acquire intraluminal data while moving through a lumen.

[0005] Generally, it is desirable to identify physiological features of a vessel to be treated. For example, a centerline of an externally imaged vessel can be determined to map a vessel route and / or to register the external image with an intraluminal image, such as an IVUS image. SUMMARY

[0006] In general, the present disclosure provides a method of identifying a vessel path between at least two points in an extraluminal image. The present disclosure can be used in pre- and / or post-procedural programs for percutaneous coronary intervention (PCI) to define a vessel shape and / or to register an extraluminal image with an intraluminal image.

[0007] The present disclosure provides a number of advantages over existing or conventional vessel path identification techniques. For example, the present disclosure requires significantly fewer points to be provided by a user (e.g., a physician) to adequately track or identify a vessel path in an extraluminal image. In addition, the present disclosure allows a user (e.g., a physician) to easily adjust an identified vessel path (or shape) and receive real-time feedback of the adjusted path.

[0008] In some embodiments, the present disclosure can be implemented as a computer- implemented method. The computer-implemented method can include: receiving, by processing circuitry, an extravascular image from an extravascular imaging device, the extravascular image containing indication information of a blood vessel; generating, by the processing circuitry, an image velocity map based on the extravascular image; receiving, by the processing circuitry, indication information of a first point on the extravascular image, the first point corresponding to a portion of the blood vessel; identifying, by the processing circuitry, a shortest distance from each of a plurality of pixels on the image to the first point based on the image velocity map; receiving, by the processing circuitry, indication information of a second point on the extravascular image, the second point corresponding to another portion of the blood vessel; determining, by the processing circuitry, a path of the blood vessel based on the second point and the shortest distance from each of the plurality of pixels to the first point.

[0009] In further embodiments, the method can include smoothing the path.

[0010] In further embodiments of the method, the path includes midpoints, and smoothing the path includes: adding intermediate points along the path on either side of the midpoints; identifying a shortest path from each of the intermediate points to the respective first and second points; selecting a line segment or a curve segment from a plurality of line segments or curve segments connecting the intermediate points based in part on the shortest path from each of the intermediate points to the respective first and second points; and forming the path from the shortest path, the selected line segment or curve segment, and the shortest path from each of the intermediate points to the respective first and second points.

[0011] In further embodiments, the method can include generating, by the processing circuitry, the image velocity map, including: debanding the extravascular image to generate a debanded extravascular image; performing brightness and / or contrast normalization on the debanded extravascular image to generate a normalized extravascular image; and darkening a centerline of the blood vessel based in part on the debanded extravascular image to generate the image velocity map.

[0012] In further embodiments, the method can include generating, by the processing circuitry, the image velocity map, further including: identifying ambient light in the debanded extravascular image; and removing the ambient light from the debanded extravascular image to form a light-corrected extravascular image, wherein the normalized extravascular image is generated based on the light-corrected extravascular image.

[0013] In further embodiments, the method can include identifying the ambient light in the debanded image based on a blur filter having a median diameter between 30 and 120 pixels.

[0014] In further embodiments, the method can include applying a Gaussian kernel to the extravascular image to deband the extravascular image.

[0015] In further embodiments of the method, the Gaussian kernel is a 3-pixel by 3-pixel diameter Gaussian kernel.

[0016] In further embodiments, the method can include iteratively applying a mask to the portion of the normalized image to progressively darken pixels corresponding to the portion of the normalized image representing the blood vessel based on a distance of the pixels to a boundary of the blood vessel.

[0017] In further embodiments, the method can include performing a gradient transform on the centerline-darkened image to generate a blood vessel velocity map.

[0018] In further embodiments of the method, the gradient transform is a Sigmoid transform or a linear transform.

[0019] In further embodiments, the method can include identifying, by the processing circuit, a shortest distance of each of a plurality of pixels on the image to the second point based on the image velocity map; receiving an indication of a movement of the first point position; and identifying an updated path of the blood vessel based on the moved first point and the shortest distance of each of the plurality of pixels to the second point.

[0020] In further embodiments, the method can include identifying, by the processing circuit, a shortest distance of each of a plurality of pixels on the image to the second point based on the image velocity map; receiving an indication of a midpoint on the extravascular image; and identifying an updated path of the blood vessel based on the midpoint, the shortest distance of each of the plurality of pixels to the first point, and the shortest distance of each of the plurality of pixels to the second point.

[0021] In some embodiments, the present disclosure can be implemented as a computing device coupled to an extravascular imaging device, such as an angiography device. The computing device can include a processor and a memory coupled to the processor, the memory containing instructions that, when executed by the processor, cause the computing device to perform any of the methods described herein.

[0022] In some embodiments, the present disclosure can be implemented as a computer-readable medium containing instructions that, when executed by a processor of a medical imaging device, cause the medical imaging device to perform any of the methods described herein.

[0023] In some embodiments, the present disclosure can be implemented as a computing device for extravascular image processing system. The computing device can include a processor; and a storage device coupled to the processor, the storage device containing instructions that, when executed by the processor, cause the computing device to: receive an extravascular image from an extravascular imaging device, the extravascular image containing an indication of a blood vessel; generate, by processing circuitry, an image velocity map based on the extravascular image; receive, by the processing circuitry, an indication of a first point on the extravascular image, the first point corresponding to a portion of the blood vessel; identify, by the processing circuitry, a shortest distance from each of a plurality of pixels on the image to the first point based on the image velocity map; receive, by the processing circuitry, an indication of a second point on the extravascular image, the second point corresponding to another portion of the blood vessel; determine, by the processing circuitry, a blood vessel path based on the second point and the shortest distance from each of the plurality of pixels to the first point.

[0024] In further embodiments of the computing device, the path includes a midpoint, and when the instructions are executed by the processor, further cause the computing device to: add intermediate points along the path on either side of the midpoint; identify a shortest path from each of the intermediate points to the respective first and second points; select a line segment or a curve segment from a plurality of line segments or curve segments connecting the intermediate points based in part on the shortest path from each of the intermediate points to the respective first and second points; form the path from the shortest path, the selected line segment or curve segment, and the shortest path from each of the intermediate points to the respective first and second points.

[0025] In further embodiments of the computing device, when the instructions are executed by the processor, further cause the computing device to: perform debanding on the extravascular image to generate a debanded extravascular image; identify ambient light in the debanded extravascular image; remove the ambient light from the debanded extravascular image to form a light-corrected extravascular image; perform brightness and / or contrast normalization on the light-corrected extravascular image; perform darkening on a centerline of the blood vessel based in part on the debanded extravascular image to form a darkened centerline image; perform gradient transformation on the darkened centerline image to generate the image velocity map.

[0026] In further embodiments of the computing device, when the instructions are executed by the processor, further cause the computing device to: identify the ambient light in the debanded image based on a blur filter with a median diameter between 30 and 120 pixels.

[0027] In some embodiments, the present disclosure can be implemented as a computer readable medium for an extravascular image processing system. The computer readable medium can include instructions that, when executed by a processor of the extravascular image processing system, cause the extravascular image processing system to: receive an extravascular image from an extravascular imaging device, the extravascular image including an indication of a blood vessel; generate, by processing circuitry, an image velocity map based on the extravascular image; receive, by the processing circuitry, an indication of a first point on the extravascular image, the first point corresponding to a portion of the blood vessel; identify, by the processing circuitry, a shortest distance from each of a plurality of pixels on the image to the first point based on the image velocity map; receive, by the processing circuitry, an indication of a second point on the extravascular image, the second point corresponding to another portion of the blood vessel; determine, by the processing circuitry, a blood vessel path based on the second point and the shortest distance from each of the plurality of pixels to the first point.

[0028] In further embodiments of the computer readable medium, the instructions that, when executed by the processor of the extravascular image processing system, cause the extravascular image processing system to: add intermediate points along the path on either side of the midpoint; identify a shortest path from each of the intermediate points to the respective first point and the second point; select a line segment or a curve segment from a plurality of line segments or curve segments connecting the intermediate points based in part on the shortest path from each of the intermediate points to the respective first point and the second point; form the path from the shortest path, the selected line segment or curve segment, and the shortest path from each of the intermediate points to the respective first point and the second point.

[0029] In further embodiments of the computer readable medium, the instructions that, when executed by the processor of the extravascular image processing system, cause the extravascular image processing system to: perform a debanding process on the extravascular image to generate a debanded extravascular image; identify ambient light in the debanded extravascular image; remove the ambient light from the debanded extravascular image to form a light corrected extravascular image; perform a brightness and / or contrast normalization process on the light corrected extravascular image; perform a centerline darkening process on the debanded extravascular image based in part on the centerline of the blood vessel to form a centerline darkened image; perform a gradient transform on the centerline darkened image to generate the image velocity map. BRIEF DESCRIPTION OF DRAWINGS

[0030] For ease of reference to the discussion of any element or action, the most significant digit or digits in a reference number refer to the figure number where that element first appears.

[0031] Figure 1 An extravascular imaging system according to at least one embodiment of the present disclosure is shown.

[0032] Figure 2 A blood vessel path identification system according to at least one embodiment of the present disclosure is shown.

[0033] Figure 3A method of identifying a vascular path is shown in accordance with at least one embodiment of the present disclosure.

[0034] Figure 4A 、 Figure 4B 、 Figure 4C and Figure 4D A vascular path planned on an extravascular image is shown in accordance with at least one embodiment of the present disclosure.

[0035] Figure 5 A method of identifying a vascular path is shown in accordance with at least one embodiment of the present disclosure.

[0036] Figure 6 A method of identifying a vascular path is shown in graphical form in accordance with at least one embodiment of the present disclosure.

[0037] Figure 7 A method of identifying a vascular path is shown in accordance with at least one embodiment of the present disclosure.

[0038] Figure 8 A method of identifying a vascular path is shown in accordance with at least one embodiment of the present disclosure.

[0039] Figure 9 A method of identifying a vascular path is shown in graphical form in accordance with at least one embodiment of the present disclosure.

[0040] Figure 10 A method of identifying a vascular path is shown in accordance with at least one embodiment of the present disclosure.

[0041] Figure 11 A method of identifying a vascular path is shown in graphical form in accordance with at least one embodiment of the present disclosure.

[0042] Figure 12 A computer readable storage medium is shown.

[0043] Figure 13 A schematic diagram of a machine is shown. DETAILED DESCRIPTION

[0044] As previously mentioned, the present disclosure provides systems and techniques for identifying a vascular path on an extravascular image. Accordingly, an exemplary vascular imaging device will now be described. Figure 1A combined internal and external imaging system 100 is shown, which includes an intraluminal imaging system 102 (e.g., an IVUS imaging system) and an extravascular imaging system 104 (e.g., an angiographic imaging system). The combined internal and external imaging system 100 also includes a computing device 106, which contains circuitry, controllers and / or processors, memory, and software configured to perform the vascular imaging and vascular path identification methods described herein. It should be understood that the systems and methods described herein do not require intraluminal imaging, and the combined imaging system is described merely for clarity of presentation. For example, the image recognition techniques described herein for identifying a vascular route or path on extravascular images can be used to register an extravascular image with a series of intravascular or intraluminal images. Typically, the intraluminal imaging system 102 can be used to generate intravascular imaging data (e.g., IVUS images), while the extravascular imaging system 104 can be used to generate extravascular imaging data (e.g., angiographic images).

[0045] The extravascular imaging system 104 can include a table 108 configured to provide sufficient space for positioning an angiography / fluoroscopy unit C-arm 110 relative to a patient 112 on a drive unit in an operating position. The C-arm 110 can be configured to acquire fluoroscopic images without contrast in the patient’s 112 blood vessels, and / or angiographic images with contrast in the patient’s 112 blood vessels.

[0046] Raw radiological image data acquired by the C-arm 110 can be transmitted to an extravascular data input port 114 via a transmission cable 116. The input port 114 can be a standalone component, or can also be integrated into or as part of the computing device 106. The input port 114 can include a processor for converting the received raw radiological image data into extravascular image data (e.g., angiographic / fluoroscopic image data), for example, in the form of real-time video, DICOM, or a series of individual images. The extravascular image data can initially be stored in a memory within the input port 114, or in a memory of the computing device 106. If the input port 114 is a component separate from the computing device 106, the extravascular image data can be transmitted to the computing device 106 via the transmission cable 116 and into an input port (not shown) of the computing device 106. In some alternatives, communication between devices or processors can occur via wireless communication, rather than cable communication as shown.

[0047] Intravascular imaging data can be, for example, IVUS data or OCT data obtained by the intraluminal imaging system 102. The intraluminal imaging system 102 can include an intravascular imaging device, such as an imaging catheter 120. The imaging catheter 120 is configured to be inserted into the patient 112 with its distal end (including a diagnostic assembly or probe 122, such as an IVUS probe) positioned near a desired imaging location of a blood vessel. Radiopaque material or markers 124 located on or near the probe 122 can provide an indication of the current position of the probe 122 in radiographic images. In some embodiments, the imaging catheter 120 and / or the probe 122 can include a guide catheter (not shown) that is inserted into a lumen of a subject (such as a blood vessel, such as a coronary artery) by a guide wire (also not shown). However, in some embodiments, the imaging catheter 120 and / or the probe 122 can be inserted into a blood vessel of the patient 112 without a guide wire.

[0048] In some embodiments, the imaging catheter 120 and / or the probe 122 can have both imaging functionality and other data acquisition functionality. For example, FFR and / or iFR data, data related to pressure, flow, temperature, electrical activity, oxygenation, biochemical composition, or any combination thereof. In some embodiments, the imaging catheter 120 and / or the probe 122 can further include therapeutic devices, such as stents, balloons (such as angioplasty balloons), grafts, filters, valves, and / or different types of therapeutic intraluminal devices.

[0049] The imaging catheter 120 is coupled to a proximal connector 126 to couple the imaging catheter 120 to an image acquisition device 128. The image acquisition device 128 can be coupled to the computing device 106 by a transmission cable 116 or a wireless connection. Intravascular image data can be initially stored in a memory within the image acquisition device 128, or can be stored in a memory of the computing device 106. If the image acquisition device 128 is a separate component from the computing device 106, the intravascular image data can be transmitted to the computing device 106 by, for example, the transmission cable 116.

[0050] The computing device 106 can also include one or more additional output ports for transmitting data to other devices. For example, the computer can include an output port for transmitting data to a data archive or storage device 132. The computing device 106 can also include a user interface (described in detail below) that includes circuitry, processing components, and instructions executable by the processing components and / or circuitry to implement the image recognition and vascular routing or pathfinding described herein, and / or dynamic registration of intravascular images and extravascular images using the recognized vascular paths.

[0051] In some embodiments, the computing device 106 can include a user interface device, such as a keyboard, a mouse, a joystick, a touch screen device (such as a smartphone or tablet), a touchpad, a trackball, a voice command interface, and / or other types of user interfaces known in the art.

[0052] The user interface can be rendered and displayed on a display 134 coupled to the computing device 106 by a display cable 136. Although the display 134 is depicted as being separate from the computing device 106, in some examples, the display 134 can be part of the computing device 106. Alternatively, the display 134 can be remote and connected wirelessly to the computing device 106. As another example, the display 134 can be part of another computing device (such as a tablet) that is different from the computing device 106, which can be coupled to the computing device 106 by a wired or wireless connection. For some applications, the display 134 includes a heads-up display and / or a head-mounted display. For some applications, the computing device 106 generates output on different types of visual, textual, graphical, haptic, audio, and / or video output devices (e.g., speakers, headphones, a smartphone, or a tablet). For some applications, the user interface rendered on the display 134 functions as both an input device and an output device.

[0053] Figure 2 A vascular path identification system 200 according to non-limiting examples of the present disclosure is shown. Generally, the vascular path identification system 200 is a system for identifying a vascular path or route from extravascular images (such as angiographic images, etc.). In some embodiments, the vascular path identification system 200 can be implemented as part of the combined internal and external imaging system 100 of Figure 1 For example, the vascular path identification system 200 includes a computing device 202, which can be the computing device 106 of Figure 1 In the vascular path identification system 200, the computing device 202 is coupled to an imager 204 (such as the C-arm 110, etc.). Likewise, the computing device 202 is coupled to a display 206 (such as the display 134, etc.).

[0054] Generally, the imager 204 can generate information elements or data that include indication information for extravascular images 218. The computing device 202 is communicatively coupled to the imager 204 and can receive data from the imager 204 that includes indication information for the extravascular images 218. Generally, the extravascular images 218 can include pixels that indicate extravascular image contrast and color. This will be described in detail below.

[0055] The computing device 202 can be any of a variety of computing devices. In some embodiments, the computing device 202 can be integrated into and / or implemented by a console of the display 206. In some embodiments, the computing device 202 can be a workstation or server that is communicatively coupled to the imager 204 and / or the display 206. In other embodiments, the computing device 202 can be provided by a cloud-based computing device, such as a compute-as-a-service system that is accessible over a network, such as the Internet, an intranet, a wide area network, etc. The computing device 202 can include a processor 208, a memory 210, input and / or output (I / O) devices 212, and a network interface 214.

[0056] The processor 208 can include circuitry or processor logic, such as any of a variety of commercially available processors. In some examples, the processor 208 can include multiple processors, multi-threaded processors, multi-core processors (whether the multiple cores are on the same chip or separate chips), and / or some other type of multi-processor architecture through which multiple physically separate processors are linked in some fashion. Further, in some examples, the processor 208 can include a graphics processing portion and can include specialized memory, multi-threading, and / or some other parallel processing capability. In some examples, the processor 208 can be an application specific integrated circuit (ASIC) or a field programmable gate array (FPGA).

[0057] The memory 210 can include logic, a portion of which includes an array of integrated circuits that forms a non-volatile memory for persistently storing data or a combination of non-volatile and volatile memory. It should be appreciated that the memory 210 can be based on any of a variety of technologies. In particular, the array of integrated circuits included in the memory 210 can be arranged to form one or more types of memory, such as dynamic random access memory (DRAM), NAND memory, NOR memory, etc.

[0058] The I / O devices 212 can be any of a variety of devices to receive input and / or provide output. For example, the I / O devices 212 can include a keyboard, a mouse, a joystick, a foot pedal, a display, a touch-enabled display, a haptic feedback device, an LED, etc.

[0059] The network interface 214 can include logic and / or features to support a communication interface. For example, the network interface 214 can include one or more interfaces that operate in accordance with various communication protocols or standards to communicate over direct or network communication links. Direct communication can be achieved using communication protocols or standards described in one or more industry standards, including successors and variants thereof. For example, the network interface 214 can facilitate communication over a bus, such as a Peripheral Component Interconnect Express (PCIe), Non-Volatile Memory Express (NVMe), Universal Serial Bus (USB), System Management Bus (SMBus), SAS (e.g., Serial Attached SCSI (SAS)), interface, Serial AT Attachment (SATA) interface, etc. Further, the network interface 214 can include logic and / or features to support communication over a variety of wired or wireless network standards, such as 802.11 communication standards. For example, the network interface 214 can be arranged to support wired communication protocols or standards, such as Ethernet, etc. As another example, the network interface 214 can be arranged to support wireless communication protocols or standards, such as Wi-Fi, Bluetooth, ZigBee, LTE, 5G, etc.

[0060] The memory 210 can include instructions 216, extravascular images 218, image velocity maps 220, A points on images 222, shortest paths to A points 224, B points on images 226, paths between A and B 228, and images with path overlays 230.

[0061] It is important to note that while A points 222 and B points 226 are often referred to individually, the A points on images 222 and the B points on images 226 and the paths between A and B 228 are not single points or paths, but rather individual segments that make up an entire path, which can have multiple segments. Thus, many embodiments of the present disclosure contemplate multiple A points on images 222 and multiple B points on images 226 (see, e.g., Figure 9 and Figure 10 ) and multiple paths, or “segments” of paths. To this end, these elements are depicted in the plural in Figure 2 In some embodiments, the processor 208 can execute the instructions 216 to receive (e.g., from a user via I / O) these points and store the points in the memory 210. Likewise, the memory 210 can store the set of shortest paths to each respective A point 224 and the paths between each adjacent pair of A points on images 222 and B points on images 226 (e.g., in the paths between A and B 228).

[0062] In some embodiments, the processor 208 can execute the instructions 216 to identify the A point 222 on the image and the B point 226 on the image using a machine learning (ML) model. For example, a ML model (not shown) can be trained to infer pixels corresponding to blood vessels from an angiogram image (e.g., using a supervised training algorithm, an unsupervised training algorithm, etc.). Subsequently, the trained ML model can be deployed such that, during operation, the A point 222 on the image and the B point 226 on the image can be inferred from the extravascular image 218.

[0063] Examples of the plurality of points and paths are provided in more detail below in Figure 9 However, for clarity, examples of a plurality of segments of a path are presented here. Figure 9 The shortest paths 916a and 916b are depicted, which are two instances of the path 228 between A and B. That is, the first point 904 and the second point 908 are instances of the A point 222 on the image, while the third point 914 represents two instances of the B point 226 on the image.

[0064] During operation, the processor 208 can execute the instructions 216 to cause the computing device 202 to receive the extravascular image 218 from the imager 204. The processor 208 can further execute the instructions 216 to generate the image velocity map 220 from the extravascular image 218, receive (or infer) an indication of the A point 222 on the image, and generate the shortest path 224 to the A point from all points 222 on the image and the image velocity map 220. Moreover, the processor 208 can execute the instructions 216 to receive (or infer) the B point 226 on the image and identify the path 228 between A and B from the B point 226 on the image along the shortest path 224 to the A point. Furthermore, the processor 208 can execute the instructions 216 to generate the image 230 with the path overlay and cause the image 230 with the path overlay to be displayed on the display 206.

[0065] Figure 3 A logic flow 300 that can be implemented by a computing device coupled to an extravascular imaging device in accordance with at least one embodiment of the present disclosure is shown. The logic flow 300 can be implemented by the computing device 106 of the combined internal and external imaging system 100 of Figure 1 or the computing device 202 of the vascular path identification system 200 of Figure 2 It should be noted that, for clarity, the logic flow 300 is described with reference to the computing device 202 of the vascular path identification system 200 of Figure 2 and the computing device 106 of the combined internal and external imaging system 100 of Figure 4A to Figure 4D is described. Figure 4A to Figure 4D An extravascular image and points indicated on the extravascular image, and a path of a blood vessel represented in the extravascular image are shown. This is described in detail below.

[0066] The logic flow 300 may begin at block 302. At block 302, "at the processing circuitry, an extravascular image containing indication information of blood vessels is received from the extravascular imaging device," meaning the extravascular image containing indication information of blood vessels can be received at the processing circuitry of the computing device. For example, processor 208 may execute instruction 216 to receive an extravascular image 218 from imager 204 (such as C-arm 110, etc.), where the extravascular image 218 is an extravascular image (such as an angiographic image, fluorescence image, etc.). For example, the extravascular image 218 may be similar to... Figure 4A An extravascular image 400 is depicted. As shown, the extravascular image 400 includes a representation of a blood vessel 402. In some embodiments, the processor 208 is executable with instructions 216 to display the extravascular image 218 as part of a graphical user interface (GUI) 232 on the display 206. In some embodiments, the processor 208 is executable with instructions 216 to provide the GUI 232, thereby allowing a user (e.g., a physician) to view angiographic images (such as the extravascular image 218), add, move, and / or delete points, and view the shape of the blood vessel (e.g., a path) in real time as points are added, moved, or deleted.

[0067] Continuing to box 304, “Generation of an Image Velocity Map Based on the Extravascular Image by Processing Circuitry,” the processing circuitry can generate an image velocity map. For example, processor 208 can execute instruction 216 to generate an image velocity map 220 from the extravascular image 218. Examples of image velocity maps are described below (e.g., Figure 5 and Figure 6 However, typically, image velocity diagram 220 includes a copy of extravascular image 218, wherein the contrast of the image has been adjusted to clearly distinguish the blood vessels represented in extravascular image 218 from other parts of extravascular image 218.

[0068] Continuing to box 306, “Receiving indication information of a first point on the extravascular image via processing circuitry, the first point corresponding to a portion of the blood vessel,” the processing circuitry may receive a point on the extravascular image corresponding to a location on (or a portion of) the blood vessel. For example, processor 208 may execute instruction 216 to receive point A 222 on the image (e.g., received from a user via an input and / or output (I / O) device 212, etc.). In some embodiments, a physician may use a touchscreen, mouse, joystick, etc., to designate point A 222 (e.g., the proximal end point) on the image of the blood vessel on the extravascular image 218. Figure 4B An extravascular image 400 and a blood vessel 402 are shown, wherein a point 404 (such as point A 222 on the image) on the extravascular image 400 is marked on the blood vessel 402. In some embodiments, the processor 208 may execute instructions 216 to cause the GUI 232 to include an indication of an extravascular image 218 with point A 222 marked on the image.

[0069] Continuing to block 308, "identify, by the processing circuit, a shortest distance between each of the plurality of pixels of the extravascular image and the first point based on the image velocity map," a shortest distance between the point received at block 306 and the plurality of pixels of the extravascular image can be determined. It should be understood that the term "shortest distance" as used herein does not refer to the shortest straight-line distance, but rather the shortest path or shortest distance along a path or vessel.

[0070] For example, in some embodiments, the processor 208 can execute the instructions 216 to identify a shortest path between the point A 222 on the image and a plurality of other pixels of the extravascular image 218. In some embodiments, the processor 208 executes the instructions 216 to identify, based on the image velocity map 220, a path between the point A 222 on the image and each pixel of a vessel (e.g., the vessel 402, etc.) associated with the extravascular image 218. In some embodiments, the processor 208 executes the instructions 216 to identify, based on the image velocity map 220, a shortest path 224 to the point A between the point A 222 on the image and each pixel (e.g., the vessel 402, etc.) of the extravascular image 218. In some embodiments, the processor 208 can execute the instructions 216 to identify the path based on a pathfinding algorithm (e.g., Dijkstra's algorithm, etc.). In these embodiments, the processor 208 can execute the instructions 216 to derive the path using the pathfinding algorithm based on the image velocity map 220 with a priority queue (e.g., a Fibonacci heap, etc.).

[0071] In some embodiments, block 308 can include (e.g., as a precursor block, or implicitly embodied in block 308) converting the image velocity map 220 to a cost map (not shown). The cost map can be used to identify a shortest distance between each of the plurality of pixels of the extravascular image and the first point. For example, a pathfinding algorithm (e.g., Dijkstra's algorithm, etc.) can use the cost map to identify a path with the lowest cost as the shortest path. It should be understood that a graph as used herein is a mathematical graph consisting of nodes and edges. In some embodiments, each node represents a pixel in the velocity map, and each edge represents a connection between adjacent pixels and a related cost of moving between adjacent pixels. The edge cost value is based on the velocity value of the traversed pixel and the length of the edge. The number of edges (e.g., the number of neighbors) for each node (e.g., pixel) can vary, but is typically 4, 8, 16, 24, 32, etc.

[0072] Continuing to block 310, "receive, by the processing circuit, indication information of a second point on the extravascular image, the second point corresponding to another portion of the blood vessel," the processing circuit can receive a second point on the extravascular image that corresponds to a location on (or portion of) the blood vessel. For example, the processor 208 can execute the instructions 216 to receive the B point 226 on the image (e.g., from a user via the input and / or output (I / O) device 212 based on an inference using a ML model, etc.). In some embodiments, a physician can use a touchscreen, mouse, joystick, etc. to specify the shortest path 224 (e.g., distal end point) to the A point on the extravascular image 218. Figure 4C The extravascular image 400 and the blood vessel 402 are shown with the point 404 on the blood vessel 402 (e.g., the A point 222 on the image) and the point 406 (e.g., the B point 226 on the image) marked on the extravascular image 400. In some embodiments, the processor 208 can execute the instructions 216 to cause the GUI 232 to include an indication of the extravascular image 218 with the A point 222 marked on the image and the B point 226 on the image.

[0073] Continuing to block 312, "determine, by the processing circuit, a blood vessel path between the first point and the second point based on the shortest distance of each of the plurality of pixels to the first point and the second point," based on the path identified in block 308, a path between the point received in block 306 and the point received in block 310 can be determined. For example, the processor 208 can execute the instructions 216 to identify the path from the shortest path 224 to the A point associated with the B point 226 on the image and generate an image with a path overlay 230 from the extravascular image 218 and the path 228 between A and B. Figure 4C The extravascular image 400 and the blood vessel 402 are shown with the point 404 on the blood vessel 402 (e.g., the A point 222 on the image) and the point 406 (e.g., the B point 226 on the image) marked on the extravascular image 400. In some embodiments, the processor 208 can execute the instructions 216 to cause the GUI 232 to include an indication of the extravascular image 218 with the A point 222 marked on the image and the B point 226 on the image.

[0074] The logical flow 300 can optionally continue to block 314 "smooth, by processing circuitry, the determined blood vessel path," in which the blood vessel path determined at block 312 can be smoothed. For example, the processor 208 can execute the instructions 216 to smooth the path (e.g., to remove jagged portions, to reduce excess length, etc.). In some embodiments, the processor 208 can execute the instructions 216 to smooth the path 228 between A and B based on a path smoothing algorithm, and generate an image 230 with path overlay from the extravascular image 218 and the smoothed path 228 between A and B. Figure 4D An extravascular image 400 is shown, in which a blood vessel 402, a point 404, a point 406, and a smoothed path 410 are marked on the extravascular image 400. In some examples, the processor 208 can execute the instructions 216 to smooth the path based on a Douglas-Peuker algorithm, based on a path sampling algorithm (e.g., sampling every n-th coordinate (e.g., 4th, 5th, 6th, 7th, etc.), based on a spline simplification algorithm, etc.

[0075] Figure 5 A logical flow 500 is shown, in accordance with at least one embodiment of the present disclosure, which can be implemented by a computing device coupled to an extravascular imaging device to generate an image velocity map. In some embodiments, the logical flow 300 can implement the logical flow 500 as part of block 304. The logical flow 500 can be implemented by Figure 1 the computing device 106 of the combined internal and external imaging system 100 of Figure 2 the computing device 202 of the blood vessel path identification system 200 of Figure 2 the computing device 202 of the blood vessel path identification system 200 of Figure 6 the computing device 202 of the blood vessel path identification system 200 of Figure 6 A series of extravascular images is shown, in accordance with at least one embodiment of the present disclosure, which are transformed to form an image velocity map.

[0076] In some embodiments, the processor 208 can execute the instructions 216 to implement the logical flow 500 "in the background" while a user (e.g., a physician) analyzes the GUI 232 depicting the extravascular image 218. The logical flow 500 can begin at block 502. At block 502 "de-speckle, by processing circuitry, an extravascular image containing blood vessel indicating information," the extravascular image can be de-speckled. For example, the processor 208 can execute the instructions 216 to de-speckle the extravascular image 218, resulting in a de-speckled extravascular image. As previously described, Figure 6 A series of extravascular images is shown, which illustrate the generation of the image velocity map 220 from the extravascular image 218. To this end, Figure 6A destigmatized image 602 formed from an extravascular image 218 is shown. Furthermore, as shown, the extravascular image 218 includes indications of blood vessels 402. In some embodiments, the processor 208 may execute instructions 216 to apply a Gaussian filter to the extravascular image 218 to form the destigmatized image 602. For example, the processor 208 may execute instructions 216 to apply a 3-pixel × 3-pixel Gaussian kernel to the extravascular image 218 to form the destigmatized image 602.

[0077] Continuing to box 504, “Identifying Ambient Light in Despiked External Image via Processing Circuitry,” ambient light in the despiked image can be identified. For example, processor 208 can execute instruction 216 to identify ambient light in despiked image 602 based on a blur filter (such as a 30-120 pixel diameter median filter, etc.). Figure 6 An ambient light 604 is shown in the extravascular image, illustrating the ambient light detected or identified from the destigmatized image 602. Continuing to block 506, “Removal of Ambient Light from the Destigmatized Extravascular Image by Processing Circuitry,” the identified ambient light can be removed from the destigmatized image. For example, processor 208 can execute instruction 216 to remove the ambient light 604 from the destigmatized image 602, thereby forming a photocorrected extravascular image 606. Figure 6 An optically corrected extravascular image 606 is shown, formed from ambient light 604 of a despotted image 602 and an extravascular image.

[0078] In some other embodiments, the blur filter may be in the form of a geometric shape (such as a circle) that is applied to the entire image. In other embodiments, multiple filters may be provided, with different filters applied to (or across) different regions of the image. For example, a first filter may be applied to the boundaries or edges of the image to remove any shadows or dark borders, after which another filter may be applied to the entire image as described above.

[0079] Continuing to box 508, "Normalizing the brightness and / or contrast of the optically corrected extravascular image by processing circuitry," the brightness and / or contrast of the optically corrected extravascular image can be normalized. For example, processor 208 can execute instruction 216 to normalize the brightness and / or ambient light of the optically corrected extravascular image 606 to form a normalized extravascular image 608a. In some embodiments, processor 208 can execute instruction 216 to apply an image normalization algorithm to the optically corrected extravascular image 606 to form a normalized extravascular image 608a. Figure 6 A normalized extravascular image 608a formed from an optically corrected extravascular image 606 is shown.

[0080] Continuing to block 510, "darkening, by processing circuitry, a centerline of the blood vessels in the normalized extravascular image," the centerline of the blood vessels in the normalized extravascular image can be darkened to form a centerline-darkened extravascular image. For example, the processor 208 can execute the instructions 216 to generate the centerline-darkened extravascular image 612 from the normalized extravascular image 608a by darkening the portions of the blood vessels represented in the image 608a. Generally, the portions of the blood vessels 402 are darkened progressively, with the centerline of the blood vessels 402 being darkened at a higher rate than the edges of the blood vessels 402. For clarity, this will be described in detail below with reference to the region 610. However, it will be appreciated that the entire image 608a is processed as described to darken the portions of the blood vessels 402. The processor 208 can execute the instructions 216 to apply a series of filters (e.g., a 3-pixel diameter minimum filter, a 3-pixel diameter maximum filter, a 5-pixel diameter maximum filter, and a 5-pixel diameter minimum filter, etc.) to the normalized extravascular image 608a to progressively darken regions further from the edges of the blood vessels to darken the centerline of the blood vessels. Figure 6 A region 610 on the normalized extravascular image 608a and the centerline-darkened extravascular image 612 is shown, as well as a series of filters applied to the normalized extravascular image 608a highlighted by the region 610. This will be described in detail below with reference to FIG. 6B. Figure 7

[0081] Continuing to block 512, "generating, by processing circuitry, an image velocity map from the centerline-darkened extravascular image based on a gradient transform," an image velocity map can be generated from the centerline-darkened extravascular image based on a gradient transform. For example, the processor 208 can execute the instructions 216 to apply a transform (e.g., a gradient transform, a linear transform, a Sigmoid transform, etc.) to the centerline-darkened extravascular image 612 to form the image velocity map 220. In some embodiments, the processor 208 can execute the instructions 216 to apply a Sigmoid function (e.g., Sigmoid(X-115.2) / 22) to the centerline-darkened extravascular image 612 to form the image velocity map 220. In other embodiments, the processor 208 can execute the instructions 216 to apply a linear function saturated to a minimum and a maximum to the centerline-darkened extravascular image 612 to form the image velocity map 220.

[0082] Figure 7 A logic flow 700 is shown for applying a mask to a normalized extravascular image to darken a centerline of one or more blood vessels in the normalized extravascular image. The logic flow 700 can be implemented by a computing device coupled to an extravascular imaging device, in accordance with at least one embodiment of the present disclosure. In some embodiments, the logic flow 500 can implement the logic flow 700 as part of block 510. The logic flow 700 can be implemented by the​Figure 1 The computing device 106 of the combined internal and external imaging system 100, or Figure 2 The computing device 202 implements the vascular path recognition system 200. It should be noted that, for clarity, reference is made to... Figure 2 The computing device 202 of the vascular path recognition system 200 and Figure 6 To describe the logical flow 700. As mentioned before. Figure 6 A series of extravascular images according to at least one embodiment of the present disclosure are shown, which are transformed to form an image velocity map.

[0083] Logic flow 700 may begin at block 702. In block 702, “Applying multiple noise removal filters to a normalized extravascular image containing vascular indication information via processing circuitry to form a destigmatized normalized extravascular image,” multiple noise removal filters may be applied to the normalized extravascular image to form a destigmatized normalized extravascular image. For example, processor 208 may execute instruction 216 to apply filters (such as filters 614a, 614b, 614c, and / or filter 614d) to the normalized extravascular image 608a, thereby forming a destigmatized normalized extravascular image 608b. In some embodiments, filters 614a, 614b, 614c, and 614d may include morphological opening and closing filters. As a specific example, processor 208 may execute instruction 216 to sequentially apply a morphological opening filter to dark pixels, and then apply a morphological closing filter to dark pixels. In some embodiments, processor 208 may execute instruction 216 to apply a morphological opening filter for dark pixels using a 3-pixel × 3-pixel circular structuring element, first applying a minimum filter (such as filter 614a) and then applying a maximum filter (such as filter 614b). Furthermore, in some embodiments, processor 208 may execute instruction 216 to apply a morphological closing filter for dark pixels using a 5-pixel × 5-pixel circular structuring element, first applying a maximum filter (such as filter 614c) and then applying a minimum filter (such as filter 614d).

[0084] Continuing to block 704, "identify, by the processing circuit, a plurality of dark vessel candidate regions from the speckle-normalized extravascular image," a plurality of dark vessel candidate regions can be identified from the speckle-normalized extravascular image. For example, the processor 208 can execute the instructions 216 to identify the dark vessel candidate region 616a, the dark vessel candidate region 616b, the dark vessel candidate region 616c, the dark vessel candidate region 616d, the dark vessel candidate region 616e, and / or the dark vessel candidate region 616f from the speckle-normalized extravascular image 608b by darkening a region of at least d pixels inside the blood vessels by s amount. The processor 208 can execute the instructions 216 to identify the dark vessel candidate region 616a by darkening a region of at least d pixels inside the blood vessels by multiplying the pixel values by s amount, where d = 1 and s = 0.31. The processor 208 can execute the instructions 216 to identify the dark vessel candidate region 616b by darkening a region of at least d pixels inside the blood vessels by multiplying the pixel values by s amount, where d = 2 and s = 0.40. The processor 208 can execute the instructions 216 to identify the dark vessel candidate region 616c by darkening a region of at least d pixels inside the blood vessels by multiplying the pixel values by s amount, where d = 3 and s = 0.48. The processor 208 can execute the instructions 216 to identify the dark vessel candidate region 616d by darkening a region of at least d pixels inside the blood vessels by multiplying the pixel values by s amount, where d = 4 and s = 0.54. The processor 208 can execute the instructions 216 to identify the dark vessel candidate region 616e by darkening a region of at least d pixels inside the blood vessels by multiplying the pixel values by s amount, where d = 5 and s = 0.60. The processor 208 can execute the instructions 216 to identify the dark vessel candidate region 616a by darkening a region of at least d pixels inside the blood vessels by multiplying the pixel values by s amount, where d = 6 and s = 0.65.

[0085] In some embodiments, the processor 208 can execute the instructions 216 to identify the dark vessel candidate regions (e.g., the dark vessel candidate region 616a, 616b, etc.) based on: • generating a fine vessel image (not shown) by performing morphological dilation of a maximum filter on the speckle-normalized extravascular image 608b using a w*h circular structuring element, where w = 2d + 1 and h = 2d + 1; • computing a soft vessel probability image (not shown) from the fine vessel image by applying a sigmoid-like function f(x) to each pixel of the fine vessel image, where f(x) = {0, if x <= 103.68; 1, if x >= 126.72; (x - 103.68) / 23.04, if 103.68 < x < 126.72}; • compute a darker fine vessel image (not shown) from the fine vessel image by multiplying each pixel value by s; and • compute dark vessel candidate regions (e.g., dark vessel candidate regions 616a, 616b, etc.) by alpha blending the normalized extravascular image 608a and the darker fine vessel image using the soft vessel probability image as the alpha channel, where alpha = 0 represents 100% pixel values of the fine vessel image and alpha = 1 represents 100% pixel values of the normalized extravascular image 608a.

[0086] Continuing to block 706, "select, by the processing circuit, one of the plurality of dark vessel candidate regions," one of the plurality of dark vessel candidate regions (e.g., debanded normalized extravascular image 608b, dark vessel candidate regions 616a, 616b, etc.) can be selected. For example, the processor 208 can execute the instructions 216 to select one of the dark vessel candidate regions (e.g., debanded normalized extravascular image 608b, dark vessel candidate regions 616a, 616b, etc.) having the smallest (e.g., darkest) pixel.

[0087] Continuing to block 708, "generate, by the processing circuit, a centerline darkened extravascular image from the normalized extravascular image and the selected one of the plurality of dark vessel candidate regions," the centerline darkened extravascular image can be generated from the normalized extravascular image and the selected one of the dark vessel candidate regions. It should be appreciated that one of the dark vessel candidate regions can be selected for each masked region or segment of the image. Thus, the logic flow 700 can be iteratively performed (as described above with respect to the region 610) across the normalized extravascular image 608a.

[0088] Returning to block 704, "generate, by the processing circuit, a plurality of dark vessel candidate regions from the normalized extravascular image," the plurality of dark vessel candidate regions can be generated from the normalized extravascular image. For example, the processor 208 can execute the instructions 216 to generate the plurality of dark vessel candidate regions (e.g., debanded normalized extravascular image 608b, dark vessel candidate regions 616a, 616b, etc.) from the normalized extravascular image 608a. Figure 2 As previously described, the memory 210 stores the shortest path 224 to the A point, which includes an indication of the shortest path between the A point 222 on the image and a plurality of other pixels (or points) of the extravascular image 218. In some embodiments, the shortest path 224 to the A point can be a list referencing each determined shortest path (e.g., between the A point and pixel 1, between the A point and pixel 2, between the A point and pixel 3, etc.). Thus, during operation, if the user moves, adds, or deletes a point (e.g., the user moves the B point, etc.), the processor 208 can execute the instructions 216 to update (e.g., in real-time as the points are moved, added, or deleted) the path 228 between the A and B points. Figure 8 A logic flow 800 that can be implemented by a computing device coupled to an extravascular imaging device to display a path and / or update a path considered herein is shown. The logic flow 800 can be implemented by Figure 1 the computing device 106 of the combined internal and external imaging system 100 of Figure 2 the computing device 202 of the vessel path identification system 200 of It should be noted that, for clarity, reference is made to theFigure 2 The computing device 202 of the vascular path recognition system 200 and Figure 9 Describe the logical flow 800. Figure 9 A series of extravascular images are shown, which illustrate points and paths between points, which can be similar to the points and paths displayed on the extravascular images (such as extravascular image 218) and shown to the user in a GUI (such as GUI 232).

[0089] Logic flow 800 may begin with logic flow 300. That is, logic flow 800 may include logic flow 300 as a first sub-flow. As previously described, logic flow 300 may be implemented to receive points on an extravascular image and determine the paths between the points. Figure 9 A series of images 902a to 902h are shown, highlighting points and the paths between them. It should be noted that these images may be extravascular images. However, for clarity, depictions or indications of blood vessels and vascular structures have been omitted so that this disclosure can focus on the points and the paths between them. As previously described, logic flow 300 includes block 306, which receives points on the images. Figure 9 Image 902a is shown, illustrating the first point 904 received and marked on the image. Logic flow 300 further includes block 308, where all paths between the point and other points on the image are identified (or derived using shortest path algorithms, etc.). For example, Figure 9 Image 902b is shown, illustrating a first point 904 and the shortest path 906 between the first point 904 and other points in the image. Logic flow 300 further includes blocks 310 and 312, wherein a second point on the image is received and the shortest path between the first point and the second point is identified. Figure 9 Image 902c is shown, illustrating a second point 908 received and marked on the image, and the shortest path 910 drawn on the image between the first point 904 and the second point 908. As previously described, the shortest path 910 can be retrieved from the previously derived shortest path 224 to point A. Therefore, the shortest path 910 can be drawn and marked on the image in real time.

[0090] The logical flow 800 can continue from the logical flow 300 to block 802. At block 802, "identify, by the processing circuit, a shortest distance between each of the plurality of pixels on the extravascular image and the second point based on the image velocity map," the processing circuit can determine the shortest distance between the second point and the plurality of pixels of the extravascular image. For example, in some embodiments, the processor 208 can execute the instructions 216 to identify the shortest path between the B point 226 on the image and the plurality of other pixels of the extravascular image 218. In some embodiments, the processor 208 executes the instructions 216 to identify the path between the B point 226 on the image and each pixel of the extravascular image 218 associated with a blood vessel (e.g., the blood vessel 402, etc.) based on the image velocity map 220. Figure 9 The image 902d is shown, showing the first point 904 and the second point 908, as well as the shortest path 906 to the first point and the shortest path 912 to the second point.

[0091] It should be appreciated that once the shortest path 906 to the first point and the shortest path 912 to the second point are derived, the shortest path 910 can be updated in real-time based on movement of the first point 904 or the second point 908. Further, adding a point between the first point 904 or the second point 908 can result in the shortest path 910 being updated in real-time.

[0092] The logical flow 800 can continue to block 804, "receive, by the processing circuit, an indication of a third point on the extravascular image, the third point corresponding to a portion of the blood vessel," the processing circuit can receive a point on the extravascular image that corresponds to a location on (or portion of) the blood vessel. For example, the processor 208 can execute the instructions 216 to receive the third point 914 (e.g., from a user via the input and / or output (I / O) device 212, based on an inference using an ML model, etc.) and display the shortest paths from the third point 914 to the first point 904 and the second point 908. Figure 9 The image 902e is shown, showing the third point 914, the updated shortest path 916a as the shortest path from the third point 914 to the first point 904, and the updated shortest path 916b as the shortest path between the third point 914 and the second point 908. The combined path of the updated shortest path 916a and the updated shortest path 916b forms the shortest path from the first point 904 to the second point 908 and through the third point 914. In this way, a user (e.g., a physician, etc.) can add a point to the extravascular image (e.g., the extravascular image 218) and have the path updated in real-time to account for the added point.

[0093] Continuing to block 806, "identify, by processing circuitry, a shortest distance between each of the plurality of pixels on the extravascular image and the third point based on the image velocity map," a shortest distance between the third point and the plurality of pixels of the extravascular image can be determined. For example, in some embodiments, the processor 208 can execute the instructions 216 to identify a shortest path between the third point 914 and a plurality of other pixels of the image. Figure 9 An image 902f is shown, showing the shortest path 918 to the third point.

[0094] Continuing to block 808, "receive, by processing circuitry, indication information to move a location of the second point on the extravascular image and update the shortest path," indication information to move a location of the second point on the image can be received. For example, the processor 208 can execute the instructions 216 to receive indication information (e.g., from a user via the I / O device 212, based on an inference using the ML model, etc.) to move a location of a point (e.g., the second point 908, etc.) on the image and update the shortest path in real-time. For example, Figure 9 An image 902g is shown, showing the second point 908 moved to a moved second point 920, and the updated shortest path 916b updated to an updated shortest path 922. The processor 208 can execute the instructions 216 to determine the updated shortest path 922 from the shortest path 918 to the third point and the moved second point 920.

[0095] Continuing to block 810, "identify, by processing circuitry, a shortest distance between each of the plurality of pixels on the extravascular image and the moved second point based on the image velocity map," a shortest distance between the moved second point and the plurality of pixels of the extravascular image can be determined. For example, in some embodiments, the processor 208 can execute the instructions 216 to identify a shortest path between the moved second point 920 and a plurality of other pixels of the image. Figure 9 An image 902h is shown, showing the shortest path 924 to the moved second point.

[0096] Accordingly, the logic flow 800 can be implemented to provide points that can be added, deleted, moved, etc., and update paths between the points in real-time, and display to a user on a GUI (e.g., the GUI 232).

[0097] Figure 10 A logic flow 1000 that can be implemented by a computing device coupled to an extravascular imaging device to smooth a path around an intermediate point considered herein is shown. The logic flow 1000 can be implemented by Figure 1 the computing device 106 of the combined internal and external imaging system 100 of Figure 2 the computing device 202 of the vascular path identification system 200 of Figure 2the computing device 202 of the blood vessel path identification system 200 and Figure 11 The logic flow 1000 is described. Figure 11 A series of extravascular images are shown, showing points, paths between points, which can be similar to the points and paths shown on an extravascular image (such as the extravascular image 218) and displayed to a user on a GUI (such as the GUI 232).

[0098] The logic flow 1000 can begin at block 1002“by processing circuitry, adding two intermediate points around a midpoint of the path,” where two intermediate points are added around a midpoint of the path. For example, the processor 208 can execute the instructions 216 to add an intermediate point on either side of the midpoint of the path. In some embodiments, the processor 208 executes the instructions 216 to add an intermediate point at a specified number of pixels (such as between 5 and 20, 12 pixels, etc.) from the midpoint along the planned path. Figure 11 An image 1102a is shown, showing a proximal point 1104 and a distal point 1106, with a path 1108 planned between the proximal point 1104 and the distal point 1106 through a midpoint 1110. Further, Figure 11 An image 1102b is shown, showing the intermediate point 1112a and the intermediate point 1112b added on either side of the midpoint 1110 along the path 1108.

[0099] Continuing to block 1004“by processing circuitry, identifying paths from the intermediate points to the respective end points,” paths from the intermediate points to the respective end points can be identified. For example, the processor 208 can execute the instructions 216 to identify a path from each intermediate point to the respective end point based on the previously determined shortest path for each end point (such as the shortest path to A point 224, etc.). Figure 11 An image 1102b is shown, showing the intermediate point 1112a and the intermediate point 1112b added around the path 1108, as well as the shortest path 1114a planned between the intermediate point 1112a and its respective end point, and the shortest path 1114b planned between the intermediate point 1112b and its respective end point.

[0100] Continuing to block 1006“by processing circuitry, selecting a line segment or a curve segment through the midpoint based on the shortest paths from the intermediate points and a series of line segments or curve segments,” a line segment or a curve segment can be identified based on the shortest paths from the intermediate points and a series of line segments or curve segments. For example, the processor 208 can execute the instructions 216 to identify a line segment or a curve segment from the series of line segments or curve segments based on the paths planned from the intermediate points. In some embodiments, the processor 208 can execute the instructions 216 to identify the line segment or the curve segment based on minimizing the cost of the entire path. For example, Figure 11An image 1102c is shown, showing a series of line segments or curve segments 1116 that pass through the midpoint 1108. The processor 208 can execute the instructions 216 to evaluate a plurality of such line segments or curve segments and select the line segment or curve segment with the lowest total path cost (e.g., selected path = min(C P + C M + C D ), where C M is the line segment or curve segment that passes through the midpoint). In some embodiments, the processor 208 can execute the instructions 216 to evaluate a series of line segments or curve segments 1116 that include line segments near the midpoint 1110 (to allow for inaccurately placed points) and / or curved line segments or curve segments (to follow curved blood vessels).

[0101] Continuing to block 1008, "form complete path through midpoint by processing circuitry from intermediate path and selected line segment or curve segment," a complete path that passes through the midpoint is formed from the path planned from the intermediate point and the selected line segment or curve segment. For example, the processor 208 can execute the instructions 216 to form a complete path from the path planned from the intermediate midpoint at block 1004 and the selected line segment or curve segment at block 1006. Figure 11 An image 1102d is shown, showing a smooth path 1118 formed from the intermediate point 1112a, the intermediate point 1112b, and the line segment or curve segment selected from the line segments or curve segments 1116.

[0102] Figure 12 A computer-readable storage medium 1200 is shown. The computer-readable storage medium 1200 can include any non-transitory computer-readable storage medium or machine-readable storage medium, such as an optical, magnetic or semiconductor storage medium. In various embodiments, the computer-readable storage medium 1200 can include an article of manufacture. In some embodiments, the computer-readable storage medium 1200 can store computer-executable instructions 1202 that can be executed by a circuit (such as the computing device 106, the processor 208, etc.). For example, the computer-executable instructions 1202 can include instructions to implement the operations described with respect to the logic flow 300, the logic flow 500, the logic flow 700, the logic flow 800, and / or the logic flow 1000. Examples of a computer-readable storage medium 1200 or machine- readable storage medium can include any tangible media capable of storing electronic data, including volatile memory or non-volatile memory, removable or non-removable memory, erasable or non-erasable memory, writeable or re-writeable memory, and so forth. Examples of computer-executable instructions 1202 can include any suitable type of code, such as source code, compiled code, interpreted code, executable code, static code, dynamic code, object-oriented code, visual code, and the like.

[0103] Figure 13A schematic diagram of a machine 1300 is shown in the example form of a computer system, within which a set of instructions 1308 (e.g., software, a program, an application, an applet, an app, or the like) can be executed to cause the machine 1300 to perform any one or more of the methodologies discussed herein. For example, the instructions 1308 can cause the machine 1300 to perform the method of Figure 13 A schematic diagram of a machine 1300 is shown in the example form of a computer system, within which a set of instructions 1308 (e.g., software, a program, an application, an applet, an app, or the like) can be executed to cause the machine 1300 to perform any one or more of the methodologies discussed herein. For example, the instructions 1308 can cause the machine 1300 to perform the method of Figure 2 the instructions 216 of FIG. 2, Figure 3 the logic flow 300 of FIG. 3, Figure 5 the logic flow 500, the logic flow 700, Figure 8 the logic flow 800, and / or Figure 10 the logic flow 1000 of FIG. 10, etc. More generally, the instructions 1308 can cause the machine 1300 to plan a vascular path on an extravascular image based on at least two points indicated on the image.

[0104] The instructions 1308 transform a general, non-programmed machine 1300 into a particular machine 1300 programmed to carry out the described and illustrated functions in the manner described. In alternative embodiments, the machine 1300 can operate as a standalone device or can be coupled (e.g., networked) to other machines. In a networked deployment, the machine 1300 can operate in the capacity of a server machine or a client machine in server-client network environments, or

[0105] Machine 1300 can include processors 1302, memory 1304, and I / O components 1342, which can be configured to communicate with one another by way of bus 1344, in examples. In an example embodiment, processors 1302, such as central processing units (CPUs), reduced instruction set computing (RISC) processors, complex instruction set computing (CISC) processors, graphics processing units (GPUs), digital signal processors (DSPs), application-specific integrated circuits (ASICs), radio-frequency integrated circuits (RFICs), another processor, or combination thereof, can include, for example, processor 1306 and processor 1310, which can execute instructions 1308. The term “processor” is intended to include multiple processors, which can be of the same type or different types. In some embodiments, the multiple processors can be configured to execute instructions 1308 in parallel. Although FIG. 13 illustrates a single bus 1344 coupling processors 1302 to memory 1304 and I / O components 1342, processors 1302 can be configured to communicate directly with one another via, for example, a crossbar switch. Figure 13 Although FIG. 13 illustrates a single bus 1344 coupling processors 1302 to memory 1304 and I / O components 1342, processors 1302 can be configured to communicate directly with one another via, for example, a crossbar switch. Further, although FIG. 13 illustrates a single bus 1344 as the sole means of

[0106] Memory 1304 can include main memory 1312, static memory 1314, and storage unit 1316, each coupled to the processors 1302 by, for example, bus 1344. The main memory 1304, static memory 1314, and storage unit 1316 can store instructions 1308 embodying any one or more of the methodologies or functions described herein. The instructions 1308 can also reside, completely or

[0107] I / O components 1342 can include a wide variety of components to receive input, provide output, produce output, transmit information, exchange information, capture measurements, and so on. The specific I / O components 1342 that are included in a machine will depend on the type of machine. For example, portable machines (e.g., mobile phones) might include a touch input device or other such input mechanisms, while a headless server machine will likely not include such a touch input device. It will be appreciated that the I / O components 1342 can include many other components that are not specifically shown, enabled, or described in FIG. 13. As examples, the I / O components 1342 can include battery or other power source, universal serial bus (USB) port, audio Figure 13The I / O components 1342 are merely exemplary in nature and are not limited to the components described. Furthermore, one or more components of the I / O components 1342 can be utilized in various embodiments. For example, the I / O components 1342 can include output components 1328 and input components 1330. The output components 1328 can include visual components (e.g., a display such as a plasma display panel (PDP), a light-emitting diode (LED) display, a liquid crystal display (LCD), a projector, or a cathode ray tube (CRT)), acoustic components (e.g., speakers), haptic components (e.g., a vibratory motor, resistance mechanisms), other signal generators, and so forth. The input components 1330 can include alphanumeric input components (e.g., a keyboard, a touchscreen configured to receive alphanumeric input, a photo-optical keyboard, or other alphanumeric input components), point-based input components (e.g., a mouse, a touchpad, a trackball, a joystick, a motion sensor, or other pointing devices), tactile input components (e.g., a physical button, a touch screen that provides location and / or force of touches or touch gestures, or other tactile input components), audio input components (e.g., a microphone), and the like.

[0108] In further example embodiments, the I / O components 1342 can include biometric components 1332, motion components 1334, environmental components 1336, or position components 1338, among a myriad of other components. For example, the biometric components 1332 can include components to detect expressions (e.g., hand expressions, facial expressions, vocal expressions, body gestures, or eye tracking), measure biosignals (e.g., blood pressure, heart rate, body temperature, perspiration, or brain waves), identify a person (e.g., voice identification, retinal identification, facial identification, fingerprint identification, or electroencephalogram-based identification), and the like. The motion components 1334 can include acceleration sensor components (e.g., accelerometer), gravitation sensor components, rotation sensor components (e.g., gyroscope), and so forth. The environmental components 1336 can include, for example, illumination sensor components (e.g., photometer), temperature sensor components (e.g., one or more thermometers that detect temperature of a

[0109] Communication can be implemented using a wide variety of technologies. The I / O components 1342 can include communication components 1340 operable to couple the machine 1300 to a network 1320 or to devices 1322 via coupling 1324 and coupling 1326, respectively. For example, the communication components 1340 can include a network interface component or other suitable components to interface to the network 1320. In further examples, the communication components 1340 can include wired communication components, wireless communication components, cellular communication components, Near Field Communication (NFC) components, Bluetooth® components (e.g., Bluetooth® low energy), Wi-Fi® components, and other components to provide connectivity to other devices by other components (e.g., as described in more detail below). components (e.g., as described in more detail below). components, and other communication components to provide for communication via other modalities. The devices 1322 can be another machine or any of a wide assortment of peripheral devices (e.g., a peripheral device coupled via a USB).

[0110] Moreover, the communication components 1340 can detect identifiers or include components operable to detect identifiers. For example, the communication components 1340 can include radio frequency identification (RFID) tag reader components, NFC smart tag detection components, optical reader components (e.g., an optical sensor to detect one- dimensional bar codes such as Universal Product Code (UPC) bar codes, multi-dimensional bar codes such as Quick Response (QR) codes, Aztec codes, Data Matrix, Dataglyph, MaxiCode, PDF417, Ultra Code, UCC RSS-2D bar codes, and other optical codes), or acoustic detection components (e.g., microphones to identify tagged audio signals). In addition, a variety of information can be derived via the communication components 1340, such as location via Internet Protocol (IP) geolocation, location via Wi-Fi® signal triangulation, location via detecting an NFC beacon signal that can indicate a particular location, and so forth.

[0111] The various memories (i.e., the memory 1304, the main memory 1312, the static memory 1314, and / or the memory of the processor 1302 and / or the storage unit 1316) can store one or more sets of instructions and data structures (e.g., software) embodying or utilized by any one or more of the methodologies or functions described herein. These instructions (e.g., instructions 1308), when executed by the processor 1302, cause various operations to implement the disclosed embodiments.

[0112] ​As used herein, the terms “machine storage medium,” “device storage medium,” and “computer storage medium” have the same meaning and are used interchangeably in this disclosure. These terms refer to one or more storage devices and / or media (such as centralized or distributed databases, and / or associated caches and servers) that store executable instructions and / or data. These terms should therefore be understood to include, but are not limited to, solid-state memory, optical media, and magnetic media, including memory internal or external to a processor. Specific examples of machine storage media, computer storage media, and / or device storage media include non-volatile memory, such as semiconductor storage devices (e.g., erasable programmable read-only memory (EPROM), electrically erasable programmable read-only memory (EEPROM), FPGAs, and flash memory devices); magnetic disks (e.g., internal hard disks and removable disks); magneto-optical disks; and CD-ROM and DVD-ROM disks. The terms “machine storage medium,” “computer storage medium,” and “device storage medium” expressly exclude carrier waves, modulated data signals, and other such media, at least some of which are covered by the term “signal medium” discussed below.

[0113] In various example embodiments, one or more portions of network 1320 may be an ad hoc network, intranet, extranet, virtual private network (VPN), local area network (LAN), wireless local area network (WLAN), wide area network (WAN), global wide area network (WWAN), metropolitan area network (MAN), the Internet, a portion of the Internet, a portion of the public switched telephone network (PSTN), a common old-style telephone service (POTS) network, a cellular telephone network, a wireless network, etc. A network, another type of network, or a combination of two or more such networks. For example, network 1320 or a portion thereof may include a wireless or cellular network, and coupling 1324 may be a Code Division Multiple Access (CDMA) connection, a Global System for Mobile Communications (GSM) connection, or another type of cellular or wireless coupling. In this example, coupling 1324 may implement any of a variety of data transmission technologies, such as Single Carrier Radio Transmission (1xRTT), Evolved Data Optimization (EVDO), General Packet Radio Service (GPRS), Enhanced Data Rate Evolution of GSM (EDGE), 3rd Generation Partnership Project (3GPP) (including 3G), 4th Generation Wireless (4G) networks, Universal Mobile Telecommunications System (UMTS), High-Speed ​​Packet Access (HSPA), Global Microwave Access Interoperability (WiMAX), Long Term Evolution (LTE) standards, other standards defined by various standards-setting organizations, other long-distance protocols, or other data transmission technologies.

[0114] The instructions 1308 can be transmitted or received over the network 1320 using a transmission medium via a network interface device (e.g., a network interface component included in the communication components 1340) and utilizing any one of a number of well- known transfer protocols (e.g., hypertext transfer protocol (HTTP)). Similarly, the instructions 1308 can be transmitted or received using a transmission medium via the coupling 1326 (e.g., a peer-to-peer coupling) on the device 1322. The term “transmission medium” and “signal medium” shall be taken to include any intangible medium that is capable of storing, encoding, or carrying the instructions 1308 for execution by the machine 1300, and includes digital or analog communications signals or other intangible media to facilitate communication of such software. Accordingly, the terms “transmission medium” and “signal medium” shall be taken to include any form of a modulated data signal, carrier wave, and so forth. The term “modulated data signal” means a signal that has one or more of its characteristics set or changed in such a manner as to encode information in the signal.

[0115] The terms used herein should be interpreted according to their ordinary meaning in the relevant technical field, the meaning indicated by the context, or if a specific definition is provided.

[0116] Reference throughout this specification to “one embodiment” or “an embodiment” does not necessarily refer to the same embodiment, although they can. Unless otherwise explicitly indicated, the words “including,” “comprising,” and the like are to be construed in an inclusive sense, as opposed to an exclusive or exhaustive sense; that is to say, in the sense of “including but not limited to.” Words using the singular or plural number also include the plural or singular number respectively, unless expressly limited to one or the other. Further, the words “herein,” “above,” “below,” and words of similar import, refer to this application as a whole and not to any particular portion thereof. When the word “or” is used in reference to a list of two or more items, that word covers all of the following interpretations of the word: any of the items in the list, all of the items in the list, and any combination of the items in the list, unless expressly limited to one or another. Any term not expressly defined herein has its ordinary meaning as understood by one of ordinary skill in the art relevant to the relevant art.

[0117] By using a real anatomical model, a more accurate surgical plan can be made compared to by statistical modeling.

Claims

1. A computer-implemented method comprising: receiving, by processing circuitry, an extravascular image from an extravascular imaging device, the extravascular image containing an indication of a blood vessel; generating, by the processing circuitry, an image velocity map based on the extravascular image; receiving, by the processing circuitry, an indication of a first point on the extravascular image, the first point corresponding to a portion of the blood vessel; identifying, by the processing circuitry, a shortest distance from each of a plurality of pixels on the image to the first point based on the image velocity map; receiving, by the processing circuitry, an indication of a second point on the extravascular image, the second point corresponding to another portion of the blood vessel; and determining, by the processing circuitry, a path of the blood vessel based on the second point and the shortest distance from each of the plurality of pixels to the first point.

2. The computer-implemented method of claim 1, further comprising smoothing the path.

3. The computer-implemented method of claim 2, wherein the path includes midpoints, and smoothing the path comprises: adding intermediate points along the path on either side of the midpoints; identifying a shortest path from each of the intermediate points to the respective first and second points; selecting a line or curve segment from a plurality of line or curve segments connecting the intermediate points based in part on the shortest path of each of the intermediate points to the respective first and second points; and forming a path from the shortest paths, the selected line or curve segment, and the shortest path of each of the intermediate points to the respective first and second points.

4. The computer-implemented method of claim 1, generating, by the processing circuitry, the image velocity map comprises: de-speckling the extravascular image to generate a de-speckled extravascular image; brightness and / or contrast normalizing the de-speckled extravascular image to generate a normalized extravascular image; and darkening a centerline of the blood vessel based in part on the de-speckled extravascular image to generate the image velocity map.

5. The computer-implemented method of claim 4, generating, by the processing circuitry, the image velocity map further comprises: identifying ambient light in the de-speckled extravascular image; and removing the ambient light from the de-speckled extravascular image to form a light-corrected extravascular image, wherein the normalized extravascular image is generated based on the light-corrected extravascular image.

6. The computer-implemented method of claim 5, further comprising identifying ambient light in the de-speckled image based on a blur filter having a median diameter between 30 and 120 pixels.

7. The computer-implemented method of claim 4, further comprising applying a Gaussian kernel to the extravascular image to de-speckle the extravascular image.

8. The computer-implemented method of claim 7, wherein the Gaussian kernel is a 3 pixel by 3 pixel diameter Gaussian kernel. ​ ​ ​ 9. The computer-implemented method of claim 4, further comprising iteratively applying a mask to portions of the normalized image to progressively darken pixels corresponding to portions of the normalized image representing the blood vessel based on a distance of the pixels to a blood vessel boundary.

10. The computer-implemented method of claim 4, further comprising applying a gradient transform to the centerline darkened image to generate the blood vessel velocity map.

11. The computer-implemented method of claim 10, wherein the gradient transform is a Sigmoid transform or a linear transform.

12. The computer-implemented method of claim 1, further comprising: identifying, by the processing circuit, a shortest distance of each of a plurality of pixels on the image to the second point based on the image velocity map; receiving an indication to move a location of the first point; identifying an updated path of the blood vessel based on the moved first point and the shortest distance of each of the plurality of pixels to the second point.

13. The computer-implemented method of claim 1, further comprising: identifying, by the processing circuit, a shortest distance of each of a plurality of pixels on the image to the second point based on the image velocity map; receiving an indication of a midpoint on the extravascular image; and identifying an updated path of the blood vessel based on the midpoint and the shortest distance of each of the plurality of pixels to the first point and the shortest distance of each of the plurality of pixels to the second point.

14. A computing device coupled to an angiography machine, the computing device comprising a processor and a memory coupled to the processor, the memory containing instructions that, when executed by the processor, cause the computing device to perform the method of any of claims 1-13.

15. A computer-readable medium containing instructions that, when executed by a processor of a medical imaging device, cause the medical imaging device to perform the method of any of claims 1-13.

Images