System and method for measuring a vessel in a body

Abstract

Methods and systems are provided for measuring a blood vessel in a body. In one example, a computer obtains at least one angiography image frame. A computer identifies pixels in at least one angiography image frame that correspond to a blood vessel of a body. The computer further identifies a medial axis of the blood vessel and determines a measurement of the blood vessel in the at least one angiographic image frame by counting pixels in a lateral direction corresponding to the medial axis of the blood vessel and converting a size in pixel to a size in size.

Description

Cross - Reference to Related Applications

[0001] This application claims priority to U.S. Provisional Patent Application No. 63 / 429,606, filed on Dec. 2, 2022, the disclosure of which is incorporated herein by reference in its entirety. Technical Field

[0002] The present disclosure generally relates to angiography, and more particularly to systems and methods for measuring the dimensions of blood vessels in angiographic images. Background Art

[0003] In coronary angiography, a catheter is advanced into a coronary artery, a radiopaque contrast agent is injected via the catheter, and a series of x-ray images are obtained by fluoroscopy. The pattern filled with the radiopaque contrast agent on the obtained images reflects the anatomy of the coronary artery. If there is a stenosis in the coronary artery that may impede blood flow into the coronary muscle, the image will reflect that stenosis of the given blood vessel. Timely and accurate identification of coronary artery stenosis has medical value; for example, coronary artery stenosis can inform interventional decisions, such as the placement of a coronary stent. Current techniques for measuring coronary artery stenosis rely on visual assessment of angiographic images and may not be accurate. Summary of the Invention

[0004] Embodiments of the present invention relate to methods, systems, and computer-readable media for measuring blood vessels in angiographic images of a body. Briefly, embodiments of the present invention can provide automatic, rapid, and accurate measurement of one or more blood vessels in an angiographic image frame by identifying pixels corresponding to one or more blood vessels in the angiographic image frame, locating the medial axis of the one or more blood vessels, and measuring dimensions transverse to the medial axis of the one or more blood vessels. The techniques described herein can be applied to any angiographic image of a body, including but not limited to angiographic images obtained at a frequency faster than the heart rate (e.g., at a frame rate faster than the subject's heart rate), and perform spatio-temporal reconstruction on the angiographic image (e.g., using wavelet transform and / or a trained neural network).

[0005] In an example embodiment, at least one angiographic image frame is obtained, and pixels corresponding to blood vessels in the at least one angiographic frame are identified by assigning a probability value (also referred to herein as "p-value") regarding whether it is a blood vessel to each pixel in the angiographic image. In an example embodiment, the p-value is assigned based on the intensity or luminance of the pixel. In an example embodiment, the p-value can range from 0.0 (not a blood vessel) to 1.0 (blood vessel). A probability threshold (e.g., 0.5) can be set to determine whether a pixel represents a blood vessel, and the p-value of each pixel can be compared with the threshold to determine whether the pixel represents a blood vessel. For example, if the p-value of a pixel exceeds the threshold, it can be determined that the pixel represents a blood vessel. In an example embodiment, each pixel in the image is characterized as representing a blood vessel or not representing a blood vessel, and the result of the characterization of each pixel (i.e., blood vessel or non-blood vessel) can be stored in a matrix, in a specific column and row of the matrix. In an example embodiment, a neural network can be used to determine whether a pixel is a blood vessel, and the neural network is trained to identify pixels corresponding to blood vessels in angiographic images. Information from the neural network can be used to generate an angiographic image in which blood vessels and other structures (such as angiographic catheters) are segmented. The term "segmentation" as used herein can refer to manually, semi-automatically, or fully automatically identifying and representing (e.g., displaying) blood vessel elements in an angiogram. "Segmentation" of blood vessels can refer to processing an image such that the blood vessel structure is represented as distinct from noise and other structures in the image field of view.

[0006] In an example embodiment, a medial axis skeletonization technique can be used to identify the medial axis of one or more blood vessels, where, for each pixel determined to be a blood vessel, it is determined whether the pixel is located along the medial axis of the blood vessel (i.e., the central longitudinal axis). Pixels determined to be located along the medial axis of the blood vessel are referred to herein as medial axis pixels. Determining whether an interested pixel is a medial axis pixel can include drawing or defining a circle (also referred to herein as a "query circle") around the interested pixel, and determining whether any pixel intersecting the query circle is a non-blood vessel pixel. If not, the diameter of the query circle is iteratively increased until the circle boundary includes one or more pixels not in the segmented blood vessel (i.e., one or more non-blood vessel pixels). If only one non-blood vessel pixel intersects the query circle, the interested pixel can be considered not to be a medial axis pixel. If two non-blood vessel pixels intersect the query circle, the interested pixel can be considered to be located along the medial axis of the blood vessel (and the diameter of the circle is considered the blood vessel diameter). If three or more non-blood vessel pixels intersect the query circle, the interested pixel can be considered a branch point pixel (e.g., where one blood vessel branches into two or more blood vessels). If the query circle fails to intersect any non-blood vessel pixels, a larger query circle can be drawn around the interested pixel, and the process can be repeated until at least one of the pixels intersecting the query circle is a non-blood vessel pixel. In an example embodiment, the first query circle can include a group of pixels around the interested pixel, and any subsequent query circle can include a group of pixels around the previous query circle. A list of all medial axis pixels in the segmented blood vessel can be generated, which corresponds to a list of pixels equidistant from the blood vessel edge. The coordinates of the medial axis pixels (e.g., rows and columns) can be used to define the medial axis of the blood vessel, which is essentially a skeletonized representation of the blood vessel (e.g., where each segment of the skeleton is only one pixel wide). Additionally, the corresponding diameter of the query circle intersecting two non-blood vessel pixels around each medial axis pixel can be considered the diameter of the blood vessel at that point along the medial axis of the blood vessel. Thus, the medial axis analysis embodiment can produce the medial axis and diameter at each point in each segmented blood vessel.

[0007] In an example embodiment, a measurement of the blood vessel diameter can be obtained by superimposing the medial axis of the blood vessel (i.e., the skeleton) onto a segmented blood vessel image (i.e., an image in which the blood vessel elements of an angiogram have been identified and represented (e.g., displayed)). By counting the blood vessel pixels in a lateral direction relative to the medial axis (i.e., perpendicular to the medial axis at the medial axis pixels), the blood vessel diameter in pixels can be obtained at each medial axis pixel in the skeleton.

[0008] In an example embodiment, pixel size calibration may be performed to convert the number of pixels across the width (diameter) of a blood vessel into units of dimension or size (e.g., millimeters). In an example embodiment, pixel size calibration may be performed by segmenting an angiography catheter in an angiography image, performing skeletonization using a medial axis technique, and determining the number of pixels across the width (diameter) of the catheter. The diameter of the catheter is known (e.g., from the FDA package insert of the catheter manufacturer) and can be used to determine the image magnification in pixels per unit size (e.g., pixels per millimeter). The image magnification can then be used as a conversion factor to convert the blood vessel diameter in pixels to a blood vessel diameter in units of size (such as millimeters).

[0009] In one form, a method is provided. The method includes: obtaining, at a computer, at least one angiography image frame; identifying, at the computer, pixels in the at least one angiography image frame that correspond to a blood vessel of a body; performing, at the computer, an identification of a medial axis of the blood vessel; and determining, at the computer, a measurement of the blood vessel in a direction transverse to the medial axis of the blood vessel in the at least one angiography image frame.

[0010] In one example, identifying pixels that correspond to a blood vessel includes: measuring the brightness or intensity of each pixel in the at least one angiography image frame, classifying the intensity of each pixel as high intensity or low intensity based on a threshold, identifying edges in the at least one angiography image frame based on the difference in intensity between adjacent pixels, identifying pixels having a high-intensity signal between two edges as a structure; and identifying pixels corresponding to the structure as pixels corresponding to the blood vessel. For example, the intensity of each pixel may be represented by a floating point number ranging from 0.0 to 1.0.

[0011] In another example, identifying pixels that correspond to a blood vessel includes assigning to each pixel in the at least one angiography image a probability value (p-value) as to whether it is a blood vessel. In one example, the p-value is assigned based on the brightness of the pixel. In one example, a probability threshold (e.g., 0.5) may be set to determine whether a pixel represents a blood vessel, and the p-value of each pixel may be compared to the threshold to determine whether the pixel represents a blood vessel. In one example, if the p-value of a pixel exceeds the threshold, the pixel may be determined to be part of a blood vessel.

[0012] In a further example, a series of angiography image frames spanning at least one cardiac cycle is obtained, and the method further includes: determining, based on measurements of the blood vessel in each angiography image frame in the series of angiography image frames, a measurement of the blood vessel in the at least one angiography image frame corresponding to a target time in the at least one cardiac cycle for which the blood vessel is measured.

[0013] In one example, identifying the medial axis of a blood vessel includes determining at a computer which pixels identified as blood vessel pixels are located along the medial axis of the blood vessel. In one example, determining whether a blood vessel pixel is located along the medial axis of the blood vessel includes drawing a query circle around the blood vessel pixel at the computer and determining at the computer whether any pixel intersecting the query circle is a non-blood vessel pixel. In one example, if (a) the query circle intersects only one non-blood vessel pixel, the blood vessel pixel at the center of the circle is considered not to be a medial axis pixel; (b) the query circle intersects two non-blood vessel pixels, the blood vessel pixel at the center of the circle is considered to be located along the medial axis of the blood vessel; or (c) three or more non-blood vessel pixels intersect the query circle, the blood vessel pixel at the center of the circle is considered to be a branch point pixel. In one example, if the query circle fails to intersect any non-blood vessel pixels, a larger query circle is drawn around the pixel of interest at the computer, and the process is repeated until at least one of the pixels intersecting the query circle is a non-blood vessel pixel.

[0014] In one example, identifying the medial axis of a blood vessel further includes obtaining a measurement of the blood vessel diameter at the computer by superimposing the medial axis of the blood vessel (i.e., the skeletonized representation of the blood vessel) onto a version of the angiographic image (e.g., which can be a segmented image such as a spatio-temporal reconstruction version of the angiographic image where the blood vessel and other structures have been segmented, or even an unsegmented image of appropriate quality). In one example, obtaining a measurement of the blood vessel diameter includes counting all blood vessel pixels across the width of the blood vessel transverse to the medial axis at the computer.

[0015] In one example, determining a measurement of a blood vessel in at least one angiographic image includes performing pixel size calibration at the computer to convert the number of pixels across the blood vessel width into a size dimension. In one example, performing pixel size calibration includes identifying a catheter in the angiographic image at the computer; determining the diameter of the catheter in pixels at the computer; retrieving the diameter of the catheter in size units (e.g., millimeters) at the computer; and determining the conversion factor between pixel units and size units at the computer. The diameter of the catheter in size units (e.g., millimeters) can be determined based on the conversion factor between pixel units and size units.

[0016] In one example, the method further includes transforming at least one angiographic image frame at the computer by segmenting the blood vessel using machine learning techniques.

[0017] In yet another example, the method further includes determining a plurality of diameter measurements along the length of a blood vessel in an angiographic image at the computer. In one example, the method further includes presenting a heat map that indicates a plurality of measurements of a blood vessel in at least one angiographic image frame.

[0018] In another example, the method further includes comparing, on a computer, the diameter of at least a portion of a blood vessel in at least one angiographic image frame with a reference measurement of the blood vessel. In an additional example, the reference measurement is the diameter of a second portion of the blood vessel that is upstream of at least a portion of the blood vessel. In yet another additional example, the method further includes: when the measured diameter of at least a portion of the blood vessel is less than a predetermined percentage of the diameter of the second portion of the blood vessel, then displaying a visual indication of stenosis at the computer adjacent to the at least a portion of the blood vessel. For example, a common guideline for stenosis determined by the American Heart Association is a stenosis of 70% or greater, that is, when the blood vessel diameter is 30% or less of the previous diameter of the blood vessel subsequent to the previous branch point.

[0019] In yet another example, the method further includes displaying, at the computer, a graphical user interface that shows one or more icons representing stents. In one example, one or more stent icons are selected for display at the computer based on the suitability of the one or more stent icons for treating the stenosis indicated in at least one angiographic image frame and / or the availability of the one or more stent icons in inventory. In one example, the graphical user interface displays information about the one or more stents. In yet another example, the method further includes moving, at the computer, the selected stent icon onto the angiographic image in response to a user input. In a further example, the method further includes displaying, at the computer, the selected stent icon at the location where the blood vessel segment is placed in the correct scale relative to the blood vessel segment. In yet a further example, the method further includes: receiving, at the computer, a user selection of a stent; and sending, at the computer, information about the selected stent to an inventory and / or billing system.

[0020] In yet another example, the method further includes obtaining, at the computer, a plurality of angiographic images of the body at a frame rate faster than the heart rate of the patient (i.e., heart rate), and performing a spatio-temporal reconstruction of the angiographic images. In a further example, the spatio-temporal reconstruction can be performed at the computer using a wavelet transform (e.g., as described in U.S. Application No. 16 / 784,073, filed on February 6, 2020, which is incorporated herein by reference in its entirety) and / or a neural network trained to detect spatio-temporal characteristics of contrast agent-containing blood vessel structures. For example, a neural network can be trained to estimate the blood vessel structure in a given image from a target image and from one or more temporally adjacent images.

[0021] In another form, a system is provided. The system includes: a communication interface configured to obtain at least one angiographic image frame; and one or more processors coupled to the communication interface, wherein the one or more processors are configured to: identify pixels in the at least one angiographic image frame corresponding to blood vessels of a body; identify a medial axis of the blood vessels; and determine a measurement of the blood vessels in a direction transverse to the medial axis of the blood vessels in the at least one angiographic image frame.

[0022] In another form, one or more non-transitory computer-readable storage media are provided. The one or more non-transitory computer-readable storage media are encoded with instructions that, when executed by a processor, cause the processor to: obtain at least one angiographic image frame; identify pixels in the at least one angiographic image frame corresponding to blood vessels of a body; identify a medial axis of the blood vessels; and determine a measurement of the blood vessels in a direction transverse to the medial axis of the blood vessels in the at least one angiographic image frame.

[0023] Other objectives and advantages of these techniques will be apparent from the specification and the drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0024] Figure 1A and Figure 1B are a side view and a partial schematic view, respectively, showing examples of a rotational x-ray system that can be used with embodiments of the present disclosure to acquire angiographic data.

[0025] Figure 2 is a schematic view of a computer system or information processing device that can be used with embodiments of the present disclosure.

[0026] Figure 3 is a perspective view of a pulse oximeter coupled to a multi-parameter patient monitor and sensors that can be used with aspects of the present disclosure to acquire cardiac signals according to an example embodiment of the present disclosure.

[0027] Figure 4 is a block diagram of an electrocardiogram (EKG) device that can be used with aspects of the present disclosure to acquire cardiac signals according to an example embodiment of the present disclosure.

[0028] Figure 5A shows an angiographic image of a plurality of blood vessels in a body according to an example embodiment of the present disclosure.

[0029] Figure 5B shows, according to an example embodiment of the present disclosure, a segmented angiographic image derived from the Figure 5A angiographic image.

[0030] Figure 5CA method for identifying the medial axis of a blood vessel according to an exemplary embodiment of the present disclosure is shown, whereby it is determined whether pixels corresponding to the blood vessel are located along the medial axis of the blood vessel, or not along the medial axis, or adjacent to a branch point of the blood vessel.

[0031] Figure 5D A skeletonized angiography image derived from a segmented angiography image of Figure 5B using a medial axis skeletonization method according to an exemplary embodiment of the present disclosure is shown.

[0032] Figure 5E An overlay of the skeletonized angiography image of Figure 5C according to an exemplary embodiment of the present disclosure and the Figure 5B segmented angiography image of

[0033] Figure 5F A segmentation of a catheter in an angiography image according to an exemplary embodiment of the present disclosure is shown.

[0034] Figure 5G A heat map indicating the size of a catheter in an angiography image according to an exemplary embodiment of the present disclosure is shown.

[0035] Figure 5H A heat map indicating a measurement of the size of a blood vessel in a body according to an exemplary embodiment of the present disclosure is shown.

[0036] Figure 6 A graphical user interface of an angiography system according to an exemplary embodiment of the present disclosure is shown.

[0037] Figure 7 is a flowchart showing a method for measuring a blood vessel in a body according to an exemplary embodiment of the present disclosure.

[0038] Figure 8 is a flowchart showing a method for identifying pixels corresponding to a structure (such as a blood vessel or a catheter in a body) in at least one angiography image frame according to an exemplary embodiment of the present disclosure.

[0039] Figure 9 is a flowchart showing a method for identifying the medial axis of a blood vessel in a body according to an exemplary embodiment of the present disclosure.

[0040] Figure 10 is a flowchart showing a method for measuring the diameter of a blood vessel in a body according to an exemplary embodiment of the present disclosure.

[0041] Figure 11 is a flowchart showing a method for converting a diameter in pixels to a diameter in size according to an exemplary embodiment of the present disclosure. Detailed Description

[0042] This document describes techniques for measuring vessels (e.g., blood vessels) within a body. The blood vessels can be arteries or veins and can have any suitable location within the body. In one particular example, the blood vessel can be a coronary artery. The body can be a mammalian body, such as a human body. In one example, a blood vessel can be captured in an angiographic image frame, the angiographic image can be segmented, the medial axis of the blood vessel can be identified, and a measurement of the blood vessel can be obtained by measuring dimensions transverse to the medial axis of the blood vessel.

[0043] Reference Figure 1A , Figure 1B and Figure 2 , illustrate exemplary systems or devices that can be used to perform embodiments of the present invention. It should be understood that such systems and devices are merely examples of representative systems and devices, and other hardware and software configurations are also suitable for use with embodiments of the present invention. Thus, the embodiments are not intended to be limited to the specific systems and devices shown herein, and it should be recognized that other suitable systems and devices can also be employed without departing from the spirit and scope of the subject matter provided herein.

[0044] First referring to Figure 1A and Figure 1B , a rotational x-ray system 28 is illustrated, which can be used (such as via fluoroscopic angiography) to obtain angiograms at a rate faster than the heart rate. When obtaining an angiogram, a contrast agent can be injected into a patient located between the x-ray source and the detector, and the x-ray projections are captured by the x-ray detector as two-dimensional images (i.e., angiographic image frames). A sequence of such image frames constitutes an angiographic study, and in some embodiments of the present invention, the angiographic image frames can be acquired at a frequency faster than the cardiac frequency to facilitate blood vessel measurement.

[0045] As Figure 1A shown, an example of an angiographic imaging system is illustrated in the form of a rotational x-ray system 28, which includes a gantry having a C-arm 30 that carries an x-ray source assembly 32 at one of its ends and an x-ray detector array assembly 34 at the other of its ends. The gantry enables the x-ray source assembly 32 and the x-ray detector array assembly 34 to be oriented at different positions and angles around a patient placed on a table 36 while providing access to the patient for a doctor. The gantry includes a base 38 that has horizontal legs 40 extending beneath the table 36 and vertical legs 42 extending upwardly at the ends of the horizontal legs 40 spaced from the table 36. A support arm 44 is rotatably fixed to the upper end of the vertical leg 42 for rotation about a horizontal pivot 46.

[0046] The horizontal pivot 46 is aligned with the centerline of the table 36, and the support arm 44 extends radially outward from the horizontal pivot 46 to support the C-arm drive assembly 47 at its outer end. The C-arm 30 is slidably fixed to the C-arm drive assembly 47 and is coupled to a drive motor (not shown) that slides the C-arm 30 to rotate about the C-axis 48, as indicated by the arrow 50. The horizontal pivot 46 and the C-axis 48 intersect each other at the system isocenter 56 located above the table 36 and are perpendicular to each other.

[0047] The x-ray source assembly 32 is mounted to one end of the C-arm 30, and the x-ray detector array assembly 34 is mounted to the other end. The x-ray source assembly 32 emits x-rays that are directed to the x-ray detector array assembly 34. Both assemblies 32 and 34 extend radially inwardly towards the horizontal pivot 46 such that the central ray of the beam passes through the system isocenter 56. Thus, during acquisition of x-ray attenuation data from a subject placed on the table 36, the central ray of the beam can be rotated about the horizontal pivot 46 or the C-axis 48 or both about the system isocenter.

[0048] The x-ray source assembly 32 includes an x-ray source that emits an x-ray beam when powered on. The central ray passes through the system isocenter 56 and impinges on the two-dimensional flat panel digital detector 58 housed in the x-ray detector array assembly 34. The two-dimensional flat panel digital detector 58 can be, for example, a 2048×2048 element two-dimensional array of detector elements. Each element generates an electrical signal representative of the intensity of the impinging x-rays, and thus representative of the attenuation of the x-rays as they pass through the patient. During scanning, the x-ray source assembly 32 and the x-ray detector array assembly 34 rotate about the system isocenter 56 to acquire x-ray attenuation projection data from different angles. In an example device, the detector array may be capable of acquiring a given number (e.g., up to fifty) of projections or image frames per second. The rate of image frames per second can determine how many image frames can be obtained for a specified scan path and speed.

[0049] Reference Figure 1B , the rotation of assemblies 32 and 34 and the operation of the x-ray source are managed by the control mechanism 60 of the x-ray system. The control mechanism 60 includes an x-ray controller 62 that supplies power and timing signals to the x-ray source assembly 32. The data acquisition system (DAS) 64 in the control mechanism 60 samples data from the detector elements and passes the data to the image reconstructor 65. The image reconstructor 65 receives the digitized x-ray data from the DAS 64 and performs high-speed image reconstruction according to the methods of the present disclosure. The reconstructed images are used as input to a computer 66 that stores the images in a mass storage device 69 or further processes the images. The image reconstructor 65 can be a stand-alone computer or can be integrated with the computer 66.

[0050] The control mechanism 60 further includes a gantry motor controller 67 and a C-axis motor controller 68. In response to motion commands from the computer 66, the motor controllers 67 and 68 supply power to the motors in the x-ray system, and the motors respectively produce rotations about the horizontal pivot 46 and the C-axis 48. The computer 66 also receives commands and scan parameters from the operator via a console 70 having a keyboard and other manually operable controls. An associated display 72 allows the operator to observe the reconstructed image frames and other data from the computer 66. The commands provided by the operator are used by the computer 66 under the guidance of a stored program to provide control signals and information to the DAS 64, the x-ray controller 62, and the motor controllers 67 and 68. In addition, the computer 66 operates a table motor controller 74, and the table motor controller 74 controls the electric table 36 to position the patient relative to the system isocenter 56.

[0051] Now referring to Figure 2 , a block diagram of a computer system or information processing device 80 (e.g., Figure 1B the image reconstructor 65 and / or the computer 66 in Figure 1A and Figure 1B the rotating x-ray system 28) is shown. The computer system or information processing device 80 can be incorporated into an angiographic imaging system (such as

[0052] The computer system 84 may include familiar computer components such as one or more central processing units (CPUs), memory or storage devices, a graphics processing unit (GPU), a communication system, interface cards, etc. As Figure 2As shown, the computer system 84 may include one or more processors 86 that communicate with a plurality of peripheral devices via a bus subsystem 88. The one or more processors 86 may include commercially available central processing units and the like. The bus subsystem 88 may include mechanisms for enabling the various components and subsystems of the computer system 84 to communicate with each other as expected. Although the bus subsystem 88 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple bus subsystems. Peripheral devices that communicate with the one or more processors 86 may include a memory subsystem 90, a disk subsystem 92, a user output device 94, a user input device 96, a communication interface 98, and the like.

[0053] The one or more processors 86 may be implemented using one or more analog and / or digital electrical or electronic components and may include microprocessors, microcontrollers, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic, and / or other analog and / or digital circuit elements configured to perform the various functions described herein, such as by executing instructions stored in the memory subsystem 90 and / or the disk subsystem 92 or another computer program product.

[0054] The memory subsystem 90 and the disk subsystem 92 are examples of physical (non-transitory) storage media configured to store data. The memory subsystem 90 may include multiple memories, including random access memory (RAM) for volatile storage of program code, instructions, and data during program execution, and read-only memory (ROM) for storing fixed program code, instructions, and data. The disk subsystem 92 may include multiple file storage systems that provide permanent (non-volatile) storage for programs and data. Other types of physical storage media include floppy disks, external hard drives, optical storage media (such as compact disc read-only memory (CD-ROM), digital video disc (DVD), and barcodes), semiconductor memories such as flash memory, read-only memories (ROMs), battery-backed volatile memories, network storage devices, and the like. The memory subsystem 90 and the disk subsystem 92 may be configured to store programming and data constructs that provide the functions or features of the technologies discussed herein. Software code modules and / or processor instructions that, when executed by the one or more processors 86, implement or otherwise provide functionality may be stored in the memory subsystem 90 and the disk subsystem 92. The memory subsystem 90 may be a non-transitory computer-readable storage medium.

[0055] The user input device 96 may include hardware and / or software elements configured to receive input from a user for processing by components of the computer system 80. The user input device may include all possible types of devices and mechanisms for inputting information into the computer system 84. These may include keyboards, keypads, touchscreens, touch interfaces incorporated into displays, audio input devices (such as microphones and speech recognition systems), and / or other types of input devices. In various embodiments, the user input device 96 may include a computer mouse, trackball, touchpad, joystick, wireless remote control, graphics tablet, voice command system, eye tracking system, etc. In some embodiments, the user input device 96 is configured to allow a user to select objects, icons, text, etc. that may appear on the monitor 82 or otherwise interact with the objects, icons, text, etc. that may appear on the monitor 82 via commands, motions, or gestures (such as clicking a button, etc.).

[0056] The user output device 94 may include hardware and / or software elements configured to output information from components of the computer system 80 to a user. The user output device may include all possible types of devices and mechanisms for outputting information from the computer system 84. These may include displays (e.g., the monitor 82), printers, touch or force feedback devices, audio output devices, etc.

[0057] The communication interface 98 may include hardware and / or software elements configured to provide one-way or two-way communication with other devices. For example, the communication interface 98 may provide an interface between the computer system 84 and other communication networks and devices (such as via an Internet connection).

[0058] In one example, the information processing device 80 may obtain a series of angiographic image frames (e.g., angiograms) at a rate faster than the heart rate (e.g., via the communication interface 98). For example, the information processing device 80 may use wavelet angiography (e.g., by employing a complex-valued wavelet transform) to generate a spatio-temporal reconstruction of the cardiac frequency phenomenon in angiograms obtained at a frequency faster than the heart rate. The spatio-temporal reconstruction may be performed according to the techniques described in U.S. Patent No. 10,123,761, issued on November 13, 2018, which is incorporated herein by reference in its entirety.

[0059] In addition to acquiring angiographic images, additional cardiac signals / data may be acquired simultaneously for use as cross-correlation targets. For example, the cardiac signals / data may be used as a reference cardiac signal for phase-indexed pixels in angiographic projections. Figure 3 and Figure 4 An exemplary device is shown for acquiring / providing a reference cardiac signal using such devices / systems in the form of a pulse oximetry system and / or an electrocardiogram (EKG) system or device.

[0060] Figure 3 is a perspective view of an example of a suitable pulse oximetry system 100 including a sensor 102 and a pulse oximeter monitor 104. The sensor 102 includes a transmitter 106 for emitting light of a specific wavelength toward patient tissue and a detector 108 for detecting the light after it has been reflected and / or absorbed by the patient's tissue. The monitor 104 may be capable of calculating physiological characteristics related to the light emission and detection received from the sensor 102. Further, the monitor 104 includes a display 110 capable of displaying the physiological characteristics and / or other information about the system. The sensor 102 is shown communicatively coupled to the monitor 104 via a cable 112, but alternatively may be communicatively coupled via a wireless transmission device or the like.

[0061] In the illustrated embodiment, the pulse oximetry system 100 further includes a multi-parameter patient monitor 114. In addition to, or alternatively to, the monitor 104, the multi-parameter patient monitor 114 may be capable of calculating physiological characteristics and providing a central display 116 for information from the monitor 104 and from other medical monitoring devices or systems. For example, the multi-parameter patient monitor 114 may display the patient's SpO2 and pulse rate information from the monitor 104 and blood pressure from a blood pressure monitor on the display 116.

[0062] In an embodiment, a computer system 80 may be configured to include hardware and software for communicating with a pulse oximetry sensor (such as Figure 3 the sensor 102 shown therein), and hardware and software for calculating physiological characteristics received from the pulse oximetry sensor and using these characteristics to extract and display cardiac frequency phenomena in accordance with the techniques described herein.

[0063] Figure 4 is a schematic diagram of an electrocardiogram (“EKG”) device 120, which is shown optionally connected to an information management system 122 via a communication link 124. A common device for acquiring an EKG is a 12-lead electrocardiograph. The EKG device 120 and the information management system 122 receive power 126 from an external source. Among other things, the information management system 122 includes a central processing unit 128, which is connected via a data link 132 to a memory unit or database 130. The CPU 128 processes data and is connected to outputs such as a printer 134 and / or a display 136. Alternatively, if the optional information management system 122 is not utilized, the electrocardiogram (EKG) device 120 may be directly connected to the printer 134 or the display 136 via the communication link 124.

[0064] In an embodiment, a software program according to the embodiments provided herein may reside in the EKG device 120, the information management system 122, or another device configured to receive signals from the EKG device 120. The EKG device 120 is connected to a plurality of patient leads 138, each patient lead 138 having an electrode 140 to receive an EKG signal from a patient 142. The EKG device 120 has a signal conditioner 144 that receives the EKG signal and filters out noise, sets thresholds, isolates the signal, and provides an appropriate number of EKG signals for the number of leads 138 to an A / D converter 146, which converts the analog signal to a digital signal for processing by a microcontroller 148 or any other type of processing unit. The microcontroller 148 is connected to a memory unit 150 similar to the memory unit 130 or any other computer-readable storage medium.

[0065] In another embodiment, the computer system 80 may be configured to include hardware and software for communicating with an EKG device (such as the EKG device 120), or for performing the functions of an EKG device by communicating directly with EKG electrodes (such as Figure 4 the electrodes 140 shown), and hardware and software for calculating physiological characteristics received from the electrodes and using such characteristics to extract and display cardiac frequency phenomena.

[0066] Figure 5A An example of an angiographic image frame 500 is shown, which can be obtained by, for example, injecting a contrast agent into the blood vessels of a subject and imaging the subject's body using a rotating x-ray system of the type shown in Figure 1A and Figure 1B The plurality of blood vessels 502 of the body are shown in the angiographic image frame 500.

[0067] In one example, a computer (e.g., including image reconstructor 65, computer 66, and / or computer system 80) can obtain an angiographic image frame 500, for example, via a rotational x-ray system 28. The computer can further identify pixels in the angiographic image frame 500 that correspond to blood vessels in the body. For example, the computer can measure the brightness or intensity of each pixel in the angiographic image frame 500 and assign a probability value (p-value) to each pixel in at least one angiographic image regarding whether it is a blood vessel. In one example, the p-value is assigned based on the brightness or intensity of the pixel. In one example, a binarization threshold can be set to determine whether a pixel represents a blood vessel, and the p-value of each pixel can be compared with the binarization threshold to determine whether the pixel represents a blood vessel. In one example, if the p-value of a pixel exceeds the threshold, it can be determined that the pixel corresponds to a blood vessel. It has been found that the p-value may jump suddenly (e.g., from 0.1 to 0.9) between pixels that are not blood vessels and pixels that are blood vessels. Thus, the binarization threshold can be set anywhere within this range; however, consistently good results have been recorded using a binarization threshold of 0.5. In one embodiment, an encoder-decoder neural network can be trained to assign a p-value regarding whether each pixel corresponds to a blood vessel to each pixel. The neural network can also be used to identify pixels in the angiographic image frame that correspond to other structures (such as catheters).

[0068] Figure 5B Shows an example embodiment according to the present disclosure from Figure 5AThe segmented angiographic image 510 derived from the angiographic image frame 500 shown in []. In this example, the segmented angiographic image 510 is generated by a neural network (e.g., having an input layer, an output layer, and three or more layers between the input layer and the output layer), which is trained to identify blood vessels in the angiographic image frame. Although the description provided herein focuses on deep learning neural networks, it should be understood that these techniques can be utilized with any suitable machine learning model, and deep learning neural networks are just one example. The machine learning model can be implemented by any suitable machine learning technique (e.g., mathematical / statistical, classifier, feedforward, recursive, convolutional, or other neural networks, etc.). For example, neural networks that can be used include an input layer, one or more intermediate layers (e.g., including any hidden layers), and an output layer. Each layer can include one or more nodes or neurons, where the input layer neurons receive inputs (e.g., image data, feature vectors of the image, etc.) and can be associated with weight values. The neurons in the intermediate layer and the output layer can be connected to one or more neurons in the previous layer and receive the outputs of the connected neurons in the previous layer as inputs. Each connection can be associated with a weight value, and each neuron can produce an output based on the weighted combination of the inputs to that neuron. For some types of neural networks (e.g., recursive neural networks), the output of the neuron can further be based on a bias value. The weight (and bias) values can be adjusted based on various training techniques. For example, the machine learning of the neural network can be performed using a training set of original angiographic images as inputs and using the corresponding manually segmented angiographic images as known outputs, where the neural network attempts to produce the provided outputs and uses the error from the outputs (e.g., the difference between the produced output and the known output) to adjust the weight (and bias) values (e.g., via backpropagation or other training techniques).

[0069] The training data can be obtained based on any suitable angiographic training images. For example, the training data can be obtained in the following ways: (1) in a laboratory environment, using animals undergoing approved angiographic studies; (2) in a laboratory environment, using a physical model of a synthetic organ pumped with a fluid mechanical pump for angiographic imaging; and / or (3) from human clinical angiographic data. The data training system can use these training components alone or in any suitable combination to train the deep learning neural network.

[0070] The neural network techniques described herein can be used in combination with the spatio-temporal reconstruction techniques described in U.S. Application No. 16 / 784,073, filed on February 6, 2020 (which is incorporated herein by reference in its entirety). The spatio-temporal reconstruction of the image can be an input to the neural network, or the spatio-temporal reconstruction can be used to process the output of the neural network.

[0071] The computer can analyze all the pixels in the angiographic image frame 500 or a subset of the pixels in the angiographic image frame 500. For example, a user (e.g., a medical professional) can select a subset of the pixels within the angiographic image frame 500, and the computer can perform measurement analysis on the pixels corresponding to one or more blood vessels within the selected subset.

[0072] The computer can analyze the pixels in the angiographic image frame 500 corresponding to the blood vessels to identify the medial axis of the blood vessels, and determine the measurements of the blood vessels based on the medial axis analysis. The computer can further display a scale to indicate the measurement (e.g., length). The scale can be, for example, automatically superimposed on the angiographic image frame 500 (e.g., superimposed on the measured blood vessel).

[0073] In some embodiments, the medial axis analysis can provide measurements of multiple (e.g., all) blood vessels in the angiographic image frame 500. For example, the computer can identify the pixels in the angiographic image frame 500 corresponding to multiple blood vessels of the body. The computer can perform medial axis analysis on the pixels corresponding to the multiple blood vessels. Based on the medial axis analysis, the computer can determine the measurements (e.g., diameter) of the multiple blood vessels in the angiographic image frame 500.

[0074] In one example, the computer can compare the measurement of a blood vessel in the angiographic image frame 500 (e.g., the measured diameter of the blood vessel) with a reference measurement of the blood vessel (e.g., the reference diameter of the blood vessel). Based on the comparison, the computer can calibrate and / or validate the medial axis analysis. The reference diameter can be obtained via an angiographic injection catheter method. For example, a neural network can identify the injection catheter in the angiographic image frame 500. The diameter of the injection catheter can be known (e.g., determined from a catalog or package insert) and used to determine the blood vessel diameter. For example, if the diameter of the injection catheter is 6 French units, then the diameter is 2 mm (3 French units are defined as equal to 1 mm). The measurement obtained from the diameter of the injection catheter can provide validation of the (multiple) diameter measurements obtained from the angiographic image frame 500.

[0075] In an example embodiment, a series / sequence of angiographic image frames spanning the cardiac cycle can be obtained. The series of angiographic image frames can be obtained from the angiogram in a time series according to the duration of the contrast agent bolus present in the imaged organ. In one example, the computer can identify the pixels in each angiographic image frame corresponding to the blood vessels. The computer can perform medial axis analysis on the pixels corresponding to the blood vessels in each angiographic image frame. Based on the medial axis analysis, the computer can determine the measurements of the blood vessels in each angiographic image frame.

[0076] In a further example, a series of angiographic image frames spans at least one cardiac cycle. Based on the measurements of the blood vessels in each angiographic image frame, the computer can determine that the measurements of the blood vessels in at least one angiographic image frame are determined at a target / preferred time in the cardiac cycle for measuring the blood vessels. As a result, the computer can select a target time in the cardiac cycle for measuring the diameter of the blood vessels.

[0077] Consider an example in which the heart is being imaged. The heart contracts during systole and relaxes during diastole. By repeating the automatic blood vessel measurement for all images across the cardiac cycle, the computer can determine the variation of the blood vessel diameter across the cardiac cycle. The computer can further select a preferred / target time point in the cardiac cycle that has a maximum value for clinical decision-making regarding blood vessel measurement. In coronary angiography, the preferred time point for coronary artery diameter measurement can correspond to the end-diastole.

[0078] Applying the automatic blood vessel measurement to all images in a coronary angiogram can generate a spatio-temporal family of blood vessel diameter measurements across each cardiac cycle during the transit of the contrast agent bolus in the coronary angiogram. The resulting measurements can help improve decisions regarding treatment, such as whether to place a coronary stent and, if so, the appropriate size of the stent.

[0079] Figure 5C An example embodiment of a method for determining whether a pixel corresponding to a blood vessel (also referred to as a "vessel pixel") is located along the medial axis of the blood vessel (i.e., the central longitudinal axis) is shown. Vessel pixels located along the medial axis of the blood vessel are also referred to herein as "medial axis pixels". In this example, the medial axis analysis technique can also be used to determine whether a vessel pixel is located near a blood vessel branch point.

[0080] Briefly, for each pixel determined to be a vessel pixel (e.g., by the above-mentioned binarization technique), the medial axis analysis technique draws a small circle (also referred to as a "query circle") around the blood vessel circle and determines whether the circle intersects one or more pixels that do not correspond to a blood vessel (also referred to as "non-vessel pixels"). If the previous query circle does not intersect at least one non-vessel pixel, the size of the query circle can be iteratively increased until the query circle intersects at least one non-vessel pixel. In an example embodiment, the first query circle can include a group of pixels surrounding the pixel of interest, and any subsequent query circle can include a group of pixels surrounding the previous query circle.

[0081] If the first query circle fails to intersect a non-vessel pixel (e.g., see query circle I in Figure 5C ), then query circles with a larger diameter are iteratively drawn (e.g., see Figure 5Cthe interrogation circle in ) until it intersects with at least one non-vascular pixel. If, at a certain diameter, the interrogation circle intersects with a single non-vascular pixel (e.g., see Figure 5C the interrogation circle II in ), then the vascular pixel at the center of the interrogation circle is considered not to be a medial axis pixel. If, at a certain diameter, the interrogation circle intersects with two non-vascular pixels (e.g., see Figure 5C the interrogation circle III in ), then the vascular pixel at the center of the interrogation circle is considered to be a medial axis pixel (i.e., located along the medial axis, at the center of the blood vessel, equidistant from the opposite edges of the blood vessel). If, at a certain diameter, the interrogation circle intersects with three or more non-vascular pixels (e.g., see Figure 5C the interrogation circle IV in ), then the vascular pixel at the center of the interrogation circle is considered to be located at a branch point of the blood vessel (e.g., where one blood vessel segment branches into two blood vessel segments).

[0082] An example embodiment of this technique can be represented as follows: For each row: For each column: If the pixel at (row, column) is segmented as a blood vessel: Draw circles with gradually increasing diameters around the pixel: If a given diameter touches only one non-segmented (i.e., non-vascular) pixel: Go to the next (row, column). If a given diameter touches two non-segmented pixels: Record (row, column) as the medial axis of the blood vessel; Record the circle diameter as the blood vessel diameter; and Go to the next (row, column). If a given diameter touches three or more non-segmented pixels: Record (row, column) as the branch position; Record the circle diameter as the branch diameter; and Go to the next (row, column).

[0083] In the example embodiment, this technique can record a list of all pixels that are equidistant from the blood vessel edge in the segmented blood vessel. That is, the coordinates (e.g., row, column) of each medial axis pixel can be recorded. The list of coordinates defines the medial axis of the segmented blood vessel, and the list of coordinates can also be considered a skeletonized representation of the segmented blood vessel. (In the example embodiment, the skeleton can be one pixel wide.) There are several ways to evaluate the adjacency of medial axis points. One is the sequence of row, column iteration. Another is the analysis from the medial axis coordinates themselves. This technique is applicable to binary, pixelated data. The natural resolution is the pixel resolution.

[0084] In an example embodiment, the technique can determine the vessel width (which can be considered to correspond to the diameter of a blood vessel in real life) at each point (i.e., medial axis pixel) along the medial axis of the blood vessel in an angiographic image frame. The width is preferably determined in the lateral (i.e., perpendicular) direction with respect to the medial axis of the blood vessel. In the above-described skeletonization technique, the lateral direction can be naturally calculated from the shape of the query circle during expansion. That is, the measurement of the width in the lateral direction can be achieved by drawing a query circle with an increasing diameter until it intersects two or more non-vessel pixels, and recording the diameter of the query circle that intersects two non-vessel pixels as the diameter of the blood vessel at that point along the medial axis. Similarly, the diameter of a query circle that intersects three or more non-vessel pixels can be recorded as the diameter of a branch. (In an example embodiment, the diameter of a query circle that intersects two or more non-vessel pixels can be recorded as multiple pixels.) Thus, the technique can provide the medial axis and diameter at each point in each segmented blood vessel in an angiographic image frame.

[0085] Figure 5D An example of a skeletonized representation 520 of a segmented angiographic image 510 obtained using the above technique is shown. Figure 5B In this example, each skeleton segment is only one pixel wide. Since the technique is capable of detecting branch points, it allows the identification of the blood vessel segments between the branch points. The segmented blood vessels are marked here with an alternating color palette.

[0086] Figure 5E An example of how to measure the blood vessel diameter according to an example embodiment of the present technique by superimposing Figure 5D the skeletonized representation 520 on Figure 5B the segmented angiographic image 510 is shown. In Figure 5E this, the resulting superimposed image is labeled 530. In this example, white pixels are blood vessel pixels, black pixels are non-vessel pixels, and the segments of the skeletonized representation are shown in alternating colors. It can be seen that the skeletonized representation 520 and the segmented angiographic image 510 are superimposed such that the segments of the skeleton are along the medial axis of each blood vessel. Briefly, for each skeleton segment, iterate from one end of the segment to the other end, and at each pixel in the skeleton, measure the diameter of the blood vessel in pixels by counting how many pixels are needed in the lateral direction to reach a black pixel. For example, the number of pixels in the lateral direction from a medial axis pixel to a black pixel (i.e., the radius of the blood vessel) can be counted and doubled to obtain the diameter. Alternatively, the number of pixels corresponding to the diameter in the lateral direction (i.e., the number of white pixels between black pixels in the lateral direction) can be counted.

[0087] Pixel size calibration can be performed to convert the number of pixels of the width (diameter) across a blood vessel into units of size or dimension (e.g., millimeters). In an example embodiment, pixel size calibration can be performed by segmenting an angiography catheter in an angiography image (e.g., see Figure 5F ), performing skeletonization using medial axis techniques, and determining the number of pixels of the width (diameter) across the catheter. The diameter of the catheter is known (e.g., from the FDA package insert of the catheter manufacturer) and can be used to determine the image magnification in pixels per unit size (e.g., pixels per millimeter). The image magnification (or its reciprocal) can then be used as a conversion factor to convert the blood vessel diameter in pixels to a vessel diameter in units of size (such as millimeters).

[0088] Figure 5F FIG. shows the segmentation of catheter 540 in angiography image 550 according to an example embodiment of the present disclosure. In this illustration, catheter 540 is shown as a white line. The segmentation of catheter 540 can be performed using a neural network, such as the neural network described above, for example, trained using the original angiography image of the catheter in the body as input and the manually segmented image of the catheter in the body as known output.

[0089] Figure 5G FIG. shows a heat map 560 of the diameter of catheter 540 in pixels. In one example, the computer can present heat map 560. Heat map 560 includes a color image of catheter 540 on the left side, where the color and hue of the catheter in the image indicate the size of the catheter, and an image legend or scale 570 on the right side of the image equates the colors / hues in the image to a number of pixels. As described above, the catheter diameter can be looked up in the FDA package insert of the catheter manufacturer and entered into the computer. Alternatively, the catheter diameter can be retrieved by the computer from a database. In the example shown, the known catheter has a diameter of 2 mm. The image magnification in pixels per millimeter can be calculated by dividing the size of the catheter in pixels by the known diameter in millimeters. For example, in the case of a catheter with a diameter of 2 mm, if the heat map indicates that the catheter is 60 pixels wide (i.e., has a diameter of 60 pixels), the image magnification will be 30 pixels per millimeter, and the conversion factor in millimeters per pixel will be 1 millimeter per 30 pixels. Of course, size units other than millimeters can also be used.

[0090] As described above, a conversion factor determined by a computer based on the known diameter of a catheter in an angiographic image and the number of pixels measured transversely across the catheter width can be used by the computer to convert the diameter of a blood vessel in pixels to the diameter of the blood vessel in a size unit (such as millimeters). For example, if the conversion factor is 1 millimeter per 30 pixels and the diameter of the blood vessel at a particular point along the medial axis of the blood vessel is measured as 30 pixels, then the diameter of the blood vessel at that point in size units will be 1 millimeter (i.e., 30 pixels x 1 millimeter / 30 pixels = 1 millimeter). In this way, the diameter of each point in each segmented blood vessel in an angiographic image frame can be automatically measured.

[0091] Figure 5H A heat map 580 indicating measurements of the blood vessels of a body in accordance with an example embodiment of the present disclosure is shown. In one example, a computer can present the heat map 580. For example, the computer can display the heat map 580 on a monitor or other type of screen. The computer can also store the heat map 580 as a digital image, which can be retrieved by the computer at any time (among other things), copied to a portable computer-readable storage medium (such as a USB thumb drive or a DVD), transmitted via an electronic communication network to another computer, or transmitted to a printer for printing. In the example shown, the heat map 580 includes a color image 585 of the blood vessels on the left side and an image legend, scale, or key 590 on the right side of the image. The color and hue of the blood vessels in the image in the color image 585 indicate the diameter of the blood vessels, and the image legend, scale, or key 590 associates the colors / hues in the image with dimensions and / or dimension ranges in size units (e.g., millimeters). Using the heat map, a medical professional can determine the diameter of a blood vessel at a point by comparing the color and hue of the blood vessel at any point along its length with the legend, scale, or key 590.

[0092] The blood vessel diameter data can be screened for evidence of local stenosis, as this can meet the criteria for coronary artery stent placement. Modern criteria are listed in the 2021 ACC / AHA / SCAI Guideline for Coronary Artery Revascularization, which is available in section 4.1 at https: / / www.jacc.org / doi / 10.1016 / j.jacc.2021.09.006. These criteria are a stenosis of 50% or more in the left main coronary artery or 70% or more in a non-left main coronary artery. The guideline states that this can be a visual estimate, but no quantitative formula for doing so is given.

[0093] As described above, the medial axis skeletonization method described herein can identify vascular segments (such as coronary artery segments) between vascular branch points. The bolus of contrast agent traveling in a series of angiographic image frames can be used by a computer to determine the direction of blood flow in a given coronary artery segment. In a standard left main coronary artery injection, the first coronary artery to receive the bolus of contrast agent is the left main coronary artery. Coronary artery morphology statistics can be generated by a computer. For example, the computer can display a graphical user interface (also referred to herein as "GUI"), where a separate screen on the graphical user interface can display a histogram of coronary artery diameters, which can be overall or broken down by coronary artery segment. To determine if a coronary artery is stenosed, the starting 10% of the length of the coronary artery can be used as a baseline. If the subsequent 90% of the length on a non-left main coronary artery drops below 70%, the computer can mark that segment in the graphical user interface (or other rendering of the image) to draw the special attention of the interpreting cardiologist. An example marker is a Figure 5H yellow arrow pointing to the right at the suspected stenosis in

[0094] The statistical system bias preferably has an excessive sensitivity and brings those areas that may require additional consideration for stent placement to human cardiology interpretation. The final decision regarding the necessity of a stent is preferably left to the interpreting cardiologist.

[0095] Figure 6FIG. 600 shows an example of a GUI that can be generated by a computer to present a selection of one or more stents 610 to a user (e.g., a cardiologist). In this example, the GUI presents the selection of the stent 610 in combination with the above-described heat map 580. In an embodiment, the size of the stent 610 in the GUI 600 can be displayed in the same scale as the blood vessel in the heat map 580, so that the user can make an accurate size comparison. In an embodiment, information about the stent 610 (e.g., brand and / or size) can be displayed in the GUI. For example, by using a suitable input device (such as a touch screen, a mouse, or a trackball), by clicking or hovering over one of the stents, the stent information can be displayed in a pop-up window. In an embodiment, the GUI 600 can provide a drag-and-drop stent selection and simulation function. For example, the user can use a suitable input device to select one of the stents 610 in the GUI, drag the stent on the heat map 580, and place the stent on the stenotic artery segment for visual inspection. Since the stent and the blood vessel are displayed in the same scale, if the stent fits, it can be selected by the user for actual implantation. For example, when the user hovers or clicks on the stent dragged onto the image, the GUI can display options to complete or cancel the selection. Figure 6 FIG. 630 shows an example result of using the drag-and-drop operation of the GUI in green at 630. The decoration of the coronary artery stent option on the GUI helps to explain the cardiologist's judgment, but the cardiologist is responsible for making the decision. The GUI can identify the selected stent, for example, as "manufacturer name, 20 mm in length, 3 mm in diameter". In an embodiment, the computer can communicate this information with the hospital inventory and / or billing system via an electronic communication network.

[0096] Figure 7 FIG. 700 is a flowchart showing a method for measuring a blood vessel in a body according to an example embodiment. At operation 701, at least one angiographic image frame is obtained at a computer. For example, the (multiple) angiographic image frames can be obtained using a rotational x-ray system 28 ( Figure 1A and 1B ), and acquired or received by a computer in the rotational x-ray system 28 or by a separate computer (e.g., an image reconstructor 65 and / or a computer 66).

[0097] At operation 702, pixels corresponding to the blood vessels in the body in at least one angiographic image frame are identified. An example method for performing this operation is shown in Figure 8 FIG.

[0098] At operation 703, the computer identifies the medial axis of the blood vessel. An example method for performing this operation is shown in Figure 9 FIG.

[0099] At operation 704, the computer determines a measurement of a blood vessel in at least one angiographic image. An example method for performing this operation is shown in Figure 10 .

[0100] Figure 8 FIG. 800 is a flow chart showing a method for identifying pixels corresponding to a blood vessel according to an example embodiment. At operation 801, the computer measures or otherwise determines the brightness or intensity of each pixel in at least one angiographic image.

[0101] At operation 802, the computer assigns a probability value (p-value) to each pixel regarding whether the pixel corresponds to a blood vessel. As described above, the p-value can be assigned based on the brightness or intensity of the pixel. In an example embodiment, this operation can be performed by a neural network trained to identify pixels corresponding to blood vessels in an angiographic image.

[0102] At operation 803, the computer can compare each p-value with a predetermined threshold.

[0103] At operation 804, based on the comparison, it is determined whether the p-value of the pixel exceeds the threshold. If the p-value exceeds the threshold, then at operation 805, the computer can identify the pixel as corresponding to a blood vessel in the operation. Conversely, if the p-value of the pixel is less than or equal to the threshold, then at operation 806, the computer can identify the pixel as non-blood vessel. The threshold can be static (e.g., a fixed number) or dynamic (e.g., determined based on one or more factors, such as the average intensity of all or some pixels in at least one angiographic image frame).

[0104] Figure 9 FIG. 900 is a flow chart showing a method for identifying the medial axis of a blood vessel according to an example embodiment.

[0105] At operation 901, for each pixel determined to be a blood vessel pixel, the computer draws a small circle (also referred to as a "query circle") around the blood vessel pixel. In one example, the first query circle can be a group of pixels around the blood vessel pixel.

[0106] At operation 902, the computer determines whether the query circle intersects one or more non-blood vessel pixels. If the query circle does not intersect at least one non-blood vessel pixel, then at operation 904, the size of the query circle can be increased, and operation 902 can be performed again. This process can be repeated until the query circle intersects at least one non-blood vessel pixel, at which point the method proceeds to operation 903.

[0107] At operation 903, the computer determines one of the following: (a) if at a certain diameter, the query circle intersects a single non-vessel pixel, the vessel pixel at the center of the query circle is determined not to be a medial axis pixel; (b) if at a certain diameter, the query circle intersects two non-vessel pixels, the vessel pixel at the center of the query circle is considered a medial axis pixel; or (c) if at a certain diameter, the query circle intersects three or more non-vessel pixels, the vessel pixel at the center of the query circle is considered to be at a branch point of the vessel. In an example embodiment, the coordinates (e.g., row, column) of each medial axis pixel can be recorded. The list of coordinates defines the medial axis of the segmented vessel. The diameter in pixels of the corresponding query circle for each medial axis pixel can also be recorded (i.e., the diameter of the query circle that intersects two non-vessel pixels).

[0108] Figure 10 FIG. 1000 is a flow chart showing a method for measuring the diameter of a vessel in pixels according to an example embodiment. At operation 1001, the computer may identify vessel pixels located along the medial axis of a vessel in at least one angiographic image. In an example embodiment, the computer may identify medial axis pixels from a list of coordinates generated by the above-described skeletonization process.

[0109] At operation 1002, the computer may determine the diameter of the vessel in pixels at one or more points along the medial axis of the vessel by counting the number of vessel pixels that extend transversely across the width of the vessel and are perpendicular to the medial axis of the vessel. In an example embodiment, this may be achieved by overlaying a skeletonized version of the angiographic image on a segmented (binary) version of the angiographic image and counting the number of vessel pixels in a transverse direction relative to the medial axis of the vessel.

[0110] At operation 1003, the computer may convert the diameter of the vessel in pixels to a diameter in units of size or dimension (such as millimeters). In an example embodiment, a conversion factor determined by the method shown in Figure 11 may be used to perform this operation.

[0111] Figure 11 FIG. 1100 is a flow chart showing a method for determining a conversion factor for converting the diameter of a vessel in pixels to a diameter of the vessel in units of size or dimension (such as millimeters) according to an example embodiment.

[0112] At operation 1101, the computer may segment a catheter in at least one angiographic image frame. For example, at operation 1102, a neural network may be used to identify pixels in at least one angiographic image frame corresponding to the catheter (referred to herein as "catheter pixels").

[0113] At operation 1103, the computer may identify catheter pixels located along the medial axis of the catheter (e.g., using the medial axis skeletonization techniques described herein).

[0114] At operation 1104, the computer may count the number of catheter pixels that extend laterally across the width of the catheter and are perpendicular to the medial axis of the catheter.

[0115] At operation 1105, the computer may determine a pixel-to-size unit conversion factor by dividing the counted number of pixels by the known diameter of the catheter. This conversion factor can then be used to convert the diameter of a blood vessel in pixels to the diameter of the blood vessel in units of size or dimension (such as millimeters).

[0116] The present invention may include methods, systems, devices, and / or computer program products at any possible integrated technical detail level. The computer program product may include a computer-readable storage medium (or media) having computer-readable program instructions thereon for causing a processor to execute aspects of the present invention.

[0117] The computer-readable program instructions described herein can be downloaded from a computer-readable storage medium to a corresponding computing / processing device, or downloaded to an external computer or external storage device via a network (e.g., the Internet, a local area network, a wide area network, and / or a wireless network). The network may include conductive transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers, and / or edge servers. A network adapter card or network interface in each computing / processing device may receive the computer-readable program instructions from the network and forward the computer-readable program instructions for storage in a computer-readable storage medium within the respective computing / storage device.

[0118] Aspects of the present invention are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of the invention. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer-readable program instructions.

[0119] These computer-readable program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions executed via the processor of the computer or other programmable data processing apparatus create a means for implementing the functions / acts specified in the flowchart and / or one or more block diagrams. These computer-readable program instructions may also be stored in a computer-readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer-readable storage medium in which the instructions are stored comprises an article of manufacture including instructions that implement various aspects of the functions / acts specified in the flowchart and / or one or more block diagrams.

[0120] The computer-readable program instructions may also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions executed on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or one or more block diagrams.

[0121] The flowchart and block diagrams in the figures illustrate the architecture, functionality, and operation of possible implementations of systems, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in the flowchart or block diagrams may represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical function(s). In some alternative implementations, the functions noted in the blocks may occur out of the order noted in the figures. For example, two blocks shown in succession may, in fact, be executed substantially concurrently, or the blocks may sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special-purpose hardware-based systems that perform the specified functions or acts, or combinations of special-purpose hardware and computer instructions.

[0122] The foregoing description is to teach one of ordinary skill in the art how to practice the subject matter of the present application and is not intended to detail all those obvious modifications and variations that will become apparent to those skilled in the art upon reading this specification. For example, techniques other than the neural network examples described herein can be used to identify pixels corresponding to blood vessels in at least one angiographic frame. In one example, instead of identifying based on a comparison of the p-value of each pixel with a threshold p-value, identification can be based on a comparison of another value associated with each pixel (such as brightness or intensity) with a threshold. In another example, instead of identifying pixels having a value exceeding a threshold as blood vessel pixels, the technique can be modified to identify pixels having a value equal to or exceeding a threshold, a value equal to or less than a threshold, or a value less than a threshold as blood vessel pixels. In yet another example, wavefront analysis can be used to identify pixels corresponding to blood vessels in an angiographic image frame. In a wavefront analysis embodiment, at least one angiographic image frame is obtained and a form of wavefront analysis (also known as wavefront set analysis or microlocal harmonic analysis) is used to determine the locations and orientations of all edges in the angiographic image. Wavefront analysis can utilize a underlying analysis called shear wave analysis, which can involve fitting shear waves to find the edges of blood vessels in the angiographic image. One edge can be identified where the wavefront changes from positive to negative at all frequencies, and another edge can be identified where the wavefront changes from negative to positive at all frequencies.

[0123] In a further modification, instead of using the query circle method to measure dimensions transverse to the medial axis of a blood vessel, the computer can (e.g., using the coordinates of medial axis pixels and adjacent medial axis pixels) determine the slope of the medial axis at each medial axis pixel, determine the direction transverse to the slope, and count the number of blood vessel pixels between non-blood vessel pixels in the direction transverse to the slope.

[0124] In yet another modification, instead of generating a graphical user interface (GUI) with vascular measurements and stenosis information, embodiments herein can generate an image data file compliant with the Digital Imaging and Communications in Medicine (DICOM) standard, which is the standard image data format in current radiology. The metadata of the DICOM format includes a overlay field that can hold pixel data (e.g., a bitmap). In this exemplary embodiment, a DICOM-compliant overlay file can be generated, which can be presented by a DICOM-compliant commercial GUI under user selection. For example, the overlay file can include a bitmap that includes at least one of the following graphical elements: a scale for measuring vascular dimensions, stenosis measurements, arrows (e.g., pointing to a suspected stenosis), numbers, and stent size recommendations. The overlay file can be configured to overlay (like a mask) on an angiography image frame when displayed, such that the information in the overlay file can be seen as combined with the vessels in the angiography image. Preferably, the overlay file is configured to match the scale of the angiography image frame, e.g., such that when the user selects to display the overlay, the arrows in the overlay file can point to the suspected stenosis in the angiography image. To generate the vascular measurements and stenosis information as a DICOM-compliant overlay, the computer can save the overlay as a separate bitmap file, or directly encode the bitmap into the overlay field of the DICOM file. For example, the computer can rasterize the information into the overlay DICOM field. DICOM-compliant software packages that can be used by radiologists or cardiologists typically have a checkbox on their GUI, and the radiologist or cardiologist can click the checkbox to display the overlay. All hospitals and radiology units have contracts with DICOM viewer companies. It is typically part of the institution's PACS (Picture Archiving and Communication System).

[0125] In its simplest form, the techniques described herein can be implemented as a cloud-based software as a service (SaaS) that receives angiographic image frames from an originating entity via an electronic communication network (such as a local area network, wide area network, and / or the Internet), performs vascular measurements, places the results in an overlay field of a DICOM-compliant file, and returns the DICOM-compliant file to the originating entity via the electronic communication network. An example data flow can include the following operations: (a) an angiography suite performs an angiogram; (b) the angiogram is received by a computer as a DICOM format file (the DICOM format allows for a family of images captured in the angiogram. In the current standard, the images are in the PixelData field of the DICOM file); (c) the computer records the address of the sending system; (d) the computer performs a spatio-temporal transformation of the angiography data; (e) the computer performs vascular measurements, identifies stenotic regions, and finds a matching stent in a stent catalog; (f) the computer creates a new DICOM format file in which the vascular measurements, stenotic regions, and associated information are presented in the overlay field of the DICOM file (and optionally, an enhanced version of the angiographic image frame (such as a segmented image), a spatio-temporal transformed image, or a sequence of images underlying a medial axis transform is appended to the PixelData field of the same DICOM file); and (g) the resulting DICOM file is sent back to the originating sender. The computer can also save a log of the entire transaction. The originating entity receives the DICOM file sent by the computer; and a radiologist and cardiologist can view the DICOM file on a computer screen using their DICOM-compliant viewers. When viewing the DICOM file, they can click on an overlay button to display vascular measurement data, etc. overlaid on the original angiographic image (or alternatively, the spatio-temporally transformed angiographic image), and unclick the overlay button to remove the overlay.

[0126] However, all such modifications and variations are intended to be included within the scope of the invention as defined by the appended claims. The claims are intended to cover any order of components and steps that effectively meet their intended objectives, unless the context specifically indicates the contrary.

Claims

1. A method, comprising: obtaining, at a computer, at least one angiographic image frame; identifying, at the computer, pixels in the at least one angiographic image frame corresponding to blood vessels of a body; determining, at the computer, a medial axis of the blood vessels; and determining, at the computer, a diameter of the blood vessels in the at least one angiographic image frame by counting pixels corresponding to the blood vessels in a direction transverse to the medial axis of the blood vessels.

2. The method according to claim 1, wherein Determining the medial axis of the blood vessels includes performing medial axis skeletonization on the pixels corresponding to the blood vessels at the computer.

3. The method according to claim 2, wherein: Performing the medial axis skeletonization includes, for each pixel corresponding to a blood vessel, drawing a query circle around the pixel at the computer and determining at the computer whether the query circle intersects one or more pixels not corresponding to blood vessels.

4. The method according to claim 3, wherein: Performing the medial axis skeletonization further includes determining at the computer that a pixel is not a medial axis pixel when the query circle intersects only one pixel not corresponding to a blood vessel.

5. The method according to claim 3, characterized in that, Performing the medial axis skeletonization further includes determining at the computer that a pixel is a medial axis pixel when the query circle intersects two pixels not corresponding to blood vessels.

6. The method according to claim 3, wherein Performing the medial axis skeletonization further includes determining at the computer that a pixel is located at a branch point of the blood vessel when the query circle intersects three or more pixels not corresponding to blood vessels.

7. The method according to claim 5, wherein Determining the measurement of the blood vessels further includes generating a segmentation image of the blood vessels using the pixels corresponding to the blood vessels; and overlaying, at the computer, the medial axis of the blood vessels and the segmentation image of the blood vessels.

8. The method according to claim 7, wherein Determining the measurement of the blood vessels further includes recording, at the computer, the diameter of a query circle intersecting two pixels not corresponding to blood vessels as the diameter of the blood vessels.

9. The method according to claim 1, further comprising converting, at the computer, the diameter of the blood vessels in pixels to a diameter in terms of size.

10. The method according to claim 1, further comprising presenting, at the computer, a color map of the blood vessels, in which the color and hue of the blood vessels correspond to the diameter of the blood vessels.

11. The method according to claim 1, further comprising detecting, at the computer, a stenosis in the blood vessels.

12. The method according to claim 11, further comprising generating, at the computer, a graphical user interface that displays at least one stent in combination with an image of the blood vessels.

13. The method according to claim 12, wherein The at least one stent and the image of the blood vessels are displayed at the same scale.

14. The method according to claim 12, wherein The graphical user interface allows the at least one stent to be dragged and placed on the image of the blood vessels.

15. The method according to claim 1 further includes generating an overlay file configured to be superimposed on the at least one angiographic image frame and be co-visible with the at least one angiographic image frame, the overlay file including at least one graphical element selected from the group consisting of a dimensional scale, an arrow, a number, and stent information for vascular measurement.

16. The method according to claim 15, characterized in that, The at least one graphical element is configured to be displayed in the same scale as the at least one angiographic image frame.

17. A system includes: One or more computer processors; One or more computer-readable storage media; Program instructions stored on the one or more computer-readable storage media for execution by at least one of the one or more computer processors, the program instructions including instructions for: Obtaining at least one angiographic image frame; Identifying pixels corresponding to a blood vessel of a body in the at least one angiographic image frame; Determining a medial axis of the blood vessel; and Determining a diameter of the blood vessel in the at least one angiographic image frame by counting pixels corresponding to the blood vessel in a direction transverse to the medial axis of the blood vessel.

18. The system according to claim 17, wherein The instructions for determining the medial axis of the blood vessel include instructions for performing medial axis skeletonization of the pixels corresponding to the blood vessel.

19. The system according to claim 18, wherein, The instructions for performing medial axis skeletonization include instructions for drawing a query circle around each pixel corresponding to the blood vessel and determining whether the query circle intersects one or more pixels not corresponding to the blood vessel.

20. The system according to claim 19, wherein, The instructions for performing the medial axis skeletonization further include instructions for determining at the computer that a pixel is not a medial axis pixel when the query circle intersects only one pixel not corresponding to the blood vessel.

21. The system according to claim 19, wherein The instructions for performing the medial axis skeletonization further include instructions for determining at the computer that a pixel is a medial axis pixel when the query circle intersects two pixels not corresponding to the blood vessel.

22. The system according to claim 19, wherein The instructions for performing the medial axis skeletonization further include instructions for determining at the computer that a pixel is located at a branch point of the blood vessel when the query circle intersects three or more pixels not corresponding to the blood vessel.

23. The system according to claim 21, wherein, The instructions for determining the measurement of the blood vessel further include: instructions for generating a segmented image of the blood vessel using the pixels corresponding to the blood vessel; and Instructions for overlaying the medial axis of the blood vessel and the segmented image of the blood vessel at the computer.

24. The system according to claim 23, wherein The instructions for determining the measurement of the blood vessel further include instructions for recording at the computer the diameter of the query circle intersecting two pixels not corresponding to the blood vessel as the diameter of the blood vessel.

25. The system according to claim 17, wherein The instructions further include instructions for converting at the computer the diameter of the blood vessel in pixels to a diameter in size units.

26. The system according to claim 17, wherein The instructions further include instructions for presenting at the computer a color map of the blood vessel, in which the color and hue of the blood vessel correspond to the diameter of the blood vessel.

27. The system according to claim 17, wherein The instructions further include instructions for detecting a stenosis in the blood vessel at the computer.

28. The system according to claim 27, wherein The instructions further include instructions for generating a graphical user interface at the computer, the graphical user interface displaying at least one stent in combination with an image of the blood vessel.

29. The system according to claim 28, wherein, The at least one stent and the image of the blood vessel are displayed in the same scale.

30. The system according to claim 28, wherein The graphical user interface allows the at least one stent to be dragged and placed on the image of the blood vessel.

31. The system according to claim 17, wherein The instructions further include instructions for generating an overlay file configured to be superimposed on and be co-visible with the at least one angiographic image frame, the overlay file including at least one graphical element selected from the group consisting of a scale for blood vessel measurement, arrows, numbers, and stent information.

32. The system according to claim 31, wherein The at least one graphical element is configured to be displayed in the same scale as the at least one angiographic image frame.

33. One or more non-transitory computer-readable storage media encoded with instructions that, when executed by a processor, cause the processor to: Obtain at least one angiographic image frame; Identify pixels in the at least one angiographic image frame corresponding to a blood vessel of a body; Determine a medial axis of the blood vessel; and Determine a diameter of the blood vessel in the at least one angiographic image frame by counting pixels corresponding to the blood vessel in a direction transverse to the medial axis of the blood vessel.

34. The computer-readable storage medium according to claim 33, wherein The instructions for determining the medial axis of the blood vessel include instructions that, when executed by the processor, cause the processor to perform medial axis skeletonization on the pixels corresponding to the blood vessel.

35. The computer-readable storage medium according to claim 34, wherein The instructions for performing medial axis skeletonization include instructions that, when executed by the processor, cause the processor to draw a query circle around each pixel corresponding to a blood vessel and determine whether the query circle intersects one or more pixels not corresponding to the blood vessel.

36. The computer-readable storage medium according to claim 35, wherein The instructions for performing the medial axis skeletonization further include instructions that, when executed by the processor, cause the processor to determine at the computer that a pixel is not a medial axis pixel when the query circle intersects only one pixel not corresponding to the blood vessel.

37. The computer-readable storage medium according to claim 35, wherein The instructions for performing the medial axis skeletonization further include instructions that, when executed by the processor, cause the processor to determine at the computer that a pixel is a medial axis pixel when the query circle intersects two pixels not corresponding to the blood vessel.

38. The computer-readable storage medium according to claim 35, wherein The instructions for performing the medial axis skeletonization further include instructions that, when executed by the processor, cause the processor to determine at the computer that a pixel is located at a branch point of the blood vessel when the query circle intersects three or more pixels not corresponding to the blood vessel.

39. The computer-readable storage medium according to claim 38, wherein The instructions for determining the measurement of the blood vessel further include instructions that, when executed by the processor, cause the processor to generate a segmented image of the blood vessel using the pixels corresponding to the blood vessel; and Overlay the medial axis of the blood vessel and the segmented image of the blood vessel at the computer.

40. The computer-readable storage medium according to claim 39, wherein The instructions for determining the measurement of the blood vessel further include: instructions that, when executed by a processor, cause the processor at the computer to record the diameter of the query circle that intersects two pixels not corresponding to the blood vessel as the diameter of the blood vessel.

41. The computer-readable storage medium according to claim 33, wherein The instructions further include: instructions that, when executed by a processor, cause the processor at the computer to convert the diameter of the blood vessel in pixels to a diameter in size units.

42. The computer-readable storage medium according to claim 33, wherein The instructions further include: instructions that, when executed by a processor, cause the processor at the computer to present a color map of the blood vessel, in which the color and hue of the blood vessel correspond to the diameter of the blood vessel.

43. The computer-readable storage medium according to claim 33, wherein The instructions further include: instructions that, when executed by a processor, cause the processor at the computer to detect stenosis in the blood vessel.

44. The computer-readable storage medium according to claim 43, characterized in that, The instructions further include: instructions that, when executed by a processor, cause the processor at the computer to generate a graphical user interface that displays at least one stent in combination with an image of the blood vessel.

45. The computer-readable storage medium according to claim 44, wherein, The at least one stent and the image of the blood vessel are displayed at the same scale.

46. The computer-readable storage medium according to claim 44, wherein The graphical user interface allows the at least one stent to be dragged and placed on the image of the blood vessel.

47. The computer-readable storage medium according to claim 33, wherein The instructions further include: instructions that, when executed by a processor, cause the processor to generate an overlay file configured to be superimposed on and co-visible with the at least one angiographic image frame, the overlay file including at least one graphical element selected from the group consisting of a scale for blood vessel measurement, arrows, numbers, and stent information.

48. The computer-readable storage medium according to claim 47, wherein The at least one graphical element is configured to be displayed at the same scale as the at least one angiographic image frame.

Images