Motion compensated wavelet angiography

Abstract

Methods and systems for extracting a heart rate angiography phenomenon of an unconstrained blood vessel object from an angiography study are provided. In one example, a computer can obtain a series of angiography image frames obtained at a rate faster than a heart rate. Each image frame can include a plurality of pixels, and each pixel can have a corresponding intensity. The computer can apply an optical flow technique to the angiography image frames to generate a plurality of paths corresponding to displacements of respective pixels from image frame to image frame. The computer can further generate a spatiotemporal reconstruction of the heart rate angiography phenomenon based on the plurality of paths and the corresponding intensities associated with the respective pixels of the paths, and output the spatiotemporal reconstruction of the heart rate angiography phenomenon for display in one or more images.

Description

TECHNICAL FIELD

[0001] The field relates generally to angiography, and more specifically, to methods and systems for reconstructing a spatiotemporal image of a moving vascular pulse wave applied to a sequence of angiography images acquired at a rate faster than the heart rate, where the blood vessels, tissue, and / or organs undergo large-scale motion at the heart rate. BACKGROUND

[0002] The heart sends blood to the body as a sequence of arterial stroke volumes. When the traveling arterial stroke volumes reach the vasculature of an organ, the arterial stroke volumes cause coordinated motion of the blood vessels and dilation of the vessel diameters to accommodate the passing arterial stroke volume. Since the heart chambers contract at the heart rate, these coordinated motions and dilations of the arterial vessels occur at the heart rate throughout the body.

[0003] In angiography, a dose of chemical contrast agent is injected into the vasculature. The contrast agent allows the blood vessels in the body to be visualized. Prior art is not suitable for spatiotemporal reconstruction of a moving vascular pulse wave in a blood vessel object that undergoes large-scale motion, such as a blood vessel that undergoes large-scale motion due to the beating of the heart. The large-scale motion of the blood vessel object hinders the measurement of the blood dynamics pulse wave travel within the object. Such large-scale motion is too large to be ignored in angiography. SUMMARY

[0004] Embodiments of the invention relate to methods, systems, and computer readable media for reconstructing heart rate phenomena in angiography data, where a blood vessel object (e.g., tissue, organ, blood vessel, etc.) undergoes large-scale motion. These techniques follow and compensate for the motion in unconstrained blood vessel objects and improve existing wavelet angiography techniques in blood vessel structures that undergo small-scale motion.

[0005] Motion in unconstrained blood vessel objects is referred to herein as large-scale or wide-amplitude or unconstrained heart rate motion. Embodiments described herein can be used to track the unconstrained heart rate motion of a blood vessel object, for example, using an adjusted moving path as a path integral. Embodiments that determine the moving path can be combined with wavelet or other time-indexed preservation angiography techniques to allow spatiotemporal reconstruction to be performed on any physiological organ, tissue, or vasculature. For example, this can enable spatiotemporal reconstruction of a pulse wave at a given point in a blood vessel that undergoes large-scale motion.

[0006] In one form, a method for extracting a heart-rate angiography phenomenon of an unconstrained blood vessel object from an angiography study is provided. The method includes: obtaining, at a computer, a series of angiography image frames obtained at a rate faster than a heart rate, wherein each image frame includes a plurality of pixels, and wherein each pixel has a corresponding intensity; applying, at the computer, an optical flow technique to the angiography image frames to generate a plurality of paths corresponding to displacements of respective pixels from image frame to image frame; generating, at the computer, a spatiotemporal reconstruction of the heart-rate angiography phenomenon based on the plurality of paths and the corresponding intensities associated with the respective pixels of the paths; and outputting for display the spatiotemporal reconstruction of the heart-rate angiography phenomenon in one or more images. The method can improve angiography imaging by compensating for motion in the blood vessel structure.

[0007] In one example, the method further includes, for each image frame, recursively integrating the displacements of a given pixel in a forward time direction and a reverse time direction to generate an optical flow trajectory of the given pixel. The recursive integration can enable tracking of object motion in the blood vessel structure.

[0008] In one example, the method further includes selecting an image frame of interest; and determining a displacement of a given pixel of the image frame of interest based on image frames within a plurality of image frames of the image frame of interest. Determining the displacement within the plurality of image frames can enable maintaining consistency with a frequency resolution of a mother wavelet function. In one example, the number of image frames can be five. In a case where a heart is beating at a rate of sixty times per minute and angiography images are acquired at five hertz, the number five can be selected. In this example, determining the displacement within five frames can be equivalent to determining the displacement of one heartbeat before and one heartbeat after the image frame of interest. Any suitable number of image frames can be selected based on a corresponding number of heartbeats before and after the image frame of interest (e.g., two heartbeats before and after, half a heartbeat before and after, etc.).

[0009] In one example, applying the optical flow technique includes applying a dense optical flow technique that measures optical flow of a plurality of pixels from image frame to image frame. The dense optical flow technique can enable computation of a displacement for each pixel.

[0010] In one example, applying the optical flow technique includes applying a sparse optical flow technique that includes: tracking a limited number of object locations from image frame to image frame; and interpolating movement of intermediate object locations. The sparse optical flow technique can enable computation of motion for only key pixels.

[0011] In one example, applying optical flow techniques includes determining each path based on a local coordinate system. The local coordinate system can reduce computational requirements by simplifying wavelet calculations. The local coordinate system can be defined, for example, by a particular location in the blood vessel. If the blood vessel is undergoing large-scale motion, a neighborhood around the particular location that defines the coordinate system can remain well-described by that coordinate system.

[0012] In one example, the method further includes providing the one or more images as a cine video sequence. The cine video sequence can provide motion-compensated, wavelet-transformed results.

[0013] In another form, a system is provided. The system includes a communication interface configured to obtain a series of angiogram image frames obtained at a rate faster than a heart rate, wherein each image frame includes a plurality of pixels, and wherein each pixel has a corresponding intensity; and one or more processors coupled to the communication interface, wherein the one or more processors are configured to: apply optical flow techniques to the angiogram image frames to generate a plurality of paths corresponding to displacements of respective pixels from image frame to image frame; generate a spatiotemporal reconstruction of a heart rate angiographic phenomenon based on the plurality of paths and the corresponding intensities associated with the respective pixels of the paths; and output the spatiotemporal reconstruction of the heart rate angiographic phenomenon for display in one or more images. The system can improve angiographic imaging by compensating for motion in the blood vessel structure.

[0014] In another form, one or more non-transitory computer-readable storage media are provided. The one or more non-transitory computer-readable storage media are encoded with instructions that, when executed by a processor, cause the processor to: obtain a series of angiogram image frames obtained at a rate faster than a heart rate, wherein each image frame includes a plurality of pixels, and wherein each pixel has a corresponding intensity; apply optical flow techniques to the angiogram image frames to generate a plurality of paths corresponding to displacements of respective pixels from image frame to image frame; generate a spatiotemporal reconstruction of a heart rate angiographic phenomenon based on the plurality of paths and the corresponding intensities associated with the respective pixels of the paths; and output the spatiotemporal reconstruction of the heart rate angiographic phenomenon for display in one or more images. The one or more non-transitory computer-readable storage media can improve angiographic imaging by compensating for motion in the blood vessel structure.

[0015] Other objects and advantages of these techniques will be more fully apparent from the description and drawings that follow. BRIEF DESCRIPTION OF DRAWINGS

[0016] FIG. 1A and FIG. 1Bare side and partial schematic views showing examples of a rotational x-ray system that can be used with embodiments of the present disclosure to acquire angiographic data.

[0017] FIG. 2 is a schematic diagram of a computer system or information processing device that can be used with embodiments of the present disclosure.

[0018] FIG. 3 is a flowchart showing an example data flow path for spatiotemporal reconstruction in the absence of constrained cardiac motion, according to embodiments of the present disclosure.

[0019] FIGS. 4A-4C are angiographic image frames showing an example of unconstrained blood vessel motion (e.g., coronary artery motion) in a pig coronary artery. FIG. 4A and FIG. 4B are sequential angiographic image frames in which the location of pixels is tracked from frame to frame using techniques provided herein. FIG. 4C is FIG. 4A superimposed with FIG. 4B to show the displacement of pixels due to unconstrained blood vessel motion (e.g., the contractility of the heart).

[0020] FIG. 5 shows an example of an optical flow path related to an example angiographic image frame of a given coronary artery, according to embodiments of the present disclosure.

[0021] FIG. 6A and FIG. 6B are example angiographic image frames showing a wavelet angiographic representation after taking into account unconstrained blood vessel motion, such as coronary artery motion, according to embodiments of the present disclosure. FIG. 6A shows the images before spatiotemporal reconstruction, and FIG. 6B shows the images after spatiotemporal reconstruction.

[0022] FIG. 7 is a flowchart showing techniques for reconstructing cardiac frequency phenomena in angiographic data of an unconstrained blood vessel object, according to example embodiments of the present disclosure. DETAILED DESCRIPTION

[0023] The present techniques can provide improvements over existing techniques for angiographic imaging of blood vessels, tissues, and / or organs having unconstrained motion.

[0024] For example, existing angiography techniques are not well suited for spatiotemporal reconstruction of moving blood vessel pulse waves through a subject with unconstrained blood vessel motion. For example, the heart is a muscular organ, and the left and right sides of the heart are lungs, which are soft organs that cannot restrict heart motion. Thus, the heart, associated blood vessels, and tissue are examples of subjects that experience wide-amplitude heart motion. While existing angiography techniques can be applied to blood vessels, tissue, and / or organs with wide-amplitude heart motion, large-scale motion of such subjects presents challenges when attempting to measure the travel of hemodynamic pulse waves within the subject.

[0025] Embodiments of the present disclosure can utilize motion tracking for unconstrained blood vessel subjects that experience motion at heart frequency. By tracking motion of a subject in a sequence of two-dimensional images, and using the tracked motion to directionally guide the spatial extent (of the images) covered by a wavelet transform (or other time-indexed preserving mathematical transform), spatiotemporal reconstruction can be applied to unconstrained heart motion subjects.

[0026] An angiography image sequence includes a set of two-dimensional image frames acquired at a rate faster than the heart frequency as a function of time. Thus, this three-dimensional data set includes two spatial dimensions and one temporal dimension. If there is motion of blood vessels and / or other structures of interest in the angiography image frames, a given location of a given structure will not necessarily occupy the same pixel coordinates in two adjacent or nearby temporal image frames.

[0027] According to embodiments of the present disclosure, a pixel can be considered as a temporal channel, where the pixel location can change from frame to frame in a sequence of angiography images.

[0028] Motion tracking is performed on image pixels to allow measurements at the heart frequency of blood vessel pulses in a moving heart blood vessel, tissue, or organ. The images can be processed based on a global coordinate system. The global coordinate system can be represented in the angiography images by a three-dimensional grid. Two dimensions can be defined by horizontal and vertical rows / columns of pixels in the images, and the third dimension can be defined by image frame number (e.g., a sequence number of a given image frame in an angiography image sequence).

[0029] Motion can be further tracked with respect to a local coordinate system, e.g., a moving blood vessel, organ, or tissue is tracked with respect to a local coordinate system. For example, each path of a subject can be determined based on a local coordinate system. A local coordinate system can be defined for each object or each key pixel in each image frame of an angiography sequence. Assume that an angiography film has a key blood vessel branch point "a" at frame tl. The spatial coordinates of object "a" in a two-dimensional image frame are x t1a and y t1a At frame t2, object "a" has shifted such that the two-dimensional coordinates of object "a" are x t2a and yt2b In a local coordinate system, object "a" in image frame ti can be located at position 0 in the x-dimension and position 0 in the y-dimension. At image frame t2, the displacement of object "a" can be maintained such that the position of object a remains at 0 in the x-dimension and 0 in the y-dimension. The portions of the blood vessel near object "a" can be assigned coordinate positions relative to "a" that remain constant from one image frame to the next.

[0030] The local coordinate system can be determined by modifying the global coordinate system based on the motion and displacement of the objects. For example, if a particular turn on the blood vessel is measured to move five pixels to the right between one image frame and the next, the blood flowing within that pixel can be considered to move the same five pixels in the same direction. In this sense, the blood vessel orients the local coordinate system. Specifically, the local coordinate system can be translated five pixels to the right relative to the global coordinate system. Thus, the global coordinate system can serve as a reference coordinate system for one or more local coordinate systems.

[0031] Thus, the pixels of the image are transformed to a local coordinate system, and a track or path is determined for the transformed pixels to track the position of the pixels over time. This track or path is fed into a wavelet transform or other transform that preserves the time index. For purposes of performing the wavelet transform, the angiogram image sequence can be considered as an x by y pixel array of length t frames (e.g., an x by y array of time signals of length t). Wavelet transforms can operate on one-dimensional sequences of numbers, and thus, if the object motion (e.g., motion within a given pixel) is large relative to the time / frequency resolution of the mother wavelet selected (e.g., if the motion is too large to be satisfactory), the techniques described herein can enable tracking of the motion of the blood vessel object. Specifically, the motion of each pixel in each frame can be estimated, and the motion can be inverted (e.g., compensated for). The wavelet transform can operate on the motion-inverted data. The motion can be recovered in the wavelet-transformed data by looking up the motion and applying them. Thus, after the wavelet computation is complete, the local coordinate system can be restored to the original coordinate system.

[0032] Referring to FIGS. 1A-3 , an example system or device that can be used to perform embodiments of the present application is shown. It should be understood that such a system and device is merely an example of one representative system and device and that other hardware and software configurations are also suitable for use with embodiments of the present application. Therefore, embodiments are not intended to be limited to the particular systems and devices shown herein, and as such it is recognized that other suitable systems and devices can be employed to carry out embodiments of the present application without departing from the spirit and scope of the subject matter presented herein.

[0033] Referring first to FIG. 1A and FIG. 1B, showing a rotational x-ray system 28 that can be used to obtain an angiogram at a rate faster than the heart rate, such as via a fluoroscopic angiogram. In acquiring an angiogram, a chemical contrast agent can be injected into a patient positioned between the x-ray source and detector, and x-ray projections are captured by the x-ray detector as two-dimensional images (i.e., angiogram image frames). A sequence of such image frames comprises an angiographic study, and according to embodiments of the present application, the angiogram image frames can be acquired at a rate faster than the heart rate to allow motion tracking and spatiotemporal reconstruction of the heart rate phenomena into a cardiac spatial angiogram.

[0034] As shown in FIG. 1, FIG. 1A An example of an angiographic imaging system is shown in the form of a rotational x-ray system 28 that includes a gantry having a C-arm 30 that carries an x-ray source assembly 32 in one of its ends and an x-ray detector array assembly 34 in the other of its ends. The gantry enables the x-ray source assembly 32 and the x-ray detector array assembly 34 to be oriented at different positions and angles about a patient placed on a table 36 while providing access to the patient for a physician. The gantry includes a base 38 having a horizontal leg 40 that extends below the table 36 and a vertical leg 42 that extends upward at an end of the horizontal leg 40 spaced apart from the table 36. A support arm 44 is rotatably fixed to an upper end of the vertical leg 42 for rotation about a horizontal pivot 46.

[0035] The horizontal pivot 46 is aligned with a centerline of the table 36, and the support arm 44 extends radially outward from the horizontal pivot 46 to support a C-arm drive assembly 47 on an outer end thereof. The C-arm 30 is slidably fixed to the C-arm drive assembly 47 and is coupled to a drive motor (not shown) that slides the C-arm 30 to rotate about a C-axis 48 as indicated by arrow 50. The horizontal pivot 46 and the C-axis 48 intersect one another at a system isocenter 56 located above the table 36 and are perpendicular to one another.

[0036] The x-ray source assembly 32 is mounted to one end of the C-arm 30, and the x-ray detector array assembly 34 is mounted to the other end thereof. The x-ray source assembly 32 emits x-rays that are directed to the x-ray detector array assembly 34. Both assemblies 32 and 34 extend radially inward to the horizontal pivot 46 so that a central ray of the beam passes through the system isocenter 56. Thus, during acquisition of x-ray attenuation data from a subject placed on the table 36, the central ray of the beam can be rotated about the system isocenter 56 about either the horizontal pivot 46 or the C-axis 48 or both.

[0037] The x-ray source assembly 32 includes an x-ray source that emits a beam of x-rays when energized. A central ray passes through the system isocenter 56 and impinges a two-dimensional flat panel digital detector 58 housed in the x-ray detector array assembly 34. The two-dimensional flat panel digital detector 58 can be, for example, a 2048 x 2048 element two-dimensional array of detector elements. Each element produces an electrical signal representative of the intensity of the impinging x-rays, and thus the attenuation of the x-rays as they pass through the patient. During a scan, the x-ray source assembly 32 and the x-ray detector array assembly 34 rotate about the system isocenter 56 to acquire x-ray attenuation projection data from different angles. In some aspects, the detector array is capable of acquiring fifty projection or image frames per second, which is a limiting factor in determining how many image frames can be acquired at a prescribed scan path and speed.

[0038] Referring to FIG. 1B , rotation of the assemblies 32 and 34 and operation of the x-ray source are governed by a control mechanism 60 of the x-ray system. The control mechanism 60 includes an x-ray controller 62 that provides power and timing signals to the x-ray source assembly 32. A data acquisition system (DAS) 64 in the control mechanism 60 samples data from the detector elements and passes the data to an image reconstructor 65. The image reconstructor 65 receives the digitized x-ray data from the DAS 64 and performs high-speed image reconstruction according to the methods of the present disclosure. The reconstructed images are used as input to a computer 66, which stores the images in a mass storage device 69 or further processes the images. The image reconstructor 65 can be a separate computer or can be integrated with the computer 66.

[0039] The control mechanism 60 also includes a gantry motor controller 67 and a C-axis motor controller 68. In response to motion commands from the computer 66, the motor controllers 67 and 68 provide power to motors in the x-ray system that produce rotation about the horizontal pivot axis 46 and the C-axis 48, respectively. The computer 66 also receives commands and scan parameters from an operator via a console 70 having a keyboard and other manually-operable control devices. An associated display 72 allows the operator to observe reconstructed image frames and other data from the computer 66. The commands provided by the operator are used by the computer 66, under the direction of stored programs, to provide control signals and information to the DAS 64, the x-ray controller 62, and the motor controllers 67 and 68. In addition, the computer 66 operates a table motor controller 74 that controls the motorized table 36 to position the patient relative to the system isocenter 56.

[0040] Referring now to FIG. 2 , a block diagram of a computer system or information processing apparatus 80 (e.g., the image reconstructor 65 and / or the computer 66 in FIG. 1B , which can incorporate an angiographic imaging system such as the x-ray system 10 of FIG. 1, is shown. The computer system 80 includes a processor 81, a memory 82, a storage device 83, and an input / output interface 84. The processor 81 can be a single processing unit or can include multiple processing units. The processor 81 can be a general-purpose processor or can be a special-purpose processor. The storage device 83 can include a hard disk drive, a floppy disk drive, a CD-ROM drive, a Blu-ray® drive, or other device suitable for reading and writing data. The storage device 83 can include a non-transitory computer-readable medium in which one or more programs or data can be stored. For example, the one or more programs can be for implementing methods and / or processing techniques associated with the present disclosure. The programs can be stored on the storage device 83 and / or the memory 82 and / or can be made available via the input / output interface 84.FIG. 1A and FIG. 1B a rotating x-ray system 28) to provide enhanced functionality in accordance with embodiments of the application or as a standalone device for motion tracking and extracting cardiac frequency phenomena from angiographic data. The information processing device 80 can be located locally to the rotating x-ray system 28 or remotely from the rotating x-ray system 28. In one example, the functionality performed by the information processing device 80 can be provided as a software as a service (SaaS) option. SaaS refers to a software application that is stored in one or more remote servers (e.g., in the cloud) and provides one or more services (e.g., angiographic image processing) to remote users. In one embodiment, the computer system 80 includes a monitor or display 82, a computer system 84 including a processor(s) 86, a bus subsystem 88, a memory subsystem 90, and a disk subsystem 92, a user output device 94, a user input device 96, and a communication interface 98. The monitor 82 can include hardware and / or software elements configured to generate a visual representation or display of information. Some examples of the monitor 82 can include familiar display devices such as a television monitor, a cathode ray tube (CRT), a liquid crystal display (LCD), etc. In some embodiments, the monitor 82 can provide an input interface such as in conjunction with touch screen technology.

[0041] The computer system 84 can include familiar computer components such as one or more central processing units (CPUs), memory or storage devices, graphics processing units (GPUs), communication systems, interface cards, etc. As shown, the computer system 84 can include one or more processors 86 that communicate with a number of peripheral devices via a bus subsystem 88. The processor(s) 86 can include commercially available central processing units, etc. The bus subsystem 88 can include the mechanism for permitting inter-component communication among the various components and subsystems of the computer system 84. Although the bus subsystem 88 is shown schematically as a single bus, alternative embodiments of the bus subsystem can utilize multiple busses. Peripheral devices in communication with the processor(s) 86 can include the memory subsystem 90, the disk subsystem 92, the user output device 94, the user input device 96, the communication interface 98, etc. FIG. 2

[0042] The processor(s) 86 can be implemented using one or more analog and / or digital electric or electronic components and can include microprocessors, microcontrollers, application specific integrated circuits (ASICs), field programmable gate arrays (FPGAs), programmable logic, and / or other analog and / or digital circuitry configured to perform the various functions described herein, such as by executing instructions stored in the memory subsystem 90 and / or the disk subsystem 92 or another computer program product. ​

[0043] Memory subsystem 90 and disk subsystem 92 are examples of physical storage media configured to store data. Memory subsystem 90 can include a number of memories including a random access memory (RAM) for volatile storage of program code, instructions and data during program execution, and a read only memory (ROM) for the storage of fixed program code, instructions and data. Disk subsystem 92 can include a number of file storage systems providing persistent (non-volatile) storage for programs and data. Other types of physical storage media include floppy disks, removable hard disks, optical storage media such as compact disk read only memory (CD-ROM), digital video disk (DVD), and bar code, semiconductor memory such as flash memory, read only memories (ROMS), battery- backed-up volatile memory, and networked storage. Memory subsystem 90 and disk subsystem 92 can be configured to store programming and data constructs that provide the functionality or features of the technologies discussed herein (e.g., motion tracking system 120 and spatio-temporal reconstruction system 130 (see FIG. 3 )) discussed herein. Software code modules and / or processor instructions implementing or otherwise providing the functionality described herein can be stored in memory subsystem 90 and disk subsystem 92 for execution by the processor(s) 86. Memory subsystem 90 can be a non-transitory computer readable storage medium.

[0044] User input devices 96 can include hardware and / or software elements configured to receive input from a user for processing by components of computer system 80. User input devices can include all possible types of input devices and mechanisms including a keyboard, a key panel, a touch screen, a touch interface incorporated into a display, audio input devices such as microphones and voice recognition systems, and / or other types of input devices. In various embodiments, user input devices 96 can include a computer mouse, a trackball, a trackpad, a joystick, a wireless remote control, a drawing tablet, a voice command system, an eye tracking system, etc. In some embodiments, user input devices 96 are configured to allow a user to select or otherwise interact with objects, icons, text, etc. that can appear on monitor 82 via commands, motions, or gestures such as clicking a button, etc.

[0045] User output devices 94 can include hardware and / or software elements configured to output information from components of computer system 80 to a user. User output devices can include all possible types of devices and mechanisms for outputting information from computer system 84. These can include a display (e.g., monitor 82), a printer, a touch or force feedback device, an audio output device, etc.

[0046] The communication interface 98 can include hardware and / or software elements configured to provide one-way or two-way communication with other devices. For example, the communication interface 98 can provide an interface between the computer system 84 and other communication networks and devices, such as via an Internet connection.

[0047] FIG. 3 A dataflow diagram is shown in which angiographic data 110 is obtained from a rotating x-ray system 28 at a faster rate than the heart rate. The angiographic data is provided to a motion tracking system 120, which processes the angiographic data to generate an optical flow path provided to a spatio-temporal reconstruction system 130. Once the spatio-temporal reconstruction system 130 receives the optical flow path, a spatio-temporal reconstruction is performed on the unconstrained blood vessel object. These embodiments are described in further detail throughout this application.

[0048] According to example embodiments of the present application, angiographic data (including angiographic image frames) can be obtained. In addition to acquiring the angiographic image frames, additional cardiac signals / data can be simultaneously acquired for use as a cross-correlation target to perform spatio-temporal reconstruction of the vascular pulse wave based on the techniques provided herein. For example, the additional cardiac signals / data can be used as a reference cardiac signal for phase-indexed pixels in the angiographic projections. Exemplary devices for acquiring / providing the reference cardiac signal in the form of a pulse oximetry system and / or an echocardiogram / electrocardiogram (EKG) system or device. In example embodiments, the output from such a device (e.g., an EKG device) can be communicated to the computer system 84 via a communication interface.

[0049] Motion tracking

[0050] According to the present embodiments, optical flow techniques are utilized in conjunction with wavelet and other time-indexing-conserving transform angiography techniques to account for unconstrained cardiac frequency motion and to improve performance for constrained blood vessels and tissue (e.g., brain, etc.).

[0051] Optical flow techniques handle the motion of objects from one image frame to the next. An object (a collection of pixels) moves from one frame to the next, and its displacement is determined, for example, by measuring increments x (changes relative to the x-axis) and y (changes relative to the y-axis). Based on the increment x and y values, a local coordinate system can be warped relative to a global coordinate system. The local coordinate system can be used to reflect the motion of pixels along a path, and the intensity of pixels along the path can be obtained and provided to wavelet transforms or other transforms. Example embodiments of the invention can utilize optical flow techniques to transform a path represented by a simple line (which may be interpolated), which is provided to a transform to become a variable path from one angiography image frame to another, including the locations of blood vessels and other structures. In other words, optical flow techniques can be used to track object motion from one frame to the next.

[0052] To represent optical flow techniques, the following notation is introduced. The two spatial dimensions are represented by x and y, and the temporal dimension by t. The intensity of an angiographic image at location x,y,t is represented as A(x,y,t). An angiographic image frame is a discrete-time frame at location t. i The image is obtained at position t, where i is 1..n, and n is the number of angiography image frames. For a specific object, such as a specific location in a blood vessel, at position t... i The frame is located at x, y, and the object is in the next angiography frame t. i+1 The displacement at point is expressed as Δ x ,Δ y Therefore, using optical flow notation, at position A(x,y,t) i The object at ) is converted to

[0053] A(x,y,t i → A(x+Δ) x,i+1 ,y+Δ y,i+1 ,t i+1 ).

[0054] In consecutive frames t i+2 The position of the object is converted to

[0055] A(x+Δ x,i+1 ,y+Δ y,i+1 ,t i+1 → A(x+Δ) x,i+1 +Δ x,i+2 ,y+Δ y,i+1 +Δ y,i+2 ,

[0056] t i+2 ).

[0057] Optical flow calculations report changes in position for each subsequent frame. The net position at a distance of k frames is from t. ito t i+k The recursion and integration of the position changes of t. The term "recursion" refers to using the flow displacement of the transformed position as the next displacement, and these frame-by-frame displacements are used to construct a trajectory / path, which can be piecewise rather than necessarily following a straight line. Thus, the trajectory of motion adjustment given by the complex wavelet transform of f(t) is derived from the integration of the optical flow along the path. The Gabor wavelet ψ(t) extends isotropically in both positive and negative time directions. Thus, the trajectory of path integration is computed as an ordered concatenation of the integrals of the optical flow in the positive t and negative t directions.

[0058] Consider an example in which between frames t1 and t2, the object is displaced dx1 in the x-dimension and dy1 in the y-dimension; further, between frames t2 and t3, the object is displaced dx2 in the x-dimension and dy2 in the y-dimension. The displacement between frame t1 and frame t3 can be obtained by integrating the displacements between the frame pair t1 and t2 and between frames t2 and t3. Specifically, the integrated displacement in the x-dimension is dx1 + dx2, and the integrated displacement in the y-dimension is dy1 + dy2. This process can be repeated for all image frames.

[0059] Thus, for each image frame, the displacements of a given pixel can be recursively integrated in the forward time direction and the reverse time direction to generate an optical flow trajectory for the given pixel. Using path integration, such as wavelet path integration, can enable compensation for motion in unconstrained vascular objects. For example, the techniques described herein can compensate for motion aliasing due to the expansion and contraction of the heart, such that a beating heart appears stationary / inert in the processed angiogram images.

[0060] As image frames get further from the center of interest, there can be a tendency for motion tracking to drift and become less accurate. This is because the motion trajectory is based on the integration of frame-to-frame displacement. True random errors can be compensated across motion tracking over a large number of frames, but small deviations in the tracking can be amplified by integrating the estimated frame-to-frame displacement into the path trajectory. Thus, when performing motion tracking in the positive or negative direction, a subset of frames can be considered relative to a specified frame that includes a specified object or central object. The specified object or central object can be a pixel. For example, for a specified frame, 3-5 frames can be processed in the positive and negative directions with reference to the specified pixel (e.g., the calculations can become more inaccurate when considering frames that are further away). In one example, an image frame of interest can be selected, and a displacement of a given pixel of the image frame of interest can be determined based on image frames within a number (e.g., five) of image frames of the image frame of interest. Once the motion of the specified frame is determined, the next frame can be selected as the specified frame, and the process is repeated. In this way, motion determination can occur along the time sequence of an angiogram, for example, in a "windowed" approach.

[0061] Any suitable optical flow technique(s) can be used to measure the motion of an object from one image frame to the next. Various techniques can have relative advantages and disadvantages. The appropriate technique for a given scene can facilitate the use of the motion of the object to identify heart rate activity through wavelet angiography.

[0062] Sparse optical flow techniques can be used to measure the motion of objects. Sparse optical flow techniques can enable the computation of motion for only key pixels. The motion of pixels between these key pixels is interpolated from the motion of nearby key pixels. The key pixels can be selected according to criteria such that the corresponding key pixels represent the same part of the same object on adjacent image frames. Examples of criteria for key pixels can be edges, and intersections of edges. Sparse optical flow techniques can include, but are not necessarily limited to, the method described by Lucas and Kanade in "An Iterative Image Registration Technique with an Application to Stereo Vision," Proceedings DARPA Image Understanding Workshop, pp. 121-130, April 1981, which is incorporated by reference herein; the Harris corner adaptation as described by Harris and Stephens in "A Combined Corner and Edge Detector," Alvey Vision Conference, 1988, which is incorporated by reference herein; and / or derivatives of these methods.

[0063] For sparse optical flow techniques, a limited number of object positions are tracked from image frame to image frame, and the movement of intermediate object positions is interpolated. This can involve the assumption of an affine or fractional linear transformation between two images. These techniques include applying a geometric transformation to sequential images by using common object positions on each of the images.

[0064] One example of sparse optical flow techniques depends on the identification of consistent features in two image frames, and involves computing a warp that allows mapping of features in one image frame to the next image frame, for example, as described by Shi and Tomasi in "Good Features to Track," IEEE Conference on Computer Vision and Pattern Recognition, June 1994, which is incorporated by reference herein. Examples of consistent features are corners and edges, which can be identified by any suitable mathematical technique. The warp can be performed by an interpolation method such as linear or cubic spline interpolation.

[0065] Other aspects can use dense optical flow techniques that measure the optical flow of all pixels from image frame to image frame. Examples include, but are not necessarily limited to, variational methods as derived by Horn and Schunck using Lagrangian equations as described in "Determining Optical Flow" (Artificial Intelligence 17, 1981, 185-203), which is incorporated by reference herein. Horn and Schunck describe a method that assumes the luminance of a pixel remains constant from one image frame to the next and determines the spatial distribution of the pixel between the one image frame and the next. Changes in the spatial distribution of the pixel can be interpreted as representing motion of the object. Other methods include A dense optical flow technique described in "Very High Accuracy Velocity Estimation using Orientation Tensors, Parametric Motion, and Simultaneous Segmentation of the Motion Field" (Proceedings of the Eighth IEEE International Conference on Computer Vision, July 2001), which is incorporated by reference herein, is based on the generation of local coordinate polynomials that facilitate the description of spatial changes from image frame to image frame. Still other methods are based on local transforms such as Hilbert transforms to generate monogenic signals and can be generalized to two dimensions to form Riesz transforms. The Riesz transforms and related transforms that occur in the frequency domain can be employed to estimate the local structure tensors of the tissue. This in turn can be used to estimate the optical flow of the biological object from one angiogram image frame to the next. Additionally, there are directional integral transforms such as shearlets, ringlets, and curvelets that can be used for optical flow information. Any of these methods can be adapted for use with the techniques provided herein.

[0066] There are other methods that can include techniques involving deformable registration, such as using convolutional networks and other deep learning / machine learning / artificial intelligence methods, which can be used to estimate optical flow, particularly to compute the motion of the object from one image frame to the next. A suitable example of mathematical deformable registration can include a diffeomorphism, which is derived from the field of topology. An example of a suitable open source software package for diffeomorphisms can be the Python DiPy library. This information, along with the object brightness, can be combined to estimate the motion of the object from one frame to the next. There are implementations of diffeomorphisms that assume that the object being tracked has already been segmented in each of the image frames of interest. The diffeomorphism can then be computed between the segmentation options in one image frame to the next. One example mathematical method that can be suitable for segmenting vascular structures and angiogram images is the Frangi filter, described by Frangi et al. in “Multiscale vessel enhancement filtering,” Medical Image Computing and Computer-Assisted Intervention, 1998, which is incorporated by reference herein.

[0067] One suitable example of a deep learning framework can be the VoxelMorph open source package as described by Balakrishnan et al. in “VoxelMorph: A Learning Framework for Deformable Medical Image Registration,” arXiv: 1809.05231 [cs.CV], September 1, 2019, which is incorporated by reference herein. VoxelMorph employs a deep learning neural network to compute a diffeomorphism between objects in corresponding image frames. In another suitable example of a deep learning framework, a neural network can be employed to interpolate cardiac angiograms at a higher video frame rate from cardiac angiograms captured at a lower rate. A package such as the Depth-Aware Video Frame Interpolation (DAIN) network model described by Bao et al. in “Depth-Aware Video Frame Interpolation,” IEEE Conference on Computer Vision and Pattern Recognition, 2019, which is incorporated by reference herein, can employ a variant of a convolutional neural network for video frame interpolation.

[0068] In other embodiments, steerable filters can be employed to detect local orientations in images. This approach can be used to estimate motion trajectories. Steerable filters can include convolution kernels and can be used for image enhancement and feature extraction. For example, if different objects are moving in different directions, a given image frame can have a set of steerable filters corresponding to different objects in the image. Each direction of motion can be represented by a steerable filter that is encoded and filtered for a particular direction of motion. In yet other examples, any suitable supervised or unsupervised machine learning model (e.g., mathematical / statistical models, classifiers, feedforward, recurrent, or other neural networks, etc.) can be used.

[0069] The above list includes examples of optical flow techniques that can be suitable for the purpose of motion compensated wavelet or time indexed saved angiography. The present embodiments are not necessarily limited to the optical flow techniques referenced above and can include any suitable optical flow technique. For example, Image Displacements[] provided by the environment (as described in Wolfram Research’s “Image Displacements” (Wolfram Language Function, https: / / reference.wolfram.com / language / ref / ImageDisplacements.html (2016)), which is incorporated by reference herein) can be used to perform wavelet angiography with motion compensation by optical flow techniques.

[0070] Furthermore, multiple objects can be tracked, each with a respective local coordinate system. For wavelet computation, the motion of all objects can be inverted (in the bookkeeping sense). In one example, the motion of all pixels can be estimated. In sparse optical flow techniques, the motion of key pixels that can be tracked computationally efficiently can be measured, and the motion of pixels between key pixels can be interpolated. In dense optical flow techniques, the motion of all pixels can be measured. Deep learning systems based on AI object recognition can use methods similar to sparse optical flow techniques. Some deep learning methods can be based on training for object recognition and / or pixel-by-pixel motion. This can be similar to dense optical flow techniques.

[0071] Wavelet Angiography / Spatio-Temporal Reconstruction

[0072] Wavelet Angiography employs complex-valued wavelet transforms to generate spatio-temporal reconstructions of cardiac frequency phenomena in angiograms acquired at rates faster than the cardiac frequency. The wavelet transform is built on the integral.

[0073] For a signal f(t) and a mother wavelet ψ, the complex continuous wavelet transform of f(t) is given by the equation:

[0074]

[0075] where s is a wavelet scale, u is a wavelet translation parameter, and the superscript * denotes complex conjugation. In wavelet angiography, a Gabor wavelet can be chosen for use as the particular wavelet ψ. The Gabor (alternatively known as Morlet) family of ψs provides an explicit balance of frequency and time resolution. The family is based on the equation:

[0076]

[0077] As described above, the signal of the heart rate in the angiogram is developed to improve the sensitivity of the angiographic imaging to arterial and venous anatomy, allowing the identification of patterns of circulation and pathological patterns, such as vascular occlusions and other blood flow states, at lower x-ray dose and / or lower intravascular contrast dose. Additionally, it can allow arterial and venous anatomy separation without the need to navigate a catheter and inject it into a distal arterial tree. The coherence of the heart rate between the circulation subsystems can be developed to allow anatomical identification of arterial and venous anatomy at lower x-ray dose and lower intravascular contrast dose.

[0078] In performing the methods described herein, angiographic data can be recorded using a digital detector device, such as those commercially available as part of scanning devices available from manufacturers such as Philips and Siemens. The digital data is then imported into computer memory. After the angiogram is imported into computer memory (without motion aliasing), a spatiotemporal reconstruction of the heart rate angiographic phenomenon can be obtained. In one example, the spatiotemporal reconstruction can be performed according to the techniques described in U.S. Patent No. 10,123,761, issued November 13, 2018; U.S. Patent Application No. 16 / 784,125, filed February 6, 2020; U.S. Patent Application No. 16 / 784,073, filed February 6, 2020; U.S. Patent Application No. 16 / 813,513, filed March 9, 2020; U.S. Patent Application No. 16 / 832,695, filed March 27, 2020; and / or U.S. Patent Application No. 16 / 841,247, filed April 6, 2020; each of these patents is incorporated herein by reference.

[0079] The angiographic data can be imported into computer memory and reformatted with a processor in the memory to give an array of time signals. A complex-valued wavelet transform is applied by the processor to each pixel-by-pixel time signal, giving an array of wavelet transforms. The pixel-by-pixel wavelet transforms are filtered by the processor for the heart rate. This is done by setting all wavelet coefficients that do not correspond to the heart wavelet scale (in the wavelet art, this term corresponds to the concept of the heart rate) to zero. The pixel-by-pixel wavelet transform data is inverse wavelet transformed by the processor into the time domain and reformatted into pixels in computer storage. Each data element (voxel) in this three-dimensional grid is a complex-valued number.

[0080] Each frame can be rendered by the processor as an image with a luminance-hue color model to represent the complex data in each pixel. The cardiac frequency amplitude is represented as luminance, and the phase is represented as hue. The q images can be rendered by the processor as a motion picture, or they can be stored by the processor as a video file format. For example, one or more images can be provided as a movie video sequence. In one example, object motion can be applied in reverse to all pixels in each image to create a new video sequence in which all objects appear to be stationary. This video can be processed with wavelet computation to identify the cardiac frequency phenomenon. Object motion can then be applied in the forward sense to restore the object motion as a result of the wavelet transform.

[0081] If the wavelet transform has a frequency resolution of, for example, ten image frames, then it can only be necessary to compute motion for adjacent regions of the ten image frames. This can be repeated for each image frame. If there is an error drift in the estimation of motion, then the error from the drift can be limited to the drift that can occur in these ten frames, rather than an aggregate drift from hundreds of frames in an angiogram sequence.

[0082] Any suitable transform that can operate on complex numbers, preserves the time index after transforming into the frequency domain, and is able to extract a spatiotemporal reconstruction of the cardiac frequency angiogram phenomenon is contemplated for use with the present technology.

[0083] FIG. 4A and FIG. 4B Two consecutive angiogram image frames 400A and 400B (one frame apart) are shown, where the angiogram image frame 400A and the angiogram image frame 400B are selected from a pig coronary angiogram taken at 30 Hz (30 Hz is faster than the cardiac frequency). The same location on the same coronary artery (called the left anterior descending artery) is indicated on the angiogram image frames 400A and 400B, respectively, labeled as coronary location 2 and coronary location 4. The spatial displacement of the same coronary location between the angiogram image frames 400A and 400B shows the magnitude of the motion. FIG. 4C An overlay 400C of the angiogram image frames 400A and 400B is shown. Pixel 6 in the overlay 400C is offset, indicating that motion has occurred from the angiogram image frame 400A to the angiogram image frame 400B.

[0084] FIG. 5An example optical flow path for a selected pixel 9 associated with a specific image frame 500 of an optical angiography film is shown. In the example, image frame 500 has a specific pixel of interest within the blood vessel, referred to as optical flow pixel 9. Optical flow trajectories 8 extending from optical flow pixel 9 in two temporal directions are shown. Optical flow trajectory 8 represents the path of the same pixel 9 during a heartbeat. The trajectory represents the integral path of a wavelet transform (e.g., Gabor wavelet transform). This shows motion tracking across the entire angiography sequence as the contrast agent bloat is injected and subsequently dissipates. Optical flow pixel 9 moves at the cardiac frequency; however, the pixel does not follow exactly the same trajectory from one heartbeat to the next due to variations in myocardial contraction and errors from motion tracking. The optical flow path is the path along which wavelet transforms and inverse transforms are performed to generate a spatiotemporal reconstruction.

[0085] This process is repeated for each frame of angiographic image, for example, to generate a sequence of motion-adjusted Gabor wavelet transforms. The Gabor wavelet transform can be filtered at cardiac frequencies and then inversely transformed. For the inverse wavelet transform, only the wavelet scale corresponding to the cardiac frequency is retained.

[0086] Each pixel grid position x, y, t is returned as complex-valued data, which can be rendered as an image using any suitable scheme, such as a luminance-hue model (e.g., where complex-valued amplitudes are rendered as luminance and heart rate phases are rendered as hue), etc. In one example, one or more techniques described in U.S. Patent No. 10,123,761 can be used to render an image.

[0087] Angiographic images are represented by a number of image intensities on a discrete grid based on the number of pixels in the horizontal (x) and vertical (y) directions. However, optical flow trajectories can return changes in object position as fractional pixel values. Therefore, in this embodiment, the discrete angiographic image grid can be converted into a spline interpolation data set that allows access to fractional pixel values ​​via interpolation.

[0088] exist FIGS. 6A-6B The image shows the results of an example motion-compensated wavelet angiography transformation. On the left side (… FIG. 6A The original angiography image frame 600A is shown, and the wavelet angiography image frame 600B on the right is shown. FIG. 6B The image above shows the results after motion processing and spatiotemporal reconstruction. Matching pixel 12 indicates the same location on the same blood vessel as labeled.

[0089] FIG. 7is a flowchart of operations for obtaining and processing data having large-scale cardiac motion according to embodiments of the technology described herein. At operation 710, a series of angiogram image frames is obtained from an angiography study at a rate faster than the cardiac frequency. For example, the angiogram data image frames can be obtained using a rotational x-ray angiography system as shown in FIG. 1A and FIG. 1B may be obtained by a computer in the angiography system or by a separate computer. The angiogram image frames can comprise a time series of angiogram image frames. Each image frame can be associated with a particular time t and can comprise a plurality of pixels. Each pixel can be located at an x-y position in the image frame and have a corresponding brightness or intensity.

[0090] At operation 720, an optical flow technique is applied to the angiogram image frames to generate a plurality of paths corresponding to the displacement of respective pixels from image frame to image frame. For example, motion tracking can be performed by the computer on the image pixels to allow measurements at the cardiac frequency of the vascular pulse in moving cardiac blood vessels, tissue, or organs. The images can represent a global coordinate system and motion can be tracked with respect to a local coordinate system. For example, the moving blood vessels, organs, or tissue can be tracked with respect to a local coordinate system. Thus, the pixels of the images can be transformed to the local coordinate system and a tracking or path can be determined for the transformed pixels to track the location of the pixels over time.

[0091] At operation 730, a spatiotemporal reconstruction of the cardiac frequency angiographic phenomenon is generated by the computer based on the plurality of paths and the corresponding intensities associated with the respective pixels of the paths. For example, the plurality of tracks or paths generated by the computer in operation 720 can be fed into a wavelet transform or other transform that preserves the time index. The pixel-wise transform can then be filtered by the computer for the cardiac frequency. This can be done by setting all transform coefficients that do not correspond to the cardiac wavelet scale (a term in the wavelet domain that corresponds to the concept of the cardiac frequency) to zero. The pixel-wise transform data can then be inverse transformed by the computer into the time domain and reformatted as pixels in computer storage. Each data element (voxel) in this three-dimensional grid can be a complex-valued number.

[0092] At operation 740, the spatiotemporal reconstruction of the cardiac frequency angiographic phenomenon is outputted for display in one or more images. For example, each frame can be rendered by the computer as an image with a brightness-hue color model to represent the complex number data in each pixel. The cardiac frequency amplitude can be represented as brightness and the phase can be represented as hue. The images can be displayed on a screen, stored as image files, and / or printed. The q images can be rendered by the computer as a motion movie or they can be stored by the processor as a video file format.

[0093] The techniques described herein are not necessarily limited to cardiac applications. Other applications can include optical angiography. For example, optical angiography of the retina in a conscious patient can capture saccadic movements of the eye. The techniques described herein can be used to compensate for this type of motion. Other sources of motion that can be compensated for by the techniques include, but are not necessarily limited to, respiratory motion, and spontaneous, conscious or unconscious motion. These techniques can be applied to any suitable target / body part (e.g., blood vessels, organs, etc.) that experiences large scale or unconstrained motion.

[0094] The techniques described herein can be implemented in conjunction with the administration of an effective amount of contrast to visualize the subject. These techniques can be applicable to measure the motion of the subject in intermediate image frames (e.g., image frames that are surrounded by other image frames in a sequence of images) when the motion determination via the optical flow path moves in both forward and reverse directions. However, it should be understood that the techniques described herein can also be applicable to measure the subject motion at the beginning and end of a sequence of images, and can be used with or without contrast.

[0095] The present application can include method, system, device and / or computer program product of any possible technical detail level of integration. The computer program product can include a computer readable storage medium (or media) having computer readable program instructions thereon for causing a processor to carry out aspects of the present application.

[0096] The computer readable program instructions described herein can be downloaded to respective computing / processing devices from a computer readable storage medium (or media), or to external computers or external storage devices via a network (for example, the Internet, a local area network, a wide area network and / or a wireless network). The network can comprise conductive transmission cables, optical transmission fibers, wireless transmission, routers, firewalls, switches, gateway computers and / or edge servers. A network adapter card or network interface in each computing / processing device can receive computer readable program instructions from the network and forward the computer readable program instructions for storage in a computer readable storage medium within the respective computing / storage device.

[0097] Aspects of the present application are described herein with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the application. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer readable program instructions.

[0098] These computer readable program instructions can be provided to a processor of a general purpose computer, special purpose computer, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create means for implementing the functions / acts specified in the flowchart and / or block diagram block or blocks. These computer readable program instructions can also be stored in a computer readable storage medium that can direct a computer, a programmable data processing apparatus, and / or other devices to function in a particular manner, such that the computer readable storage medium having instructions stored therein comprises an article of manufacture including

[0099] The computer readable program instructions can also be loaded onto a computer, other programmable data processing apparatus, or other device to cause a series of operational steps to be performed on the computer, other programmable apparatus or other device to produce a computer implemented process, such that the instructions which execute on the computer, other programmable apparatus, or other device implement the functions / acts specified in the flowchart and / or block diagram block or blocks.

[0100] The flowchart and block diagram in the FIGURE(s) illustrate the architecture, functionality, and operation of possible implementations of systems, methods and computer program products according to various embodiments of the present application. In this regard, each block in the flowchart or block diagrams can represent a module, segment, or portion of instructions, which comprises one or more executable instructions for implementing the specified logical functions (s). In some alternative implementations, the functions noted in the blocks can occur out of the order noted in the FIGURE(s). For example, two blocks shown in succession can in fact be executed substantially concurrently or the blocks can sometimes be executed in the reverse order, depending upon the functionality involved. It will also be noted that each block of the block diagrams and / or flowchart illustrations, and combinations of blocks in the block diagrams and / or flowchart illustrations, can be implemented by special purpose hardware-based systems that perform the specified functions or acts

[0101] The above description is intended to teach any person skilled in the art to whom this application is provided that how to overcome any obvious problems with an implementation of the inventive subject matter recited in the claims based on the patent specification content. The above description is not intended to detail all of those obvious modifications and variations that will become apparent to the skilled artisans on reading the patent specification content. However, all such obvious modifications and variations are intended to be included within the scope of the present application defined by the appended claims. The claims are intended to cover any and all possible combinations of the claimed features with any and all possible steps for accomplishing the intended purpose unless a specific context dictates otherwise.

Claims

1. A method for extracting a heart-rate angiography phenomenon of an unconstrained blood vessel object from an angiography study, the method comprising: obtaining, at a computer, a series of angiography image frames obtained at a rate faster than a heart rate, wherein each image frame comprises a plurality of pixels, and wherein each pixel has a corresponding intensity; applying, at the computer, an optical flow technique to the angiography image frames to generate a plurality of paths corresponding to displacements of respective pixels from image frame to image frame; generating, at the computer, a spatiotemporal reconstruction of a heart-rate angiography phenomenon based on the plurality of paths and the corresponding intensities associated with respective pixels of the paths; and outputting for display of the spatiotemporal reconstruction of the heart-rate angiography phenomenon in one or more images, wherein the method further comprises: recursively integrating, for each image frame, the displacements of a given pixel in a forward temporal direction and a reverse temporal direction to generate an optical flow trajectory of the given pixel.

2. The method of claim 1, further comprising: selecting an image frame of interest; and determining a displacement of a given pixel of the image frame of interest based on image frames within a plurality of image frames of the image frame of interest. applying the optical flow technique comprises: applying a dense optical flow technique that measures optical flow of the plurality of pixels from image frame to image frame; or 3. The method of claim 1, wherein, applying a sparse optical flow technique that tracks a limited number of object locations from image frame to image frame and interpolates movement of intermediate object locations. applying the optical flow technique comprises: determining each path based on a local coordinate system.

4. The method of claim 1, wherein, 5. The method of claim 1, further comprising: displaying the one or more images as a cinematic video sequence.

6. A system, the system comprising: a communication interface configured to obtain a series of angiography image frames obtained at a rate faster than a heart rate, wherein each image frame comprises a plurality of pixels, and wherein each pixel has a corresponding intensity; and one or more processors coupled to the communication interface, wherein the one or more processors are configured to: apply an optical flow technique to the angiography image frames to generate a plurality of paths corresponding to displacements of respective pixels from image frame to image frame; generate a spatiotemporal reconstruction of a heart-rate angiography phenomenon based on the plurality of paths and the corresponding intensities associated with respective pixels of the paths; and output for display of the spatiotemporal reconstruction of the heart-rate angiography phenomenon in one or more images, wherein the one or more processors are further configured to: recursively integrate, for each image frame, the displacements of a given pixel in a forward temporal direction and a reverse temporal direction to generate an optical flow trajectory of the given pixel. the one or more processors are further configured to: select an image frame of interest; and 7. The system of claim 6, wherein, determine a displacement of a given pixel of the image frame of interest based on image frames within a plurality of image frames of the image frame of interest. ​ ​ 8. The system of claim 6, wherein, The one or more processors are further configured to: apply a dense optical flow technique that measures optical flow of the plurality of pixels from image frame to image frame; or apply a sparse optical flow technique that tracks a limited number of object locations from image frame to image frame and interpolates movement of intermediate object locations.

9. The system of claim 6, wherein, The one or more processors are further configured to: determine each path based on a local coordinate system.

10. One or more non-transitory computer-readable storage media encoded with instructions that, when executed by a processor, cause the processor to: obtain a series of angiogram image frames obtained at a rate faster than a heart rate, wherein each image frame comprises a plurality of pixels, and wherein each pixel has a corresponding intensity; apply an optical flow technique to the angiogram image frames to generate a plurality of paths corresponding to displacement of respective pixels from image frame to image frame; generate a spatiotemporal reconstruction of a heart rate angiographic phenomenon based on the plurality of paths and the corresponding intensities associated with respective pixels of the paths; and output for display in one or more images the spatiotemporal reconstruction of the heart rate angiographic phenomenon, wherein the instructions further cause the processor to: for each image frame, recursively integrate the displacement of a given pixel in a forward time direction and a reverse time direction to generate an optical flow trajectory of the given pixel.

11. The one or more non-transitory computer-readable storage media of claim 10, wherein, The instructions further cause the processor to: select an image frame of interest; and determine a displacement of a given pixel of the image frame of interest based on image frames within a plurality of image frames of the image frame of interest.

12. The one or more non-transitory computer-readable storage media of claim 10, wherein, The instructions further cause the processor to: apply a dense optical flow technique that measures optical flow of the plurality of pixels from image frame to image frame; or apply a sparse optical flow technique that tracks a limited number of object locations from image frame to image frame and interpolates movement of intermediate object locations.

Images