Intrinsic contrast optical cross-correlation wavelet angiography

Abstract

A time-series optical image sequence of a patient is obtained at a rate faster than the heart rate, wherein the time-series image sequence captures one or more physical properties of an intrinsic contrast. A cross-correlation signal is obtained from the patient. A wavelet transform analysis is applied to the time-series optical image sequence to generate a spatiotemporal representation of a heart rate phenomenon. The cross-correlation wavelet transform analysis includes performing a wavelet transform on the time-series optical image sequence to obtain a wavelet transform signal, cross-correlating the wavelet transform signal with the cross-correlation signal to obtain a cross-correlation signal, filtering the cross-correlation signal at the heart rate to obtain a filtered signal, and performing an inverse wavelet transform on the filtered signal to obtain a spatiotemporal representation of the time-series optical image sequence. An image of the heart rate phenomenon is generated.

Description

[0001] Cross-reference to related applications

[0002] This application claims priority to U.S. Provisional Application No. 62 / 829,290, filed on April 4, 2019, the contents of which are incorporated herein by reference in their entirety. Technical Field

[0003] The field generally relates to reconstructing spatiotemporal images of moving vessel pulse waves using wavelet mathematics or similar mathematical techniques applied to a sequence of angiographic images acquired at a rate faster than the heart rate, and specifically to reconstructing spatiotemporal images using intrinsic contrast. Background Art

[0004] Angiography is an x-ray study in which a chemical contrast agent is injected into the blood and a series of x-rays is obtained. The injected chemical contrast agent passes through the vascular system as a bolus and from the arterial subsystem through the capillaries to the venous subsystem. The chemical contrast agent blocks the passage of x-rays, causing the geometry of the passing x-rays to produce an image when it reaches a two-dimensional x-ray detector array. An example of an intravenously administered optical chemical contrast agent is indocyanine green (ICG). The chemical contrast agent is contained within the vessel wall, so its spatial distribution corresponds to the structure within the vascular lumen. Blood in the systemic circulation flows in a unidirectional series from the heart to the arteries, through the capillaries to the veins, and then back to the heart. Therefore, angiography can be used to distinguish the characteristics of the arterial and venous subsystems. Therefore, the relative timing of images within an angiographic image sequence conveys information about the vascular anatomy and physiology of a particular circulatory subsystem. Angiographic images based on the injection of chemical contrast agents generally show spatial resolution of vascular structures (such as blood vessels) from non-vascular tissues because the vessel wall forms a sharp boundary of the chemical contrast agent. Spatial resolution can be increased by increasing the injected dose of chemical contrast agents. However, increasing the dose often produces morbid side effects, such as increasing the risk of renal damage, or, for patients with impaired renal function, toxic side effects. Thus, intravenous administration of chemical contrast agents has many disadvantages, including toxicity, and is generally not suitable for individuals with renal disease or other medical conditions.

[0005] U.S. Patent No. 10,123,761, which is incorporated herein by reference in its entirety, teaches the use of cross-correlated wavelet mathematics applied to angiographic images acquired at a rate faster than the heart rate to reconstruct cine images of moving blood vessel pulse waves. However, this method is based on capturing data about heart rate variations and motion using structures containing contrast agents injected intravenously.

[0006] Other imaging techniques are known in the art. For example, the Euler magnification method taught by Wu in US9811901 and US9324005 has been used to produce a magnified visual effect. These techniques rely on mathematical operations based on Taylor expansions that include magnification terms. The procedures taught by Wu require the user to select a spatial frequency cutoff filter and a magnification factor α. However, these techniques are different from the techniques provided in the present application and do not provide a method for reconstructing heart rate phenomena in angiographic data using intrinsic contrast.

[0007] Other imaging techniques, such as optical coherence tomography, have been used to generate three-dimensional images of limited-range structures of tissues, including vascularized tissues. This technique can be used to inject chemical contrast agents, such as fluorescein, intravascularly to generate three-dimensional optical angiograms. As taught by Chen (US6549801 and US7359062) and others, the travel of red blood cells produces a Doppler effect on the wavelength of light used in optical coherence tomography.

[0008] While a variety of imaging techniques are available, there is a need for imaging techniques that can reconstruct heart rate phenomena in angiographic data without the need for intravenous administration of contrast agents and / or with improved patient safety conditions. Summary of the invention

[0009] Embodiments of the present invention relate to systems, methods, and computer-readable media for reconstructing heart rate phenomena in angiographic data using an intrinsic contrast. A time-series optical image sequence of a patient is obtained by an optical system at a rate faster than the heart rate, wherein the time-series image sequence captures one or more physical properties of the intrinsic contrast. A cross-correlation signal is obtained from the patient. A cross-correlation wavelet transform analysis is applied to the time-series optical image sequence to produce a spatiotemporal representation of the heart rate phenomenon. The cross-correlation wavelet transform analysis may include performing a wavelet transform on the time-series optical image sequence to obtain a wavelet transform signal, cross-correlating the wavelet transform signal with the cross-correlation signal to obtain a cross-correlation signal, filtering the cross-correlation signal at the heart rate to obtain a filtered signal, and performing an inverse wavelet transform on the filtered signal to obtain a spatiotemporal representation of the time-series optical image sequence. An image of the heart rate phenomenon is generated from the spatiotemporal representation.

[0010] In one aspect, the spatiotemporal reconstruction is displayed in one or more reconstructed images.

[0011] In another aspect, a simultaneously measured cardiac signal is obtained and cross-correlated with the wavelet transformed signal.

[0012] In one aspect, the images are obtained by optical microscopy, near infrared microscopy, ultrasound or Doppler technology.

[0013] In another aspect, the image is obtained by imaging a body having a vascular structure.

[0014] In one aspect, intrinsic contrast includes thermal radiation, optical reflectance, or autofluorescence.

[0015] In another aspect, the reconstructed angiogram is provided as a cine video sequence.

[0016] In one aspect, a second temporal image sequence with an intravenously injected chemical contrast agent is acquired and a second spatiotemporal representation of the images obtained from the intravenously injected chemical contrast agent is generated. The spatiotemporal representation of the images obtained with the intrinsic contrast is paired with the second spatiotemporal representation of the images obtained with the intravenously injected chemical contrast agent. The second spatiotemporal representation of the images obtained with the intrinsic contrast is calibrated based on the pairing.

[0017] Other objects and advantages of these techniques will be apparent from the description and drawings. BRIEF DESCRIPTION OF THE DRAWINGS

[0018] Example embodiments of the present invention are described with reference to the accompanying drawings, in which:

[0019] Figure 1A and 1B is a schematic diagram of an example system for optical angiography with intrinsic contrast in accordance with aspects of the present disclosure. Figure 1A A system for acquiring visible light and near infrared signals in an operating room setting is shown. Figure 1B A mobile system is shown.

[0020] Figure 2 The use of an indocyanine green contrast agent administered as an intravascular contrast agent according to aspects of the present disclosure is shown. Figure 1A Visible and near-infrared angiographic images acquired with the system.

[0021] Figure 3 The use of intrinsic contrast in the near infrared spectroscopy rather than intravascularly administered chemical contrast agents is shown in accordance with aspects of the present disclosure. Figure 1A Visible and near-infrared angiographic images acquired with the system.

[0022] Figure 4 Visible wavelength intrinsic contrast optical angiography for visualization within vascular structures of the skin and face was compared to chemical intravascular contrast x-ray angiography of the skull in accordance with aspects of the present disclosure.

[0023] Figure 5 is a block diagram of a computer system or information processing device that may be used with aspects of the present disclosure.

[0024] Figure 6 is a flow chart illustrating a technique for intrinsic contrast optical angiography according to aspects of the present invention. DETAILED DESCRIPTION

[0025] Current technology focuses on the spatiotemporal reconstruction of heart rate phenomena using intrinsic contrast and cross-correlation wavelet angiography. As used herein, the term "cross-correlation wavelet angiography" refers to the reconstruction of spatiotemporal images of moving blood vessel pulse waves by wavelet mathematics applied to a sequence of angiographic images acquired at a rate faster than the heart rate, for example, as described in U.S. Pat. No. 10,123,761, which is incorporated herein by reference in its entirety.

[0026] In other aspects, other mathematical techniques can be used to reconstruct a spatiotemporal image of a moving vessel pulse wave with an intrinsic contrast. For example, the intrinsic contrast can be performed using other mathematical computation techniques, including windowed Fourier transforms. In yet other examples, the intrinsic contrast can be used with other techniques involving bandpass filters and amplification, as described in co-pending patent application Ser. No. 16 / 832,695, filed on Mar. 27, 2020, which is incorporated herein by reference in its entirety.

[0027] The present technique allows for spatiotemporal reconstruction using intrinsic contrast rather than intravascularly administered contrast agents. This approach improves safety and is suitable for patients with renal disease. Optionally, the present technique can be used to perform intrinsic contrast cross-correlation wavelet angiography and intravascularly administered contrast agent cross-correlation wavelet angiography. Another option is to perform intrinsic contrast cross-correlation wavelet angiography in combination with conventional angiography using existing techniques.

[0028] As used herein, the term "intrinsic contrast" refers to a property of the circulatory system that can be recorded without the administration of chemical contrast agents and is localized to the circulatory system. Such properties have a measurable distribution that can be used to form a spatiotemporal reconstructed image. Examples of physical properties include changes in thermal radiation (detectable in the infrared and near infrared spectral electromagnetic ranges), changes in optical reflectance in the visible electromagnetic spectrum (such as at a wavelength of 640 nm, where the optical difference between oxyhemoglobin and deoxyhemoglobin is maximized), changes in thermal emission (see, e.g., U.S. Pat. No. 5,637,871 and U.S. Pat. No. 3,335,716), and changes in autofluorescence (where illumination in the near infrared spectrum causes porphyrins and other molecules in the blood to fluoresce in the near infrared spectrum), changes in optical tissue motion, or changes in any other physiological property that reflects vascular anatomy and physiology and can be imaged.

[0029] Given that the heart delivers blood to the body as a series of pulsatile volumes, intrinsic contrast (similar to intravenous contrast) varies with heart rate. Furthermore, this variation may be greater within blood vessels because the vessels are connected to the heart by a continuous column of fluid. Therefore, heart rate variations present within the spatial distribution of intrinsic contrast may be a source of added spatial resolution because the heart rate occurs within or near the vessels because the blood is a continuous column of fluid connected to the ventricles. In some aspects, the heart rate of a human patient may be in the range of 20-200 beats per minute.

[0030] The heart beat produces heart rate variations in the intrinsic contrast and increases the signal-to-noise ratio in the imaging of the intrinsic contrast, thereby providing a strategy for improved spatial resolution of the intrinsic contrast in the derived images. In addition, the arterial circulation subsystem and the venous circulation subsystem may contain vascular pulse waves that exhibit arteriovenous phase locking at the heart rate. This provides a strategy for resolving the contradiction between the arterial circulation subsystem and the venous circulation subsystem.

[0031] Therefore, the internal characteristics of the body or the characteristics with internal contrast reflect the vascular anatomy and physiological changes that can be captured via imaging. The present technology uses intrinsic contrast to apply cross-correlation wavelet angiography to movie optical data. Under stable physiological conditions, the intrinsic contrast does not change between one cardiac cycle and the next, nor does it provide a bolus travel timing mechanism to distinguish the arterial circulation subsystem from the venous circulation subsystem (although this distinction can be obtained from the relative oxyhemoglobin and deoxyhemoglobin signals at 640nm). Although the intrinsic contrast does not provide a bolus travel mechanism to separate the venous circulation subsystem from the arterial circulation subsystem, the arterial path and the venous path are not synchronized with each other, and this phase difference can be used as a standard, thereby allowing the intrinsic contrast angiography to separate the arterial circulation system from the venous circulation system in the intrinsic contrast optical angiography. Therefore, rather than relying on bolus travel timing, the identification of selected anatomical and / or physical structures combined with filtering of structures pulsating in phase and / or complementary phase can distinguish the arterial subsystem from the venous subsystem.

[0032] With intrinsic contrast, the signal difference between the foreground and background within the vessel is smaller, so the spatial resolution of the intrinsic contrast is reduced compared to angiography with injected chemical contrast agents. Therefore, angiographic images using intrinsic contrast may have less temporal and spatial resolution than images obtained from injected chemical contrast agents. However, with intrinsic contrast, there is no toxicity associated with the administration of chemical contrast agents. The present technology provides methods for spatiotemporal reconstruction of vascular pulse waves from movie images of these intrinsic contrast phenomena obtained at a rate faster than the heart rate by cross-correlation wavelet methods. In various aspects, intrinsic contrast cross-correlation wavelet angiography can be calibrated to chemical contrast cross-correlation wavelet angiography.

[0033] In other aspects, the signal from the intrinsic contrast can be cross-correlated with the external timing signal. The cross-correlation wavelet method can use the cardiac signal as the cross-correlation. For example, based on an external cardiac signal (e.g., a time trace of an electrocardiogram or pulse oximeter), according to the sampling theorem of Nyquist, Shannon, and Kotelnikov, angiographic and cardiac signal data can be collected at a faster rate than the heart rate using the intrinsic contrast and operated on by complex-valued cross-correlation wavelet mathematics.

[0034] In other aspects, a measured or estimated heart rate signal may be used, where the shape of the heart rate curve is known / assumed to be constant, and the intervals between heart beats are the same.

[0035] In other aspects, a calibration may be performed where intrinsic contrast cross-correlation wavelet angiography is performed simultaneously with conventional angiography using an injected intravascular contrast agent. For example, x-ray angiography may be performed where a contrast agent is injected into a patient to attenuate the passage of x-rays, which is performed simultaneously with intrinsic contrast cross-correlation wavelet angiography. For example, optical contrast angiography may be performed where a chemical agent such as ICG is injected into a patient to allow detection using an optical system with a near infrared filter. Intrinsic contrast wavelet angiography may be performed for a period of time before and after the passage of an injected vascular optical contrast agent to calibrate the intrinsic contrast wavelet angiography with the intravascular injected contrast agent wavelet angiography.

[0036] In various aspects, a calibration step may be performed such that reconstructed cine images of spatiotemporal heart rate phenomena using intrinsic contrast are compared with cine images obtained from conventional angiography using intravascular administration of chemical contrast agents. Imaging methods that may be used for conventional chemical contrast angiography may include optical (including near infrared bands) and / or fluorescent x-ray.

[0037] Initially, a calibration chemical contrast angiography may be performed in all subjects who subsequently undergo intrinsic contrast cross-correlation wavelet angiography. However, in other aspects, the chemical contrast angiography portion of the procedure may be reduced and eventually eliminated by determining the agreement between intrinsic contrast and conventional chemical contrast angiography.

[0038] The calibration examples listed herein relate to x-ray angiography and infrared optical angiography, but other calibration angiography techniques may be employed, such as those employing magnetic resonance imaging and computed tomography. Optical coherence measurements (with the Doppler effect) at sufficiently high speeds may also be used to acquire optical coherence image measurements at a rate faster than the heart rate. In this example, the amount of data captured from Doppler optical coherence tomography and from optical coherence tomography of tissue motion may be used as the computational basis for the spatiotemporal reconstruction of a cross-correlation wavelet computation method of cardiac frequency phenomena.

[0039] Any suitable imaging technique may be used for intrinsic contrast cross-correlation wavelet angiography, including but not limited to optical, near infrared, ultrasound, Doppler (optical or acoustic), and visible light. In some aspects, Doppler data may include wavelength shifts associated with the movement of red blood cells toward or away from the imaging component. Doppler data may be preprocessed to account for this wavelength shift, thereby extracting and processing information related to heart rate based on the techniques provided in U.S. Patent No. 10,123,761. For non-Doppler methods, changes in tissue echogenicity may be a form of intrinsic contrast. For example, the amount of blood in a given volume element may change in echogenicity as the heart rate changes.

[0040] In an intrinsic contrast cross-correlation wavelet angiography calibration embodiment, a chemical contrast agent is injected into a blood vessel and images are recorded at a rate faster than the heart rate. These movie images, including the intrinsic contrast optical images and the intravascular injection chemical contrast agent angiography images, are acquired close in time or simultaneously and can provide further amplification of signals that are opposite to either method alone. The intrinsic contrast and chemical contrast agent angiography movie images can be cross-correlated wavelet reconstructed to reveal spatiotemporal heart rate phenomena. The cross-correlation wavelet angiography from the chemical optical contrast agent wavelet angiography can be compared and contrasted with the reconstructed wavelet angiography from the intrinsic contrast wavelet optical angiography to calibrate the intrinsic contrast wavelet angiography based on the basic vascular physiology revealed by the chemical contrast agent optical wavelet angiography.

[0041] Heart rate changes with intrinsic optical contrast agents are optical manifestations of vascular phenomena that can be measured without intravascular injection of chemical contrast agents and are present in various vascularized bodies. Optical intrinsic contrast agents include, but are not limited to, optical signatures of tissue motion measured at visible wavelengths, heat emitted by blood flow and metabolism, and hemoglobin in its oxygenated and deoxygenated states. Intrinsic optical contrast agents measure in the visible and near infrared portions of the electromagnetic spectrum. The activity of the heart produces heart rate changes in the intrinsic optical contrast agents, and the underlying heart rate changes of the intrinsic contrast agents can be visualized using cross-correlation wavelet spatiotemporal reconstruction.

[0042] These techniques can be performed using standard amounts of chemical contrast, reduced amounts of chemical contrast, or no chemical contrast, and unlike bolus methods, the intrinsic contrast signal does not decay over time. In the following examples, visual display of the operating field on the monitor display is optional.

[0043] Figure 1A An example system 1000 for performing intrinsic contrast cross-correlation wavelet angiography is shown. System 1000 includes an operating microscope 3, which is configured to collect light from a surgical field of view 1 of a patient and split the light into two paths: a visible light path and a near infrared (NIR) light path. For example, light from the operating field 1 including the patient's view can be collected by a lens 4 and directed to a beam splitter 2. In this example, the beam splitter splits the visible light into optical paths. One path including visible light can be directed by the beam splitter 2 into a magnifying binocular 1010 for viewing by the surgical surgeon. Although the examples provided herein refer to binocular / stereoscopic viewing systems, any suitable viewing or display device can be used, including monocular systems, display screens, etc. Another path can be directed by the beam splitter 2 via an NIR filter 1020 to a camera 1030 (e.g., a charge coupled device (CCD) camera). Camera 1030 can convert the received signal into a digitized form that can be provided to a computing system 1040. In some aspects, the CCD camera can have a frame rate of 30 frames per second. In other aspects, the frame rate can vary between 30 and 240 frames per second. In other aspects, the frame rate can vary between 10 and 60 frames per second. In general, the camera will be sensitive to the wavelength of the heart rate phenomenon being studied (e.g., near infrared for thermal variations in heart rate, hemoglobin peaks for arterial and venous blood volume variations, etc.). In an example embodiment, the CCD camera may have an ISO of 50 to 5400 or higher. Preferably, the CCD camera will display a linear signal response over a large dynamic range to capture the sinusoidal behavior of the heart rate. In an example embodiment, the camera may have a minimum spatial resolution of 640×480 pixels at 1× to 10× spatial magnification power.

[0044] The computing system 1040 can process the received signals based on the intrinsic contrast cross-correlation wavelet angiography technology provided herein to generate an intrinsic contrast angiography map on the display screen 1050. The operating microscope 3 can be optionally mounted on a mounting system 5 above the operating field of view. For example, the mounting system 5 may include a rod 5a extending upward from the console to hold the computing system 1040, a rotatable arm 5b extending laterally from the upper end of the rod, and a microscope bracket 5c suspended from the end of the arm. The operating microscope and mounting system described herein are shown only by way of example, and the present technology is applicable to any operating microscope and / or mounting system. In some aspects, visible light can also be provided to the CCD camera 1030, where the visible light is processed to be displayed on the display screen 1050. An external cardiac device 1060 (e.g., EKG, pulse oximeter, etc.) can be used to record external cardiac signals for cross-correlation. In some cases, there may be an optional external light source 1070, or in the case of ultrasound, there may be an ultrasound transmitter 1075. It will be appreciated that intrinsic contrast angiograms obtained using the systems described herein may be displayed by the computing system 1040, stored in non-volatile memory, and / or printed.

[0045] Figure 1B Another embodiment of a system for performing intrinsic contrast cross-correlation wavelet angiography is shown, wherein the intrinsic contrast technique is performed using a camera from a portable device (e.g., a mobile phone, tablet, etc.). In this example, light from the diagnostic field of view 1110 is recorded by a portable device with video recording capabilities (e.g., recording images at a rate faster than the heart rate). An external cardiac device 1120 may also be present for recording the cross-correlation signal. The video image and the cross-correlation signal may be transmitted wirelessly (or by wire) to a computer system 1140 to obtain a spatiotemporally reconstructed heart rate signal and display the signal on a display screen 1150. Alternatively, processing and / or display may be performed by the portable device 1130. In some cases, an optional external light source 1160 may be present, or in the case of ultrasound, an ultrasound transmitter 1165 may be present.

[0046] The external cardiac signal as a cross-correlation signal provides a method to improve the signal-to-noise ratio because the signal generated by the artifact source does not change with the heart rate. Therefore, the cross-correlation wavelet technique of US Pat. No. 10,123,761 can be applied in the context of the present technique, where the external cardiac signal is used as a cross-correlation.

[0047] Figure 2An example of optical angiography using injection of intravascular contrast agent is shown. In this example, the surface of a human brain exposed in a surgical area 1 under clinical craniotomy is presented while the patient is under anesthesia. During craniotomy, a clamp is placed at the neck of a cerebral aneurysm to reduce the risk of aneurysm rupture again. During optical angiography, the field of view is illuminated with near-infrared light, and the intravascularly administered chemical contrast agent (e.g., indocyanine green (ICG), which emits light in the near-infrared spectrum) absorbs light of this wavelength and releases light in the near-infrared wavelength range. The light from the exposed brain surface in the surgical area 1 enters a surgical microscope 1000 having a beam splitter 2.

[0048] In this example, unfiltered light is recorded by a video camera or CCD camera 1030 to produce a visible video 3 recording of an angiographic field in the visible spectrum. In this example, selected features in the visible video 3 are annotated for clarity, including the frontal lobe (F), temporal lobe (T), retractor (R), aneurysm neck (neck), aneurysm clip (clip), and aneurysm parent artery (parent artery). The same beam splitter 2 sends light through a near infrared filter to a separate video camera or CCD camera and computer system, which captures near infrared video 4 including near infrared optical angiography, wherein the near infrared signal is generated from a bolus of chemical contrast agent delivered through the vascular system. For example, as shown in U.S. Patent No. 10,123,761, the near infrared video 4 undergoes a transformation to produce a cross-correlated wavelet angiographic reconstruction 5 of spatiotemporal phenomena occurring at the heart rate. The heart rate phenomena in the cross-correlated wavelet angiographic reconstruction 5 can be rendered at the pixel level according to a color model legend 6, which includes amplitude as brightness and phase as hue. Using this color model, pixels with no cardiac frequency activity are rendered black, while pixels with larger amplitudes are rendered bright. The phase of cardiac frequency activity cycles through the entire range of [-π, π] and is rendered using a periodic hue-based color bitmap. Any suitable color model (e.g., grayscale, brightness, etc.) may be applied to any cross-correlation wavelet angiogram presented in this application.

[0049] Figure 3 An example of optical angiography based on intrinsic contrast collected in the near-infrared spectrum rather than by injection of chemical intravascular contrast agents is shown. Figure 3 The elements in Figure 2 The elements in are similar, except that the cross-correlation wavelet angiography reconstruction 11 is based on angiography with only intrinsic contrast. (Near-infrared video 10 shows no signal because near-infrared chemical contrast agent was not administered intravascularly). The cross-correlation wavelet angiography technique is able to distinguish moving vascular pulse waves without the use of chemical contrast agent ICG. This effect may be caused by the autofluorescence of tissue under near-infrared light stimulation.

[0050] In this example, the exposed human brain surface in the surgical area 7 under the clinical craniotomy state is presented. The intrinsic contrast in the near-infrared optical range exists due to the combination of infrared thermal emission and light reflection of oxygenated hemoglobin and deoxygenated hemoglobin in the near-infrared optical range. Light from the exposed brain surface in the surgical area 7 enters a surgical microscope 1000 with a beam splitter 8. The unfiltered light is recorded by a video camera or a CCD camera to generate a visible video 9 record of the angiography field in the visible spectrum via a computing system 1040. For clarity, important features in the visible video 9 are annotated. For example, the same beam splitter 8 sends light to another separate video recording device through a near-infrared filter 24, which captures a near-infrared video 10 including intrinsic contrast near-infrared optical angiography. (In other aspects, the near-infrared signal can be digitized and provided to the computer system 1040 for display on a display screen). The near-infrared signal emitted by the exposed brain surface is less than the signal emitted by the administration of the intravascular chemical contrast agent ICG, and therefore, the near-infrared video 10 image is displayed as black. However, even though the video appears dark, the signal from the intrinsic contrast is still present and the pixels have organized the heart rate variations, allowing the pulsating intrinsic contrast to be distinguished from the background signal. As taught in U.S. Pat. No. 10,123,761, the spatiotemporal distribution of the heart rate phenomenon is detected and rendered into a movie image by cross-correlation wavelet angiography reconstruction 11. The figure shows an example of using intrinsic contrast cross-correlation wavelet angiography.

[0051] As previously described, the wavelet processing techniques disclosed in U.S. Pat. No. 10,123,761 can be used to generate intrinsic contrast angiograms. In the absence of motion aliasing, operations may include: importing angiographic data consisting of data consisting of n×m pixels×q frames into a computer memory; reformatting the angiographic data using a processor to generate an array of n×m time signals, each with a sample length of q; applying a complex-valued wavelet transform to each pixel-level time signal by the processor to generate an array of n×m wavelet transforms; filtering the pixel-level wavelet transforms for cardiac frequency by the processor; performing an inverse wavelet transform on the pixel-level wavelet transform data into the time domain by the processor and reformatting the format into q frames of n×m pixels; and rendering each frame by the processor as an image having a brightness-hue-color model to represent a complex reference in each pixel.

[0052] In the case of motion aliasing and cross-correlation, operations may include: performing a high-frequency resolution wavelet transform on the angiography time-intensity curve by a processor; then re-formatting the angiography data into an array of n×m time signals, each sampled of length q, by the processor; performing a high-time wavelet transform on each pixel-by-pixel level time signal by the processor to generate an n×m array of high-time resolution wavelet transforms; cross-correlating in the wavelet domain, performing a high-time resolution wavelet transform on each pixel-by-pixel level by performing a single high-frequency resolution wavelet transform on the entire angiography time-intensity curve by the processor; performing an inverse wavelet transform on the n×m array of cross-correlation signals in the wavelet domain to generate an n×m array of time-domain time signals each of length q by the processor; and re-formatting the n×m time signals of length q into q frames of n×m pixels, each pixel being complex valued, by the processor. Additional details may be found in U.S. Patent No. 10,123,761, which is incorporated herein by reference as previously noted.

[0053] Figure 4 An example of optical angiography of the human face based on intrinsic contrast obtained from the visible spectrum is shown. A visible spectrum camera recording images of the human face shows that the same cross-correlation wavelet algorithm can capture heart rate by capturing small heart rate motions in the head and facial tissues caused by the pulsating kinetic energy of blood pumping through the facial vessels. Thus, intrinsic contrast can be used to reconstruct a cine image of heart rate variations using cross-correlation wavelet mathematics of images acquired at a rate faster than the heart rate. Thus, by using intrinsic contrast in optically imaged tissue, a quantitatively accurate measurement of heart rate phenomena can be formed.

[0054] In various aspects, cross-correlation wavelet mathematics with independently measured cardiac signals can be used to optically image heart rate phenomena in those areas that participate in the circulatory system, including the skin. In this example, a chemical contrast x-ray angiogram of the skull is used to calibrate and facilitate the interpretation of an intrinsic contrast optical angiogram near the same anatomical structure. For example, a subject wears a finger pulse oximeter 12 to non-invasively generate an external cardiac signal 13 as a cross-correlation for a cross-correlation wavelet reconstruction 15. A video clip 14 of a human face in visible wavelengths is acquired to generate an intrinsic contrast optical facial angiogram 17 based on the cross-correlation wavelet reconstruction 15. After calculations by the cross-correlation wavelet reconstruction 15, a reconstruction of the heart rate phenomenon is presented to the right of the intrinsic contrast optical facial angiogram 17.

[0055] The figure shows an x-ray angiogram of a human head 16 in front and back projections with intravascular administration of chemical contrast into the right internal carotid artery. As a calibration method, a conventional x-ray angiogram with intravascular administration of chemical contrast is shown on the left side of the intravascular chemical contrast x-ray head angiogram and cross-correlation wavelet reconstruction 16. After processing through the cross-correlation wavelet reconstruction 15, the reconstructed heart rate phenomenon is presented on the right side of the intravascular chemical contrast x-ray head angiogram and cross-correlation wavelet reconstruction 16. The figure shows the intrinsic contrast optical angiogram and cross-correlation wavelet reconstruction 14, the intravascular chemical contrast x-ray head angiogram and cross-correlation wavelet reconstruction 16 and their cross-correlation wavelet reconstruction in two-dimensional images, but it will be understood that these are example frames shown for illustrative purposes from a movie sequence of images acquired at a rate faster than the heart rate.

[0056] The embodiments herein refer to x-ray angiography and infrared optical angiography, but other calibrated angiography techniques may be employed, such as those employing magnetic resonance imaging and computed tomography.

[0057] The examples provided herein demonstrate intrinsic contrast optical cross-correlation wavelet angiography of the brain surface and skin. These techniques can be applied to other vascularized organs (such as the retina) or other open surgical procedures.

[0058] Reference now Figure 5 , showing a computer system or information processing device 80 (e.g., Figure 1A Computer system 1040 or Figure 1B 140 in FIG. 1 , which may be incorporated into an intrinsic contrast angiography imaging system (such as Figure 1A System 1000 or Figure 1B The system 1000 in the embodiment of the present invention can be used to provide enhanced functionality or as a stand-alone device for extracting heart rate phenomena from angiographic data. In one embodiment, the computer system 80 includes a monitor or display 82, a computer 84 (which includes (multiple) processors 86, a bus subsystem 88, a memory subsystem 90, and a disk subsystem 92), a user output device 94, a user input device 96, and a communication interface 98. The monitor 82 may include hardware and / or software elements configured to generate a visual representation or display of information. Some examples of the monitor 82 may include familiar display devices such as television monitors, cathode ray tubes (CRTs), liquid crystal displays (LCDs), etc. In some embodiments, the monitor 82 may provide an input interface, such as in combination with touch screen technology.

[0059] The computer 84 may include familiar computer components, such as one or more central processing units (CPUs), memory or storage devices, a graphics processing unit (GPU), a communication system, interface cards, etc. Figure 5 As shown, the computer 84 may include one or more processors 86 that communicate with a plurality of peripheral devices via a bus subsystem 88. The (multiple) processors 86 may include commercially available central processing units, etc. The bus subsystem 88 may include a mechanism for enabling the various components and subsystems of the computer 84 to communicate with each other as intended. Although the bus subsystem 88 is schematically shown as a single bus, alternative embodiments of the bus subsystem may utilize multiple bus subsystems. Peripheral devices that communicate with the (multiple) processors 86 may include a memory subsystem 90, a subsystem 92, a user output device 94, a user input device 96, a communication interface 98, etc.

[0060] The memory subsystem 90 and the subsystem 92 are examples of physical storage media configured to store data. The memory subsystem 90 may include multiple memories including random access memory (RAM) for volatile storage of program code, instructions, and data during program execution, and read-only memory (ROM) for storing fixed program code, instructions, and data. The subsystem 92 may include multiple file storage systems that provide permanent (non-volatile) storage for programs and data. Other types of physical storage media include floppy disks, removable hard disks, optical storage media (such as CD-ROMs, DVDs, and bar codes), semiconductor memories such as flash memory, read-only memory (ROM), battery-backed volatile memory, network storage devices, etc. The memory subsystem 90 and the subsystem 92 may be configured to store programming and data structures that provide the functions or features of the technology discussed herein. Software code modules and / or processor instructions that implement or otherwise provide functionality when executed by (multiple) processors 86 may be stored in the memory subsystem 90 and the subsystem 92.

[0061] The user input device 94 may include hardware and / or software elements configured to receive input from the user for the components of the computer system 80 to process. The user input device may include all possible types of devices and mechanisms for inputting information to the computer system 84. These may include keyboards, keypads, touch screens, touch interfaces incorporated into displays, audio input devices (such as microphones and voice recognition systems) and other types of input devices. In various embodiments, the user input device 94 may be implemented as a computer mouse, trackball, tracking pad, joystick, wireless remote controller, drawing board, voice command system, eye tracking system, etc. In some embodiments, the user input device 94 is configured to allow the user to select objects, icons, texts, etc. that may appear on the monitor 82 via commands, motions or gestures (such as clicking buttons, etc.) or otherwise interact with objects, icons, texts, etc. that may appear on the monitor 82.

[0062] User output devices 96 may include hardware and / or software elements configured to output information from components of computer system 80 to a user. User output devices may include all possible types of devices and mechanisms for outputting information from computer 84. These may include displays (e.g., monitor 82), printers, touch or force feedback devices, audio output devices, etc.

[0063] The communication interface 98 may include hardware and / or software elements configured to provide one-way or two-way communications with other devices.

[0064] For example, communication interface 98 may provide an interface between computer 84 and other communication networks and devices (such as via an Internet connection).

[0065] According to embodiments of the present invention, it should be appreciated that in addition to acquiring angiographic images, additional cardiac signals / data may also be simultaneously acquired to serve as cross-correlation targets to perform spatiotemporal reconstruction of vascular pulse waves based on the techniques provided herein. For example, cardiac signals / data may be used as reference cardiac signals for phase-indexed pixels in angiographic projections.

[0066] Figure 6 The high-level operation of the technology provided herein is shown. At operation 710, a time-sequential image sequence obtained at a rate faster than the heart rate is acquired, wherein the image sequence captures one or more physical characteristics of the intrinsic contrast. At operation 720, a wavelet transform analysis is applied to the optical image sequence to produce a spatiotemporal representation of the heart rate phenomenon. At operation 730, an image of the heart rate angiographic phenomenon is generated from the spatiotemporal representation.

[0067] Beneficially, embodiments provided herein include systems, methods, and computer-readable media for spatiotemporal reconstruction of heart rate phenomena in angiographic data using intrinsic contrast. These techniques can be applied together with a hardware system designed to obtain angiographic images, particularly angiographic systems, to obtain images of patients. Compared to existing angiographic methods, these techniques provide improvements in the art, i.e., allowing visualization of spatiotemporal heart rate phenomena in the absence of chemical contrast agents. Additionally, the signal does not decay or decay over time, allowing for continued visualization of spatiotemporal heart rate phenomena, which may be helpful in surgical procedures or continued monitoring of patient status.

[0068] The heart rate variations include a source of information in the intrinsic contrast that increases the signal-to-noise ratio when performing a spatiotemporal cross-correlation wavelet reconstruction of the heart rate variations in the intrinsic contrast. In addition, the embodiments presented herein utilize the arteriovenous phase-locking phenomenon to distinguish between the arterial and venous circulation subsystems in the cine images by analyzing their consistent heart rate phase relationship.

[0069] These techniques can be used to screen for medical abnormalities, such as blockages in limb or head arteries, thereby selecting patients for further screening. In various aspects, multiple optical recordings can be obtained, and different locations of the body can be screened for abnormalities (e.g., between the head and legs, between arms and legs, etc.). Additionally, the present technology can be applied to perform optical retinal angiography without the use of contrast agents. In some aspects, interruptions in the heart rate phenomenon may occur before pathological changes, which enables the present technology to diagnose disease earlier than the prior art.

[0070] The present techniques provide improvements in the fields of medical technology and medical imaging. These techniques can be used to detect circulatory abnormalities with reduced or no contrast. Additionally, the techniques can be made visible through the processing of video technology. By detecting changes in heart rate activity, patients can be quickly screened and further testing can be performed as indicated. Additionally, intrinsic contrast angiograms can be obtained over a period of time, with the most recent intrinsic contrast angiogram serving as a baseline. Changes in the baseline over time may indicate the development of underlying circulatory or cardiac disease.

[0071] These inventive techniques improve operating or surgical imaging systems designed for use in surgical settings, diagnostic imaging systems, and other types of optical imaging systems by enabling such systems to use inherent contrast to extract and display information about heart rate phenomena.

[0072] These techniques offer further advantages for critically ill or intraoperative patients, as they can take intravenous access for granted. For example, the need to coordinate intravenous contrast with optical capture dominates the entire procedure and causes interruptions to other medical steps. For patients without intravenous access, this approach can avoid the discomfort and inconvenience of intravenous access.

[0073] Medically significant rapid or transient phenomena may occur without warning, such that an injected contrast agent may not be circulating at a given moment, making imaging and analysis of the phenomenon impossible. With intrinsic contrast, temporary, post hoc, or continuous cardiac rhythm phenomena can be analyzed in images acquired at a faster rate than the heart rate, since they are always present in living patients.

[0074] In other aspects, multiple types of inputs (eg, light, near infrared, heat, sound, etc.) may be obtained and processed, and the resulting outputs combined to enhance heart rate detection compared to a single input.

[0075] It will thus be seen that the objects set forth above are efficiently obtained, among those objects which are obvious from the foregoing description and because certain changes may be made in implementing the above-mentioned method and (multiple) constructions without departing from the spirit and scope of the invention, it is intended that all matter contained in the above description and shown in the accompanying drawings shall be interpreted as illustrative and not limiting.

[0076] It is also to be understood that the following claims are intended to cover all generic and specific features of the invention described herein and that all statements of the scope of the invention that may be said to come therebetween in language.

[0077] The technology provided herein has been described in terms of preferred embodiments, and it is recognized that equivalents, alternatives, and modifications, aside from those explicitly stated, are possible and are within the scope of the appended claims.

Claims

1. A method for extracting cardiac frequency phenomena from a sequence of optical images acquired by an optical system at a rate faster than the cardiac frequency, comprising: acquiring a time-sequential sequence of optical images of the patient acquired at a rate faster than a heart rate, wherein the time-sequential sequence of optical images captures one or more physical characteristics of an intrinsic contrast; acquiring a measured or estimated heart rate signal from the patient independently of the time-sequential optical image sequence; Applying a cross-correlation wavelet transform analysis to the time-sequential optical image sequence to generate a spatiotemporal representation of the heart rate phenomenon, wherein the cross-correlation wavelet transform analysis comprises: performing a wavelet transform on the time-sequential optical image sequence to obtain a wavelet transform signal; Cross-correlating the wavelet transform signal with the heart rate signal to obtain a cross-correlation signal; filtering the cross-correlation signal at a heart rate to obtain a filtered signal; and performing an inverse wavelet transform on the filtered signal to obtain a spatiotemporal representation of the time-sequential optical image sequence; and An image of the heart rate phenomenon is generated from the spatiotemporal representation.

2. The method of claim 1, further comprising displaying the spatiotemporal representation in one or more reconstructed images.

3. The method of claim 1, further comprising obtaining a simultaneously measured cardiac signal and cross-correlating the simultaneously measured cardiac signal with the wavelet transformed signal.

4. The method according to claim 1, characterized in that The time-sequential optical image sequence is obtained by optical microscopy, near-infrared microscopy, ultrasound or Doppler technology.

5. The method according to claim 1, characterized in that The time-sequential optical image sequence is obtained by imaging a body having a vascular structure.

6. The method according to claim 1, characterized in that Intrinsic controls include thermal radiation, optical reflectance, or autofluorescence.

7. The method according to claim 1, characterized in that The reconstructed angiograms are provided as cine video sequences.

8. The method of claim 1, further comprising: acquiring a second temporal sequence of images having an intravenously injected chemical contrast agent and generating a second spatiotemporal representation of images obtained from the intravenously injected chemical contrast agent; pairing the spatiotemporal representation of the image obtained using an intrinsic contrast with the second spatiotemporal representation of the image obtained from the intravenously injected chemical contrast agent; as well as The second spatiotemporal representation of the image obtained using an intrinsic contrast is calibrated based on the pairing.

9. An optical system for extracting cardiac rate angiographic phenomena obtained at a rate faster than the cardiac rate, the system comprising: one or more computer processors; one or more computer-readable storage media; Program instructions stored on the one or more computer-readable storage media for execution by at least one of the one or more computer processors, the program instructions comprising instructions for: acquiring a time-sequential sequence of optical images of the patient acquired at a rate faster than a heart rate, wherein the time-sequential sequence of optical images captures one or more physical characteristics of an intrinsic contrast; acquiring a measured or estimated heart rate signal from the patient independently of the time-sequential optical image sequence; Applying a cross-correlation wavelet transform analysis to the time-sequential optical image sequence to generate a spatiotemporal representation of the heart rate phenomenon, wherein the cross-correlation wavelet transform analysis comprises: Performing wavelet transform on the time-series optical image sequence to obtain a wavelet transform signal; cross-correlating the wavelet transform signal with the heart rate signal to obtain a cross-correlation signal; filtering the cross-correlation signal at a heart rate to obtain a filtered signal; and performing an inverse wavelet transform on the filtered signal to obtain a spatiotemporal representation of the time-sequential optical image sequence; and An image of the heart rate phenomenon is generated from the spatiotemporal representation.

10. The system according to claim 9, characterized in that The program instructions further include the following instructions: The spatiotemporal representation is displayed in one or more reconstructed images.

11. The system according to claim 9, characterized in that The program instructions further include the following instructions: A simultaneously measured cardiac signal is obtained and cross-correlated with the wavelet transformed signal.

12. The system according to claim 9, characterized in that The images are obtained by optical microscopy, near infrared microscopy, ultrasound or Doppler technology.

13. The system of claim 9, wherein: Images are obtained by imaging the body having vascular structures.

14. The system of claim 9, wherein: Intrinsic controls include thermal radiation, optical reflectance, or autofluorescence.

15. The system of claim 9, wherein: The reconstructed angiograms are provided as cine video sequences.

16. The system of claim 9, wherein: The program instructions further include the following instructions: acquiring a second temporal sequence of images having an intravenously injected chemical contrast agent and generating a second spatiotemporal representation of images obtained from the intravenously injected chemical contrast agent; pairing the spatiotemporal representation of the image obtained using an intrinsic contrast with the second spatiotemporal representation of the image obtained from the intravenously injected chemical contrast agent; as well as The second spatiotemporal representation of the image obtained using an intrinsic contrast is calibrated based on the pairing.

17. A computer program product for extracting a heart rate phenomenon obtained at a rate faster than the heart rate, the computer program product comprising one or more computer readable storage media, the one or more computer readable storage media collectively having program instructions contained therein, the program instructions being executable by a computer to cause the computer to: acquiring a time-sequential sequence of optical images of the patient acquired at a rate faster than a heart rate, wherein the time-sequential sequence of optical images captures one or more physical characteristics of an intrinsic contrast; acquiring a measured or estimated heart rate signal from the patient independently of the time-sequential optical image sequence; Applying a cross-correlation wavelet transform analysis to the time-sequential optical image sequence to generate a spatiotemporal representation of the heart rate phenomenon, wherein the cross-correlation wavelet transform analysis comprises: performing a wavelet transform on the time-sequential optical image sequence to obtain a wavelet transform signal; Cross-correlating the wavelet transform signal with the heart rate signal to obtain a cross-correlation signal; filtering the cross-correlation signal at a heart rate to obtain a filtered signal; and performing an inverse wavelet transform on the filtered signal to obtain a spatiotemporal representation of the time-sequential optical image sequence; and An image of the heart rate phenomenon is generated from the spatiotemporal representation.

18. The computer program product of claim 17, wherein: The program instructions further include instructions for performing the following operations: The spatiotemporal representation of the heart rate angiographic phenomenon is displayed in one or more reconstructed images.

19. The computer program product of claim 17, wherein: The program instructions further include instructions for performing the following operations: A simultaneously measured cardiac signal is obtained and cross-correlated with the wavelet transformed signal.

20. The computer program product of claim 17, wherein: The program instructions further include instructions for performing the following operations: acquiring a second temporal sequence of images having an intravenously injected chemical contrast agent and generating a second spatiotemporal representation of images obtained from the intravenously injected chemical contrast agent; pairing the spatiotemporal representation of the image obtained using an intrinsic contrast with the second spatiotemporal representation of the image obtained from the intravenously injected chemical contrast agent; as well as The second spatiotemporal representation of the image obtained using an intrinsic contrast is calibrated based on the pairing.

Images