Systems and methods for x-ray image feature detection and registration - Patents.com
The method addresses alignment and registration issues in X-ray imaging by detecting and excluding contrast clouds and guidewires, enhancing diagnostic accuracy and procedural precision in intravascular procedures.
Patent Information
- Application Number
- JP2023108995
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- Priority Date
- 2015-11-18
- Filing Date
- 2023-07-03
- Publication Date
- 2024-11-15
- Estimated Expiration
- 2036-11-18
AI Technical Summary
Existing X-ray imaging methods for intravascular procedures face challenges in achieving accurate alignment and registration due to factors like contrast agent obscuration, guidewire interference, and complex vascular structures, which hinder diagnostic accuracy and stent planning.
A method for detecting and excluding contrast clouds, registering vascular trees, and tracking guidewires in X-ray images using software-based processing to improve cross-frame alignment, including techniques for contrast cloud detection, guidewire extraction, and vascular tree registration across multiple frames.
Enhances diagnostic accuracy by stabilizing vessel centerlines, reducing registration errors, and improving the alignment of X-ray and intravascular images, facilitating precise stent deployment and procedural accuracy.
Smart Images

Figure 0007584581000001 
Figure 0007584581000002 
Figure 0007584581000003
Abstract
Description
[Technical field]
[0001] CROSS-REFERENCE TO RELATED APPLICATIONS This application claims priority to U.S. Provisional Patent Application No. 62 / 257,213, filed November 18, 2015, the contents of which are incorporated herein by reference in their entirety.
[0002] Technical Field In part, the present disclosure relates generally to the field of vascular and peripheral vascular imaging and data collection. More particularly, the present disclosure relates in part to image feature detection and analysis. [Background technology]
[0003] X-ray imaging provides an important diagnostic tool in various disciplines. For example, interventional cardiologists use X-ray-based imaging systems in various procedures and tests. As a specific type of X-ray imaging, X-ray fluoroscopy is commonly used to perform angiographic imaging of blood vessels. Visualizing arteries during catheterization is a valuable diagnostic tool. Such visualization helps plan and perform catheter placement and stent deployment. As a result, achieving accurate visualization is a key technical requirement for X-ray imaging and tracking features and objects associated therewith. Numerous imaging and co-registration challenges can arise, making such accuracy difficult to achieve. In particular, when X-ray-based imaging methods are combined with intravascular imaging using optical coherence tomography or ultrasound probes, the imaging and co-registration challenges become even more complex. Various factors related to the various imaging methods and devices used to position and guide the imaging device can also negatively affect registration and co-registration. These factors can create additional problems that must be addressed.
[0004] For example, the injection of contrast agents to enhance angiographic imaging of blood vessels can obscure optical, acoustic and other imaging probes placed within arteries. The use of guidewires and catheters in the vascular system can also obscure certain landmarks or otherwise interfere with imaging and analysis tools. As a result, all of these factors and others make it difficult to perform diagnostic intravascular imaging and image display and registration as part of the diagnostic process. In addition, challenges related to imaging, image analysis and registration also negatively impact stent planning and other procedures that rely on such imaging data and associated diagnostic information.
[0005] The present disclosure addresses these challenges and others. [Prior art documents] [Patent documents]
[0006] [Patent Document 1] U.S. Patent No. 9,351,698 Summary of the Invention [Means for solving the problem]
[0007] The present disclosure generally relates to the field of data collection, imaging, image processing and related feature detection of vascular and peripheral vascular systems. In part, the present disclosure relates to methods for detecting the location and size of contrast clouds in X-ray images, more particularly, for example, for sequences of X-ray images during intravascular imaging. These techniques are useful when contrast agents are introduced into blood vessels or other body lumens as part of various angiographic or other X-ray related imaging methods. Injection or other delivery of contrast agents makes arteries radio-opaque for angiographic or other X-ray related imaging methods. For example, in imaging arteries for cardiac procedures, contrast agents are introduced into one or more arteries. In part, the present disclosure relates to methods for detecting parameters related to contrast clouds in X-ray images, for example, for sequences of X-ray images. These parameters can include the location and size of a given contrast cloud.
[0008] A method for detecting and extracting metal wires, such as guidewires used in coronary procedures, from X-ray images is also described herein. Additionally, a method for registering vascular trees for one or more images, such as in a sequence of X-ray images, is also disclosed. In part, the disclosure relates to processing, tracking and registering angiographic images and elements in such images. Registration can be performed on images from intravascular imaging modalities, such as optical coherence tomography (OCT) or intravascular ultrasound (IVUS).
[0009] In part, the present disclosure relates to detecting contrast clouds on X-ray image frames, allowing cloud-containing regions of the X-ray images to be excluded from one or more subsequent X-ray image processing stages to increase cross-frame registration accuracy. The detection / identification of contrast cloud regions provides a stable vascular centerline that increases the accuracy of cross-frame registration between X-ray frames. Additionally, thanks to contrast cloud detection, radiopaque marker detection in X-ray images can be improved from a more stable vascular centerline. In an embodiment, the vascular centerline is also referred to as a trace, and vice versa.
[0010] In part, the disclosure relates to a method for registering a vascular tree or one or more segments of a vascular tree across multiple frames. In part, the disclosure relates to a method for extracting branch locations from a skeleton generated from angiograms and fusing the same branch locations on a series of frames. In an embodiment, the method includes eliminating false branches. If not eliminated, such false branch detections would be used to generate paths through the vascular tree, interfering with accurate image registration and other subsequent processing steps. In part, the disclosure relates to a method for representing branches based on their characteristics (features), such as take-off angle, arc length position, absolute angle, intensity, size (scale). In an embodiment, the take-off angle is measured relative to the parent vessel. Various image processing methods for performing such feature extraction can be used as described herein.
[0011] In part, the present disclosure relates to a method for associating the same anatomical branch extracted from multiple angiograms based on clustering in feature space, including methods for filtering excess data and completing missing representations. In part, the present disclosure relates to a method for selecting a suitable branch cluster that can act as one or more anchor points for refinement reducing frame cross image registration. Frame cross image registration facilitates position continuity between x-ray images, allowing improved accuracy in tracking positions, and allowing a frame of reference for stent deployment and other intravascular procedures performed using angiograms.
[0012] In some embodiments, the method further includes expanding one or more candidate regions or the fused contrast cloud region to provide a safety zone to exclude tracking of markers. In some embodiments, the method further includes co-registering the OCT image data and a plurality of the angiography image frames. In some embodiments, the method further includes applying a smoothing filter to one or more angiography frames such that elements with a particular desired scale are preserved.
[0013] In some embodiments, the method further includes adaptively thresholding the smoothed image, whereby small scale image elements are removed to generate a binary image, in which pixels in a contrast cloud region are a first value or threshold, e.g., an intensity value or threshold in the binary image. In some embodiments, the method further includes selecting a contrast cloud neighborhood to include a cloud region having the first value in the binary image and overlaying the neighborhood with one or more angiography frames. In some embodiments, the method further includes counting or scoring an amount of pixels in the neighborhood that include the first value. In some embodiments, the method further includes adaptively thresholding pixels that include the first value and removing pixels from each neighborhood that include values other than the first value.
[0014] In an embodiment, the method further includes detecting the guidewire in the plurality of angiography frames, generating one or more image masks, and using the guidewire location and the one or more masks to find a stable anatomical anchor point. In an embodiment, the determined anchor point is selected as a distal end point of the vascular centerline. Determining such an end, in an embodiment, improves vascular centerline generation.
[0015] In one embodiment, the method further includes plotting a plurality of clusters associated with the bifurcations and using the clusters to perform cross-frame registration between angiographic frames based on arc length interpolation between vessel segments defined between two anchor points, such as bifurcations, bends, and vessel centerline end points in each frame. In one embodiment, the method further includes grouping a plurality of anatomically associated bend points by using any suitable shortest or optical path determination algorithm, such as the Viterbi algorithm. Such algorithms or other methods can be used to determine a likely sequence of locations and associated most likely paths through the vessel section. In one embodiment, the cost criterion is based on bend angle change or curvature or curvature analog, bend location along the vessel centerline, and difference in bend angle deviation between consecutive frames.
[0016] In some embodiments, the method further includes applying an image processing transform, such as a kernel matrix or other image transformation operator or matrix, to one or more frames to remove or modify a feature in at least one frame. The method can include generating a mask to perform feature extraction during one or more subsequent image processing stages, where the feature is the guidewire in the image.
[0017] In part, the disclosure relates to a processor-based method of co-registering angiography image data with intravascular image data obtained during a pullback through a blood vessel, the method including: storing a plurality of frames of optical coherence tomography data in a memory; storing a plurality of frames of angiography image data in a memory; processing the plurality of frames of angiography image data to generate a set of cross-frame co-registered angiography data; generating vascular centerlines for the plurality of frames of angiography image data; detecting probe markers in the plurality of frames of angiography image data; tracking positions of the probe markers along one or more vascular centerlines; and co-registering the plurality of frames of angiography image data and the plurality of frames of optical coherence tomography data using the tracked positions.
[0018] In an embodiment, the method further includes generating a score indicative of a level of confidence in the co-registration between the frames of angiography image data and the frames of optical coherence tomography data. In an embodiment, co-registrating the plurality of frames of angiography image data and the plurality of frames of optical coherence tomography data includes generating, using a computing device, a co-registration table, the co-registration table including the angiography image frames, a plurality of OCT timestamps per frame, a plurality of angiography timestamps per frame, and the optical coherence tomography image frames, and including a score measurement for each co-registered position.
[0019] In some embodiments, the method further includes displaying representations of the stent in the OCT images and the angiography images in a user interface using the co-registration table and a computing device. In some embodiments, the method further includes identifying branches in one or more OCT images or angiography images using the co-registration table and a user interface configured to display the branches.
[0020] In an embodiment, X-ray image registration can be performed with respect to one or more X-ray images as part of the cross-frame registration. Image registration can also be performed with respect to or using representations of arterial segments, such as sections of a vascular tree. These representations can include skeletons, vessel or vascular tree coordinate or location systems, geometric models of one or more vessels, and curves or centerlines tracing paths through one or more vessels. These various representations can be detected, generated, or registered with respect to one or more other images from another imaging modality, such as OCT, IVUS, or other intravascular imaging modalities.
[0021] <Embodiments Related to Imaging Cloud> In part, the present disclosure relates to contrast cloud detection methods and diagnosis and analysis of angiography image frames that include one or more detected contrast cloud regions.
[0022] In one aspect, the disclosure relates to a system of one or more computing devices that can be configured to perform a particular operation or action by virtue of having installed thereon software, firmware, hardware, software-based image processing modules, or combinations thereof, that, when operated, cause the system to perform the action. One or more computer programs can be configured to perform a particular operation or action by virtue of including instructions that, when executed by a data processing device, cause the device to perform the action. One general aspect includes a computing device-based method of detecting one or more regions of interest in one or more X-ray images. The method includes storing in an electronic memory device a set of angiographic image frames acquired during a first time period that includes one or more time periods during which contrast agent is in blood vessels; detecting a plurality of candidate contrast cloud regions in the one or more angiographic frames; and combining the plurality of candidate contrast cloud regions to generate a fused contrast cloud region, the fused contrast cloud region having a cloud boundary. Other embodiments of this aspect include corresponding computer systems, devices, and computer programs stored in one or more computer storage devices, each configured to perform the actions of the method.
[0023] In various embodiments, implementations may include one or more of the following features: The method further includes excluding angiographic image data located within a fused contrast cloud region from one or more subsequent data processing methods. In some embodiments, the one or more subsequent data processing methods include generating vascular centerlines. The method further includes generating one or more vascular centerlines for one or more angiographic image frames. In the method, the one or more subsequent data processing methods include cross-frame registration of two or more angiographic image frames of the set of angiographic image frames. The method further includes performing cross-frame registration of two or more angiographic image frames of the set of angiographic image frames.
[0024] In an embodiment, the method further includes generating a set of cross-frame locations. In the method, the one or more subsequent data processing methods include a process selected from the group including anchor point extraction; arc length interpolation; and cross-frame location generation. The method further includes dilating one or more candidate regions or fused contrast cloud regions to provide an expanded exclusion zone, and further includes excluding angiographic image data located within the expanded exclusion zone from one or more subsequent data processing methods. The method further includes co-registering the OCT image data and the plurality of angiographic image frames. The method further includes applying a smoothing filter to the one or more angiographic frames such that elements having a particular desired scale are preserved. The method further includes adaptively thresholding the smoothed angiographic frames from which small-scale elements have been removed to generate a binary image. In the binary image, one or more pixels within the contrast cloud region are a first value in the binary image.
[0025] In one embodiment, the method further includes selecting a contrast cloud neighborhood to include a cloud region having the first value in the binary image and overlaying the neighborhood with one or more angiography frames. The method further includes counting or scoring a number of pixels in the neighborhood that include the first value. The method further includes adaptively thresholding pixels that include the first value to remove pixels from each neighborhood that include values other than the first value. The method further includes generating a mask and using the mask to detect stable anatomical anchor points that serve as proximal endpoints of vascular centerlines.
[0026] In an embodiment, the method further comprises detecting a plurality of anatomical features in the set of angiographic image frames; generating clusters of the detected anatomical features; and performing cross-frame registration between the angiographic frames using the clusters, where a cluster refers to a single anatomical feature extracted from a plurality of frames. In an embodiment, the anatomical feature is a branch or inflection point in a vascular tree.
[0027] In one embodiment, the method further comprises grouping a plurality of anatomically associated inflection points using a shortest path finding algorithm to identify a likely path through the associated inflection points, where the likely path is identified in response to one or more criteria selected from the group including: inflection angle change, curvature, curvature analog, inflection position along the centerline, and difference in inflection angle deviation between successive frames. The method further comprises applying an image processing transform to one or more frames to remove or modify a feature in at least one frame to generate a mask for performing feature extraction during one or more subsequent image processing steps, where the feature is a guidewire in the image.
[0028] In an embodiment, the method further includes co-registering the plurality of frames of angiography image data and the plurality of frames of optical coherence tomography data using a co-registration table. The co-registration table includes angiography image frames, a plurality of OCT timestamps per frame, a plurality of angiography timestamps per frame, and an optical coherence tomography image frame, and includes a score measurement for each co-registered position. The method further includes displaying a representation of the stent in the OCT image and the angiography image in a user interface using the co-registration table and a computing device. The method further includes identifying a branch in one or more of the OCT image or the angiography image using the co-registration table and a user interface configured to display the branch.
[0029] In an embodiment, the method further includes displaying, with a diagnostic system, a plurality of frame transverse registered angiographic images, wherein the plurality of frame transverse registered angiographic images are selected from the set.
[0030] In this method, one or more steps of the method are implemented using a diagnostic system including an input for receiving the set of frames from an angiography system, one or more electronic memory devices for storing the set, one or more computing devices in electrical communication with the input and the one or more memory devices, and instructions, image filters, and image processing software modules executable by the one or more computing devices to perform one or more steps of the method. Implementations of the described techniques may include hardware, methods or processes, or computer software on a computer-accessible medium or may be stored in a computer-readable medium, such as a non-transitory computer-readable medium.
[0031] Embodiments Related to Guidewire Detection and Extraction In part, this disclosure relates to systems and methods for detecting a guidewire in one or more frames of angiographic data, and in part, this disclosure relates to various diagnostic and image processing methods suitable for operating on angiographic images that include one or more guidewire segments.
[0032] In an embodiment, the disclosure relates to one or more computer systems that can be configured to perform certain operations or actions by virtue of having installed thereon software, firmware, hardware, or combinations thereof that, when operated, cause the system to perform the actions. One or more computer programs can be configured to perform certain operations or actions by virtue of having instructions that, when executed by a data processing device, cause the device to perform the actions. One general aspect includes a processor-based method of detecting one or more regions of interest in one or more X-ray images. The method includes storing in an electronic memory device a set of angiographic image frames acquired during a first time period; detecting a guidewire in one or more frames of the set; detecting an arterial segment in one or more frames of the set; and generating a plurality of frame transverse positions for a group of frames. The group of frames includes: The processor-based method of detecting one or more regions of interest also includes one or more frames that include the detected guidewire. The processor-based method of detecting one or more regions of interest also includes one or more frames that include one or more of the detected arterial segments. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices. Each is configured to perform the actions of the method.
[0033] Implementations may include one or more of the following features. The method further includes: The method may also include performing arc length interpolation on one or more detected arterial segments in one or more frames of the set. The method further includes performing a cross-frame registration of the set of angiographic image frames using the multiple cross-frame locations. The method further includes identifying multiple anatomical features in one or more of the frames. In the method, identifying the anatomical features includes generating a cluster including a set of detected anatomical features across multiple frames, where the cluster indicates that the detected anatomical features are the same anatomical feature imaged in different frames at different times. The method further includes: The method may also include detecting multiple candidate contrast cloud regions in one or more angiographic frames. The method further includes excluding regions of one or more of the angiographic frames identified as including contrast cloud regions from a centerline generation process. The method further includes defining a proximal endpoint of one or more vascular centerlines using endpoints selected from one or more of the candidate contrast cloud regions. and defining a distal end point of one or more vascular centerlines using the detected guidewire end point. The method further includes generating a plurality of vascular centerlines for a plurality of the angiographic image frames in the set. The method further includes defining a distal end point of one or more vascular centerlines using the detected guidewire end point. The method further includes performing a pullback of an intravascular data collection probe during the first time period. The method includes detecting a guidewire in one or more frames of the set.
[0034] In an embodiment, the method may also include applying a plurality of filters to the angiographic image frames. The method may also include adaptively thresholding the filtered angiographic frames. The method may also include operating on the adaptively thresholded angiographic frames with an intensity filter to generate an intensity filtered frame. The method may also include detecting guidewire portions in the intensity filtered frame. In the method, the plurality of filters include a morphological filter and a ridge enhancement filter. In an embodiment, the computing device includes further instructions for performing arc length interpolation on one or more detected arterial segments in one or more frames of the set. In an embodiment, the computing device includes further instructions for performing a frame cross registration of the set of angiographic image frames using the plurality of frame cross positions.
[0035] In an embodiment, the computing device includes further instructions for detecting a plurality of candidate contrast cloud regions in one or more angiography frames. In an embodiment, the computing device includes further instructions for defining a proximal endpoint of one or more vascular centerlines using endpoints selected from one or more of the candidate contrast cloud regions. In an embodiment, the computing device includes further instructions for defining a distal endpoint of one or more vascular centerlines using endpoints of the detected guidewire. In an embodiment, the computing device includes further instructions for generating a plurality of vascular centerlines for a plurality of the angiography image frames in the set. In an embodiment, the computing device includes further instructions for defining a distal endpoint of one or more vascular centerlines using endpoints of the detected guidewire. In an embodiment, the computing device includes further instructions such that detecting a guidewire in one or more frames of the set includes applying a plurality of filters to the angiography image frames. The system may also include adaptively thresholding the filtered angiography frames. The system may also include operating on the adaptively thresholded angiography frames with an intensity filter to generate intensity filtered frames. The system may also include detecting a guidewire portion in the intensity filtered frames. In one embodiment, the plurality of filters includes a morphological filter and a ridge enhancement filter. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.
[0036] One general aspect includes a system for detecting one or more features in an angiographic image. The system includes: one or more memory devices and a computing device in communication with the memory device, the memory device including instructions executable by the computing device to cause the computing device to: store in an electronic memory device a set of angiographic image frames acquired during a first time period; detect a guidewire in one or more frames of the set; detect an arterial segment in one or more frames of the set; and generate a plurality of frame transverse positions for a group of frames. The system also includes one or more frames including the detected guidewire. The system also includes one or more frames including one or more of the detected arterial segments. Other embodiments of this aspect include corresponding computer systems, apparatus, and computer programs stored on one or more computer storage devices, each configured to perform the actions of the method.
[0037] Implementations may include one or more of the following features: In some embodiments, the computing device includes further instructions for performing arc-length interpolation on one or more detected arterial segments in one or more frames of the set. In some embodiments, the computing device includes further instructions as described herein. The system may also include performing a cross-frame registration of the set of angiography image frames using the multiple cross-frame locations. In some embodiments, the computing device includes further instructions for detecting multiple candidate contrast cloud regions in one or more angiography frames.
[0038] In an embodiment, the computing device includes further instructions for defining a proximal endpoint of one or more vascular centerlines using endpoints selected from one or more of the candidate contrast cloud regions. In an embodiment, the computing device includes further instructions for defining a distal endpoint of one or more vascular centerlines using endpoints of a detected guidewire. In an embodiment, the computing device includes further instructions for generating a plurality of vascular centerlines for a plurality of the angiography image frames in the set. In an embodiment, the computing device includes further instructions for defining a distal endpoint of one or more vascular centerlines using endpoints of a detected guidewire.
[0039] In an embodiment, the computing device includes further instructions including detecting a guidewire in one or more frames of the set. The system may also include applying a plurality of filters to the angiographic image frames. The system may also include adaptively thresholding the filtered angiographic frames. The system may also include operating on the adaptively thresholded angiographic frames with an intensity filter to generate an intensity filtered frame. The system may also include detecting a guidewire portion in the intensity filtered frame. In an embodiment, the plurality of filters includes a morphological filter and a ridge enhancement filter. Implementations of the described techniques may include hardware, methods or processes, or computer software on a computer accessible medium.
[0040] Anatomical Feature Detection Embodiments In part, the disclosure relates to an anatomical feature detection and clustering-based verification method. In one aspect, the disclosure relates to a processor-based method for detecting one or more regions of interest in one or more x-ray images, the method including: storing a set of angiography image frames acquired during a first time period in an electronic memory device; generating a plurality of vascular centerlines for a plurality of the angiography image frames; generating a binary image of the angiography image frames for the group of angiography image frames; and generating a skeleton image from the binary image.
[0041] In an embodiment, the method includes applying a rib filter or a temporal filter to the skeleton image. In an embodiment, the method includes detecting one or more anatomical features in the skeleton image. In an embodiment, the anatomical features are a plurality of branches. In an embodiment, the anatomical features are a plurality of inflection points. In an embodiment, the method includes detecting the anatomical features includes generating a cluster including a set of the detected anatomical features across a plurality of frames, where the cluster indicates that the detected anatomical features are the same anatomical features imaged in different frames at different times. In an embodiment, the method includes applying a vascular intersection filter to the skeleton image. In an embodiment, the skeleton image is generated from vascular centerlines. In an embodiment, the method includes generating a plurality of clusters, where each cluster is a single anatomical feature extracted from a group of frames. In an embodiment, the method includes generating one or more distance metrics between two or more clusters.
[0042] In one embodiment, the distance metric is a Euclidean metric. In one embodiment, the method includes validating an anatomical feature as a result of its presence in two or more angiographic image frames. In one embodiment, the method includes collapsing the clusters to generate a set of clusters, each having a single representative from each frame of interest. In one embodiment, the method includes selecting one or more clusters. In one embodiment, the clusters are selected based on a parameter selected from the group consisting of arc length standard deviation, normalized arc length standard deviation, angle difference standard deviation, proximity to other clusters, average number of redundant anatomical feature records per frame, and average number of missing branch records per frame.
[0043] While the present invention relates to various aspects and embodiments, it is understood that the various aspects and embodiments disclosed herein can be integrated together in whole or in part as appropriate. Thus, each embodiment disclosed herein can be incorporated in each aspect in various degrees as appropriate for a given implementation, and steps from various methods can be combined without limitation.
[0044] Other features and advantages of the disclosed embodiments will become apparent from the following description and the accompanying drawings. [Brief description of the drawings]
[0045] The drawings are not necessarily to scale, with emphasis generally being placed upon illustrative principles. The drawings are to be considered in all respects as illustrative and are not intended to limit the disclosure, the scope of which is defined only by the claims. [Figure 1] 1 is a schematic diagram of an X-ray based imaging system and an intravascular imaging and data collection system according to certain exemplary embodiments of the present disclosure. [Figure 2A]FIG. 1 is a schematic diagram of an angiography image processing and frame tracking system and its components suitable for co-registering an X-ray image with an intravascular image by generating multiple frame cross-positions according to an exemplary embodiment of the present disclosure. [Figure 2B] 1 is an exemplary cross-registration table including various parameters for performing cross-registration between frames and output from performing imaging using an X-ray system during OCT or other intravascular pullback in accordance with an exemplary embodiment of the present disclosure. [Figure 2C] FIG. 1 is a schematic diagram of three OCT image frames and three corresponding angiography image frames acquired during pullback of an OCT probe with a radiopaque marker through an artery, according to an exemplary embodiment of the present disclosure. [Diagram 3] 1 is a flow chart illustrating a method for detecting contrast clouds in one or more X-ray images generated by injecting a contrast agent into a blood vessel, according to an exemplary embodiment of the present disclosure. [Figure 4] 1 is an X-ray image frame of a blood vessel injected with contrast agent after a noise removal process has been applied to the image frame, according to an exemplary embodiment of the present disclosure. [Diagram 5] FIG. 2 illustrates an exemplary binary image of a contrast cloud after adaptive thresholding has been applied to an X-ray image frame of a blood vessel, according to an exemplary embodiment of the present disclosure. [Figure 6] FIG. 13 illustrates an example image resulting from generating a bright pixel score or count or other pixel-based or intensity-based metric in a predefined neighborhood around a contrast cloud, according to an example embodiment of the present disclosure. [Figure 7] 1A-1C are diagrams illustrating example images in which potential locations and sizes of contrast clouds are detected from image frames according to certain exemplary embodiments of the present disclosure. [Figure 8] FIG. 13 illustrates an exemplary fused cloud mask from multiple image frames in accordance with certain exemplary embodiments of the present disclosure. [Figure 9] 1 is a flowchart illustrating a method for detecting a metal wire, according to an example embodiment of the present disclosure. [Figure 10] FIG. 1 illustrates an exemplary image enhanced to show an elongated structure, such as a candidate wire, according to certain exemplary embodiments of the present disclosure. [Figure 11] 1A is an exemplary image of a wire and its two end points with a bottom hat filter applied, and B is an exemplary image with an image processing filter such as a ridge filter applied, according to an exemplary embodiment of the present disclosure; [Figure 12] FIG. 2 illustrates an exemplary intensity threshold image of a portion of a vascular system with a wire placed within the vessel, according to certain exemplary embodiments of the present disclosure. [Figure 13] FIG. 2 illustrates an exemplary mask suitable for use during an image processing method for detecting a wire, such as an intravascular guidewire, in an X-ray image, in accordance with certain exemplary embodiments of the present disclosure. [Figure 14] 1 is a flowchart illustrating a method for aligning vascular tree segments in multiple frames according to one or more detected bends and branches of the vasculature, according to an exemplary embodiment of the present disclosure. [Figure 15A] 1 is an angiography frame pre-processed prior to skeleton image generation, according to an exemplary embodiment of the present disclosure. [Figure 15B] 1 is an original angiography image after application of image processing and data analysis according to certain exemplary embodiments of the present disclosure. [Figure 15C] 1 is a skeleton of one of the major blood vessels in the above image after application of image processing and data analysis, according to an exemplary embodiment of the present disclosure. [Figure 16] A is a subset of an angiographic image corresponding to a portion of a vascular system according to an exemplary embodiment of the present disclosure, and B is a representation of that portion of the vascular system showing branches as part of a vascular tree and the associated angle β between the vascular branch and its originating (parent) vessel according to an exemplary embodiment of the present disclosure. [Figure 17A] 1 is a plot showing multiple clusters generated from plotting normalized arc length (vertical axis) of branch segments against angle measurements (horizontal axis) in accordance with an exemplary embodiment of the present disclosure. [Figure 17B] FIG. 17B illustrates an angiography image frame depicting various branches (2, 1, and 3) associated with a particular cluster in the plot of FIG. 17A, according to an exemplary embodiment of the present disclosure. [Figure 18] A and B are frames of angiographic image data based on an exemplary embodiment of the present disclosure, showing paths traced through various vessel segments superimposed on the image, with labels identifying junctions or bends identified using the methods and systems described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS
[0046] The present disclosure relates to various methods, systems, and apparatus relating to x-ray imaging, such as angiography, and its applications in cardiology. In particular, the present disclosure relates to co-registrating features relating to frames of angiography data across or between such frames. The present disclosure also relates to various methods for improving such co-registration, for example, by reducing errors or detecting structures associated with the frames of angiography data.
[0047] Examples of such error reduction methods and other angiography or peripheral vasculature imaging enhancements, several of which are discussed in detail herein. These embodiments relate to contrast cloud detection, extracting or identifying wires in frames of X-ray image data, and tracking or registering features and devices to the vasculature, for example, with respect to angled branches and bifurcations or guidewires. These embodiments reduce errors that may propagate through other registration processes and lead to additional errors and inaccuracies. Ultimately, such errors may prevent proper cross-frame registration between angiography frames, and co-registration of other imaging modalities with such angiography frames. Errors may also interfere with tracking and co-registration of probe motion for probes that include one or more markers, such as radiopaque markers.
[0048] Interventional cardiologists use fluoroscopy combined with contrast injection, such as for angiographic imaging. The contrast agent spreads through the vascular tree, making it visible by x-ray. Typically, a time-varying contrast cloud forms near the catheter tip. The location or point of delivery of the contrast solution is highly variable, with a blob or cloud-like shape. This highly variable structure can hide the underlying anatomical information and confuse various image processing and computer vision algorithms, such as tracking and object or feature detection. The cloud can have different lobes or regions with irregular shapes. As a result, OR or other combination or union operators can be used to combine the various detected regions and either synthesize them or define them with an envelope or boundary to define the overall cloud region. In one embodiment, the contrast cloud defines a data exclusion region, defined by the union of the various regions or neighborhoods determined to be contrast-containing regions.
[0049] The presence of a contrast cloud may itself generate undesired imaging artifacts and errors. Similarly, the presence of one or more guidewires in an artery during imaging may lead to imaging errors and misinterpretation of the guidewire. Additionally, the tortuous and overlapping nature of the vascular system itself may make it difficult to track where a given intravascular probe or other medical device is located relative to an angiographic frame of data. Also, it is difficult for a human observer and an automated diagnostic system to determine which displayed segments of a vessel in a frame of angiographic data correspond to and align with the twists and turns and overlapping sections of a vascular system with its various branches and pathways. Some of the methods and systems described herein facilitate solutions to these difficulties. In view of the above, in part, the present disclosure relates to a method for detecting the location and spread of a contrast cloud generated during a contrast-enhanced x-ray scan, e.g., x-ray angiography.
[0050] Additionally, various wires may be used to guide catheters, balloons, stents or other devices during various procedures, diagnostics and intravascular imaging techniques. As part of the display of information to a user as part of diagnostic and intravascular imaging techniques, the present disclosure relates to a method for determining the location of a wire and / or wire tip from a frame of data, such as an angiography frame. Furthermore, this information may be used to support and enhance the user's viewing and interpretation of x-ray and intravascular images of the vascular system. The vascular system includes a variety of tortuous paths tracing various branches and trunks. As a result, the present disclosure also describes a method for vascular tree registration in a sequence of x-ray images. In this way, the guesswork of how overlapping arterial branches in an x-ray correspond to a branched three-dimensional tree needs to be navigated and interpreted as the branches of the three-dimensional tree change and move from frame to frame in response to heartbeats or other phenomena. The above features help increase the accuracy of frame-to-frame registration by addressing factors that may cause registration errors.
[0051] These category embodiments and others described herein can be used in a variety of x-ray imaging systems, including those working in concert with optical coherence tomography, ultrasound or other imaging and data collection systems. Intravascular imaging techniques are valuable tools that can be used in place of or in combination with x-ray fluoroscopy or other x-ray imaging systems. By looking inside the blood vessels, these imaging techniques can obtain high-resolution data on the condition of the blood vessels for a given subject. Combining these intravascular images with frame-crossing registered angiography images obtained during intravascular imaging, solving some of the challenges of contrast cloud noise, overlapping branches and guidewire artifacts, directly improves diagnostic accuracy.
[0052] As a result, intravascular imaging techniques such as optical coherence tomography (OCT) and acoustic techniques such as intravascular ultrasound (IVUS) are also described herein. For example, such vascular imaging is used by physicians to diagnose, locate, and treat vascular disease during interventions such as bypass surgery or stent placement. Figure 1 illustrates an exemplary system 2 for implementing one or more embodiments of the present invention, including an X-ray imaging system 4, such as an angiography system.
[0053] The data acquisition system 2 includes a non-invasive imaging system, such as nuclear magnetic resonance, X-ray, computer-assisted tomography or other suitable non-invasive imaging technique, as shown by system 4. As a non-limiting example of such a non-invasive imaging system, an angiography system 4 suitable for generating cines, for example, is shown. The angiography system 4 may include an X-ray fluoroscopy system. The angiography system 4 is configured to non-invasively image a subject S, thereby generating frames of angiography data, typically in the form of frames of image data. This X-ray imaging is performed while a pullback procedure is performed with the probe, and blood vessels in a region R of the subject S are imaged using angiography as well as one or more imaging techniques, such as OCT or IVUS. The imaging results of the non-invasive scan (left and right images in display 7) and intravascular imaging results, such as from OCT or IVUS, are shown on the center panel of the display 7. In addition to the display, the probe used to collect the intravascular data may be disposable and may be connected to a patient interface unit or PIU as part of the system 2.
[0054] The angiography system 4 communicates with an angiography data storage and image management system 12. In one embodiment, the system 12 can be implemented as a workstation or server. In one embodiment, data processing related to the collected angiography signals is performed directly on the detector of the angiography system 4. Images from the system 4 are stored and managed by the angiography data storage and image management system 12. In one embodiment, a subsystem, server or workstation handles the functions of the system 12. In one embodiment, the entire system 4 generates electromagnetic radiation, such as x-rays. The system 4 also receives such radiation after passing through the subject S. Furthermore, the data processing system 12 uses the signals from the angiography system 4 to image one or more regions of the subject S, including the region R. In one embodiment, the system 12 and the intravascular system 18 are all part of one integrated system.
[0055] As shown in this particular example, the region of interest R is a subset of the vasculature, such as a particular blood vessel, or peripheral vasculature. The region R can be imaged using OCT or other intravascular modalities. A catheter-based data collection probe 30 is introduced into the subject 10 and positioned within the lumen of a particular blood vessel, such as a coronary artery. The probe 30 can be various types of data collection probes, such as OCT probes, FFR probes, IVUS probes, probes combining features of two or more of the above, and other probes suitable for intravascular imaging. The probe 30 typically includes a probe tip, one or more radiopaque markers, optical fibers, and torque wires. Additionally, the probe tip includes one or more data collection subsystems, such as an optical beam director, an acoustic beam director, a pressure detector sensor, other transducers or detectors, and combinations of the above.
[0056] For probes that include an optical beam director, the optical fiber 33 is in optical communication with the probe with the beam director. The torque wire defines a bore in which the optical fiber is disposed. In FIG. 1, the optical fiber 33 is shown without the torque wire surrounding it. Additionally, the probe 30 also includes a sheath, such as a polymer sheath (not shown) that is part of the catheter. The optical fiber 33, which in the context of an OCT system is the sample arm of an interferometer, is optically coupled to a patient interface unit (PIU) as shown.
[0057] The patient interface unit PIU includes a probe connector suitable for receiving and optically coupling to an end of the probe 30. Typically, the data collection probe 30 is disposable. The PIU includes suitable joints and elements based on the type of data collection probe being used.
[0058] For example, a combination OCT and IVUS data collection probe requires an OCT and IVUS PIU. The PIU typically also includes a motor suitable for pulling back the torque wire, sheath and optical fiber 33 disposed therein as part of a pullback procedure. In this manner, the blood vessels of the subject 10 can be imaged longitudinally or in cross section. The probe 30 can also be used to measure specific parameters such as FFR or other pressure measurements.
[0059] Additionally, the PIU is connected to one or more intravascular data acquisition systems 18. The intravascular data acquisition system 18 can be an OCT system, an IVUS system, another imaging system, and combinations of the above. For example, in the context of the probe 30 being an OCT probe, the system 18 can include an interferometer sample arm, an interferometer reference arm, a photodiode, a control system, and a patient interface unit. Similarly, as another example, in the context of an IVUS system, the intravascular data acquisition system 18 can include ultrasound signal generation and processing circuitry, noise filters, a rotatable joint, a motor, and an interface unit.
[0060] In one embodiment, the data collection system 18 and the angiography system 4 have a shared clock or other timing signal configured to synchronize the timestamps of the angiography video frames with the timestamps of the OCT image frames. In one embodiment, the angiography system 12 runs various image processing and feature detection and other software based processes as indicated by 15a, 15b, 15c. In one embodiment, the angiography system 12 runs various image processing and feature detection and other software based processes as indicated by 15a, 15b, 15c. These processes may include a contrast cloud detection process, a feature extraction process, related wire detection and feature extraction, an inter-frame registration process, a cross-frame registration process, and other processes, methods, and steps described herein.
[0061] In general, software-based processes 15a, 15b, 15c are designed to reduce errors in cross-frame registration and to perform other processes described herein, such as to detect features in x-ray images and flag them for inclusion or exclusion in subsequent processing stages. In this manner, contrast clouds can be detected and then flagged by such software processes so that regions of the cloud are not used for processes that are negatively affected by position uncertainty and noise in those regions.
[0062] In an embodiment, it is advantageous for the contrast cloud to be located near a proximal end point of the vessel imaged by X-ray. As part of the process of determining the centerline, the contrast cloud location can be used to select an end point, helping to select a centerline end point from a set of candidate end points or to define a given centerline end point. In an embodiment, it is advantageous for the guidewire to be located near a distal end point of the vessel imaged by X-ray. As part of the process of determining the centerline, the guidewire location can be used to select an end point, helping to select a centerline end point from a set of candidate end points or to define a given centerline end point.
[0063] The present disclosure can be embodied as one or more computer program products, i.e., one or more modules of computer program instructions encoded on a computer readable medium for execution by or for controlling the operation of a data processing apparatus. The computer readable medium can be a machine readable storage device, a machine readable storage substrate, a memory device, or a combination of one or more of them. The term "data processing apparatus" encompasses all apparatus, devices, and machines for processing data, including, for example, a programmable processor, a computing apparatus, e.g., a computer or multiple processors or computers. An apparatus can include, in addition to hardware, code that creates an execution environment for the computer program in question, e.g., code constituting a processor firmware, a protocol stack, a database management system, an operating system, or a combination of one or more of them.
[0064] A computer program (also known as a program, software, software application, script or code) can be written in any form of programming language, including compiled or interpreted languages, and can be deployed in any form, including as a stand-alone program or as a module, component, subroutine or other unit suitable for use in a computing environment. A computer program does not necessarily correspond to a file in a file system. A program can be stored in part of a file that holds other programs or data (e.g. one or more scripts stored in a markup language document), in a single file dedicated to the program in question, or in several coordinated files (e.g. files that store one or more modules, subprograms or portions of code). A computer program can be deployed to be executed on one computer or on several computers located at one site or distributed across several sites and interconnected by a communications network.
[0065] The processes and logic flows described in this disclosure may be performed by one or more programmable processors executing one or more computer programs to perform functions by operating on input data and generating output. The processes and logic flows may also be performed, and apparatus implemented, by special purpose logic circuitry, such as an FPGA (field programmable gate array) or an ASIC (application specific integrated circuit).
[0066] Processors suitable for the execution of a computer program include, by way of example, both general purpose and special purpose microprocessors, and any one or more processors of any kind of digital computer. Typically, a processor receives instructions and data from a read-only memory or a random access memory or both. The essential elements of a computer are a processor for executing instructions and one or more memory devices for storing instructions and data. Typically, a computer will include one or more mass storage devices, such as magnetic, magneto-optical or optical disks, for storing data, or will be operatively coupled to receive data from and transfer data to such storage devices, or both.
[0067] A computer or computing device may include a machine-readable medium or other memory that includes one or more software modules for displaying a graphical user interface, such as an interface. The computing device may exchange data, such as monitoring data or other data, using a network. The network may include one or more wired, optical, wireless or other data exchange connections.
[0068] A computing device or computer may include a server computer, a client user computer, a control system, an intravascular or angiographic diagnostic system, a microprocessor, or any computing device capable of executing a set of instructions (sequential or otherwise) that specify actions to be taken by the computing device. Additionally, the term "computing device" should also be construed to include any collection of computing devices that individually or jointly execute a set (or sets) of instructions to perform any one or more of the software features or methods or operate as one of the system components described herein.
[0069] 1, various other types of data can be collected regarding the region R of the subject and other parameters of the subject of interest, including location information, vessel diameter, vessel branching location, area and vicinity of pixel intensity variation, and other data.
[0070] The data acquisition system 2 may include one or more displays 7 showing angiography frames of data, OCT frames, and user interfaces for OCT and angiography data. Mutual registration of angiography frames to other angiography frames may be permitted. The displays 7 may also show other controls and features of interest.
[0071] Non-invasive image data generated using the angiography image analysis and processing system 12 can be transmitted, stored, and processed on one or more servers or workstations, which can be the system 12 or system 18 shown in FIG. 1. The intravascular image processing system 16 can be in electrical communication with the PIU and the image processing subsystem 18. The subsystem 18 includes various software modules for tracking marker positions and performing co-registration between intravascular and x-ray image frames. Intravascular image data, for example frames of intravascular data generated using the data collection probe 30, can be routed via the PIU 35 to a data collection and processing system 45 coupled to the probe. A video frame grabber, for example a computer board configured to capture angiography image data from the system 12, can be used in various embodiments.
[0072] As part of the overall arrangement of steps and process flow 50, as shown in FIG. 2A, a sequence of x-ray images 51 are generated as an output of the angiography system 4 and transmitted to the data acquisition system 12 for image processing and storage. In one embodiment, x-rays are obtained during pullback intravascular probing through an artery. Each artery is part of the vascular system and may lead to various confluences or branches and one or more branches. These branches and branches may diverge at various angles from the artery of interest, e.g., a section of the artery that is simultaneously undergoing a pullback imaging procedure using OCT or IVUS. The guidewire extraction subsystem and method 53 may operate on and transform the x-rays to remove the appearance of the guidewire used to position the probe 30 in a given image frame. The position and terminal end point and other points along the detected guidewire may also be evaluated and used as anchor points as part of other image and data processing and analysis described herein. In one embodiment, as used herein, references to "extract" may be considered to refer to "detect" or "discriminate" and vice versa.
[0073] As discussed herein, imaging of such arteries is enhanced by the introduction of a radiopaque contrast agent. A contrast cloud is created that is visible as a region on each frame of an X-ray image. The cloud forms in the vicinity of where the contrast agent is introduced. The data acquisition system includes a contrast cloud detection subsystem and / or method 55 that operates on the X-ray image frames to characterize and / or detect the contrast cloud and its region. This subsystem can be implemented in software modules 15a, 15b or 15c or combinations thereof as part of system 12. The contrast cloud detection module 55 can be implemented using one or more software components that operate on the X-ray images to detect features of the image or transform, such as by increasing or decreasing image attributes of a given frame. The various flow charts, steps and processes shown in FIG. 2A or described elsewhere herein can be performed using the intravascular and angiography systems described herein as well as other computing devices and diagnostic control systems and processors.
[0074] In addition to performing detection and image processing steps to identify and exclude problematic contrast-containing regions of angiography frames, the present disclosure also includes processing steps and software-based methods 60 for generating vascular centerlines. Centerline generation can be improved as a result of the exclusion of contrast cloud regions. Once centerlines are generated, anchor point extraction steps and / or methods 65 can be used to detect any suitable anatomical features in the x-ray images, such as bifurcations and bends. These features can be used for various diagnostic and image processing methods. In an embodiment, once feature extraction has been performed on these structures, clusters and groups of representative anatomical features per frame, such as bifurcations and bends, can be used to mitigate co-registration errors and low confidence scores.
[0075] As a result, there is increased accuracy between which angiography frames are shown on the display relative to their counterpart intravascular image frames that are co-registered or cross-frame based registered. The co-registration and cross-frame registration processes facilitate diagnostic review, stent deployment review, and stent planning. As a result, mitigating registration errors through contrast cloud detection, guidewire extraction, and bifurcation and bend detection is important to achieve accurate co-registration and diagnose arterial or stent conditions by end users. In various embodiments, registration includes co-registration and cross-frame registration, or vice versa. In an embodiment, the contrast cloud detection process 55 and the guidewire extraction process 53 can generate position values for defining centerline endpoints. For example, a first centerline endpoint value C1 and a second centerline endpoint value C2 can be generated from the contrast cloud detection 55 and the guidewire extraction 53, respectively, or vice versa. The generation of proximal and distal endpoint values for the centerline using the contrast cloud detection data and the guidewire detection data increases the confidence of the centerline and reduces additional levels of calculation. This is because angiographic image data is used to inform the location of the ends of a given centerline.
[0076] As shown in FIG. 2A, an arc-length interpolation stage and related steps 67 can be performed with respect to the detected anchor points, and any anatomical landmarks or features that can be detected on the X-ray images can be used to identify corresponding vessel segments in all angiographic frames. In an embodiment, the anatomical landmarks or features include bifurcations, bends, as one or more of bifurcation and bend information to identify corresponding vessel segments in all angiographic frames. By using arc-length based interpolation, frame cross-positions can be generated. The frame cross-positions facilitate tracking the positions of the probe's radiopaque markers across the angiographic frames. The markers are moving elements that transition through various points during the pullback as shown in FIG. 2C. For example, these positions can be tracked relative to anatomical features or other landmarks in the angiographic frames. In an embodiment, tracking of these positions is combined with tracking of the radiopaque markers to perform co-registration with the intravascular imaging data.
[0077] FIG. 2B shows a co-registration table generated by one or more of the systems of FIG. 1 after the angiography frames have undergone cross-frame co-registration, i.e., co-registration between different angiography frames. The table in FIG. 2A shows different times for the OCT and angiography frames, since they typically operate on different systems or data sampling clocks or time periods. The (x,y) positions of the probe markers in each frame are displayed in the table. A score, which is an indication of the reliability of the registration between the angiography and OCT frames, is also shown. Angio index is the frame number for the angiography image sequence. FIG. 2C shows a schematic representation of the markers on the probe moving to different (x,y) spatial positions over time and which OCT frame corresponds to the associated angiography frame. The marker moves through locations A, B, and C in each of the three intravascular image frames (OCT-0, OCT-1, OCT-2) and three angiography frames (Angio-0, Angio-1, Angio-2) and can be registered frame-to-frame and across frames.
[0078] Methods and Analysis Related to Contrast Cloud Feature Extraction / Detection In part, the present disclosure relates to a method for improving tracking accuracy of proximal vascular endpoints, which are typically located close to and perturbed by proximity to contrast clouds. Detection of contrast clouds allows for improved stabilization of proximal vascular endpoints in OCT-angiography co-registration. In particular, the presence of contrast clouds complicates OCT-angiography co-registration and cross-frame registration between X-ray frames, creating uncertainty, for example, when determining a centerline with endpoints that maintain anatomical location across angiography frame sets.
[0079] During X-ray guided procedures, physicians use X-ray scans combined with contrast agents to visualize blood vessels and heart chambers. During contrast injection, a contrast cloud may form near the contrast-leading catheter. The contrast cloud is typically amorphous and varies in shape and size in different image frames acquired during a scan. The contrast cloud may occlude or hide underlying structures, potentially leading to poor performance of various image processing and computer vision algorithms. Detecting the location and extent of the contrast cloud from single or multiple image frames establishes a refined region of interest when applying image processing and computer vision algorithms.
[0080] For other purposes, it may be used as an image landmark. Essentially, those regions or detected contrast clouds can be flagged as noisy or inconsistent regions, for which position data and other image information used for other subsequent image and data processing is excluded. This is because data from such noisy or inconsistent regions can introduce errors that propagate through other subsequent image data transformations, which in turn cause additional errors and registration inaccuracies.
[0081] A contrast cloud detector can be utilized to generate a binary image. The bright components of the binary image are the areas of the image that contain cloud regions. Alternatively, in other embodiments, the bright areas can be inverted and the dark areas used for cloud detection. In one exemplary embodiment, a fused mask can be generated from the binary images from multiple image frames, which can be generated using a variety of techniques, including using a pixel-wise OR operation.
[0082] A post-filtering stage can be used to remove small component(s) that are outside the region of interest. These cloud regions or sums of cloud regions or ORed combinations of cloud regions define the regions to be excluded. Marker tracking is not performed in such regions or subsets thereof. These cloud regions can also be excluded from the frame traversal process, centerline generation and other processes described in this paper. The ORed combination is obtained from performing an OR operation to combine the cloud regions into a combined or fused cloud region. In this way, multiple candidate cloud regions can be combined to increase the chances of properly excluding regions where contrast clouds are likely to be present. These regions, if used, would be a source of error and marker position uncertainty, with detrimental effects on subsequent processing stages.
[0083] Thus, in one embodiment, when tracking the opaque markers of the imaging probe to the x-ray image frames generated using system 4, regions identified as containing contrast clouds are excluded and no marker tracking is performed therein. The same exclusion applies to other detections and processes described herein. Contrast cloud regions in an image can also be excluded from cross-frame analysis. By detecting the contrast cloud and defining a region associated with it, anatomical locations near or on the contrast cloud can be tracked with greater accuracy.
[0084] Furthermore, the detected contrast cloud boundaries or edges provide a basis for identifying which frames can be used to initiate a registration process or other processes, such as those described herein. As a result, for example, greater tracking accuracy improves vascular centerline determination, and thus improves the accuracy achieved when determining frame crossing positions. This reduces imaging artifacts as well as misalignment between vascular segments after a co-registration process is performed, such as by generating a co-registration table as shown in FIG. 2B.
[0085] FIG. 3 shows a flow chart 100 of a method for detecting contrast clouds for each input image. In step A1, the image is input to the system for processing for contrast cloud detection. In step A2, the image is denoised to produce a smoother image. This step can be an optional step, but can improve the image and can be important for noisy x-ray images. If the image is of better quality, step A2 can be skipped. FIG. 4 shows an example image of an x-ray image 150 of a blood vessel 170 with contrast agent 160 regions after image denoising has been performed. Knowing the location of the cloud 160 allows for improved consistency and stability of the vessel centerline, which leads to improved accuracy of mutual registration and tracking of the probe's markers.
[0086] In step A3, optionally, a morphological filter is applied to the image to smooth it in terms of intensity and shape (by reducing the number of gaps) and improve cloud uniformity. In an embodiment, steps A2 and A3 can be combined in a common denoising step. In an embodiment, steps A2 and A3 are both optical. In step A4, a first adaptive thresholding is used on the image to produce a binary image. FIG. 5 shows an example image where adaptive thresholding has been used to produce a binary image 180. As can be seen in FIG. 5, pixels or parts of the image in FIG. 4 that were darker regions, including areas of potential contrast clouds, are represented by bright white regions 200, 235, 240, 250 in the binary image 180 of FIG. 5. In FIG. 5, the binary image is shown after adaptive thresholding has been performed.
[0087] In step A5, for each image pixel in the binary image generated in step A3, the number of bright pixels within a neighborhood area resembling the typical size of a contrast cloud that requires detection is counted. The typical neighborhood area around the contrast cloud can be disk-shaped, rectangular, or any arbitrary shape. In an embodiment, the neighborhood dimension is less than about 5 mm. In an embodiment, the neighborhood dimension ranges from about 1 mm to about 4 cm. In an embodiment, the dimension is a diameter, a chord, or a line segment. Figure 6 shows an exemplary image 290 resulting from counting bright white pixels 300, 325, 330, 335 in the bright white region of the image of Figure 5 that are included in a predefined neighborhood around the contrast cloud 300.
[0088] In step A6, adaptive thresholding is used for each pixel from the image generated in step A5. The adaptive threshold used is a threshold related to the size of the neighborhood used to generate the image in step A5, as shown in Figure 6. In step A7, a component filter is used to remove large components from the image generated in step A6. In Figure 6, the image 290 shown results after the step of counting bright pixels in a predefined neighborhood has been performed.
[0089] In step A7, a dilation mask step is optionally used using a mask expansion image processing operator such as an image expansion kernel to increase the cloud mask size. FIG. 7 shows an example image 350 in which the potential location and size of contrast clouds from a single x-ray frame are detected. A dilated cloud mask 360 is shown with an edge or boundary 370. This is used to generate a cloud mask of the contrast cloud in step A10. The cloud masks, individually and together, can define regions of uncertainty where contrast clouds exist. Expanding the mask reduces the chance of tracking the probe in cloud regions. In one embodiment, cloud regions are detected and defined as exclusion zones.
[0090] In steps B1-B4, an optional processing step can be used to fuse cloud masks from multiple images. In step B1, cloud masks from multiple images are used together to generate a single fused mask. In step B2, a pixel-wise OR operator can be used to obtain a merged contrast cloud mask that incorporates information from multiple X-ray frames. After obtaining the merged mask, in step B3, another component-based filter can be used to remove small components or components outside the region of interest. The use of multiple X-ray frames is advantageous given that the clouds expand and disperse over a period of time following the initial delivery of the contrast solution.
[0091] In step B4, the cloud masks from each frame can be fused as shown in FIG. 8. FIG. 8 shows an example image 160 of a fused contrast cloud mask 410 from an X-ray image sequence. This mask can be generated and applied to the image to identify contrast cloud regions. It may include a buffer zone around as a safety factor. These identified contrast cloud regions can then be stored in memory as ignore / avoid regions when performing additional image processing. In one embodiment, the fused cloud mask in FIG. 8 is derived by taking a pixel-wise OR between multiple cloud masks.
[0092] Methods and Analyses Related to Guidewire and Thin Element Detection In another aspect, a method is provided for detecting the position of a wire, such as a thin metal wire, in an X-ray image, e.g., an X-ray angiogram. Metal wires used in procedures of the type described herein can include guidewires or wires with physiological gauges for measuring physiological conditions in the area around the wire. For example, the wires with physiological gauges or detectors can be in the form of pressure wires or wires with gauges for measuring any other condition, including but not limited to temperature, magnetism, impedance, and current and / or voltage.
[0093] The guidewire extraction methods described herein can be used to produce stable and consistent vascular centerlines across multiple frames. Once the guidewire is detected and extracted, the systems and methods described herein can use the guidewire position information to define a consistent and stable position. For example, in one embodiment, the detected guidewire position is selected to define one of the vascular centerline end points in all angiography frames. In one embodiment, the detected guidewire position selected after detection is the distal detected guidewire position. In this manner, having the guidewire located in a distal portion of the investigated vessel can be used to reduce registration and frame crossing errors.
[0094] Automatic extraction of wire position and / or wire tip position is important for various diagnostic procedures. For example, a metal wire can include a physiological measurement gauge, and the position of the wire can be used to automatically associate measurements from the gauge with its corresponding anatomical location. In another example, automatic trajectory extraction can be applied when a metal wire is moved in the X-ray scan area. In addition, when a wire is anchored to a specific anatomical location, automatic detection of a consistent anatomical location can be achieved using this method.
[0095] FIG. 9 shows a flow chart 500 of a method for detecting wires in an X-ray image. The process flow steps can be performed on an X-ray image, such as an angiogram image, or multiple images, such as an angiogram sequence or cine. In step 520, image smoothing of the X-ray image or image sequence is performed to enhance elongated structures in the image. The filter is a modified anisotropic diffusion filter, and in each iteration, the filter coefficients are derived from the original image intensity combined with a blob and ridge detector, a Laplacian of Gaussian (LoG). In one embodiment, the elongated structures include one or more of blood vessels, guidewires, ribs, or other edge-containing elements in the image data. FIG. 10 shows an exemplary image of an X-ray image of a blood vessel, where a wire has been detected after image smoothing has been performed. In FIG. 10, the image has been enhanced by performing an image smoothing step. As part of this process, the elongated structures are highlighted. A guidewire W is shown on the right, with end points P1 and P2. For arterial branches A1 and A2, two branches B1 and B2 are shown.
[0096] In step 525, a morphological filter capable of eliminating wide structures in the image is applied to the image, as shown in FIG. 11A. In one embodiment, the morphological filter is a bottom hat filter. In one embodiment, the morphological filter is any filter configured or constrained to enhance or select small scale features such as thin elements. FIG. 11A shows an example image 600 after the morphological filter has been applied. In one embodiment, the morphological filter is a bottom hat filter. A suitable morphological filter allows enhancement of dark elongated elements in the image that have a typical scale for the structuring elements used in a given morphological filter, e.g., a bottom hat filter. In another embodiment, the morphological filter can be replaced by a median filter that produces similar results.
[0097] At step 530, a ridge enhancement filter or detector or a vessel segmentation filter is applied, as shown in Figure 11B. In one embodiment, such a filter or detector is implemented using a ridge enhancement filter, such as a Frangi filter or other suitable filter, that is applied to the image to enhance lines in the image, as shown in Figure 11B. The ridge enhancement filter may include a Hessian filter, a Frangi filter, or other ridge or edge detector.
[0098] FIGURE 11B shows an example image 605 after a ridge enhancement filter has been applied to enhance ridge structures in the image. Thus, the ridge enhancement filter is used for extraction of thin and elongated features in an image. The ridge enhancement filter output is thresholded to produce a binary image containing thin and elongated dark elements that appear as bright pixels in the thresholded image.
[0099] In step 535, the processing step is to perform an adaptive thresholding on the input image to reject bright areas in the image. Metal wires are radiopaque and appear as dark strips on an X-ray image. An adaptive bisection threshold is applied to reject image areas with intensity values of no interest. Thus, in one embodiment, bright areas with an intensity greater than a certain threshold associated with an intensity value or value range corresponding to a dark value can be rejected. Figure 12 shows an example image 610 in which bright areas of the image have been rejected as a result of the process of performing an intensity thresholding image processing step.
[0100] In step 540, the ridge enhancement filter output and the adaptive intensity filter results are merged using a pixel-wise AND operator to obtain a merged metal wire mask component. In an embodiment, the angiography image processing software modules and methods described herein connect and filter wire fragments detected in images. In step 550, the wires can be extracted in fragments and other components in the image that are not related to the wires can be detected. The wire fragments can be joined together using a combined measurement of takeoff angle and distance between the fragments.
[0101] In an optional step, component post-filtering and / or component thinning can be performed to remove elements from the surrounding area that may have been stitched together during wire detection. In step 560, the results of the image processing of the previous steps are used to generate a mask of the detected wire. FIG. 13 shows an example image of the wire mask 620 generated in step 560. The mask 620 is shown with two levels, one intensity level associated with the wire W and the remainder of the mask is a second intensity level that is dark or black corresponding to the background of the image. The wire W has end points P1 and P2, respectively, as shown. The mask 620 is displayed after performing the thinning process 560. In an embodiment, one or more of the end points P1 and P2 can be used to set the end points of the vascular centerline. In an embodiment, guidewire detection is performed to identify one or more distal guidewire end points. The one or more distal guidewire end points can be selected to define the vascular centerline end points.
[0102] In one embodiment, prior to tracking the vascular centerline distal end points in one embodiment, a wire masking step is performed. Application of the wire mask assists in tracking the distal end points of the vascular centerline for one or more angiography frames with improved accuracy. Identification of contrast cloud regions and avoiding such regions during marker tracking and other image processing steps facilitates having stable and consistent centerline end points. Furthermore, having stable and consistent end points increases the accuracy of cross-frame registration.
[0103] Vascular tree alignment Various medical applications require accurate mapping between the same anatomical locations captured in different frames during motion and deformation. For example, angiographic imaging of arteries during intravascular imaging pullback and subsequent stent deployment is one such application. The heart is a fast moving organ with complex deformations. As a result, such mapping can be difficult to produce. This is because angiography provides a 2D view of a tortuous 3D system in which components of the vascular tree, branches and guidewires, implants, stents and other intravascular imaging devices overlap each other, creating ambiguities and overlapping areas in the angiographic image that do not actually exist in the subject's vascular system. Various embodiments of the present invention, as disclosed herein, provide methods and systems for performing such mapping using anatomical landmarks, such as bends, anchor points and branches.
[0104] In addition, the present disclosure also relates to a method for detecting branch extraction on angiography images. A method for grouping branches from multiple X-ray frames is also disclosed. Furthermore, a method for detecting and grouping vascular bend locations from multiple X-ray frames is disclosed. Motion estimation for 3D vasculature can also be performed using the detection methods described herein and tracking landmarks and their relative motion between frames over time. By incorporating detection of anchor points based on branches and bends across different X-ray frames, a method for improving the accuracy of cross-frame registration can be achieved. Other detection and exclusion processes described herein can also help improve such cross-frame registration.
[0105] In part, the present disclosure relates to a method for registering vascular trees, vascular tree segments, or other vascular components imaged on multiple X-ray frames, e.g., frames of angiography data. In an embodiment, the method can use anchor extraction, inflection point or branch point extraction, e.g., as a step or stage in a given registration between a first frame and a second frame. In an embodiment, the method can use vessel extraction, e.g., as a step or stage in a given registration between a first frame and a second frame.
[0106] As described herein, angiographic images may include one or more bifurcations. This disclosure describes image data processing and analysis stages to extract or otherwise identify bifurcations on a frame-by-frame or across frames. In an embodiment, a method is described for grouping bifurcations from multiple X-ray frames. This disclosure describes image data processing and analysis stages to extract or otherwise identify vascular bends on a frame-by-frame or across frames. In an embodiment, a method is described for grouping vascular bends and bifurcations from multiple X-ray frames.
[0107] Described herein is a method for registering vascular trees extracted from different contrast-enhanced X-ray frames, for example during X-ray angiography. In one embodiment, for each vascular branch of interest, a vascular centerline is known or treated as known. The process of detecting such a centerline can be performed using various methods described in U.S. Patent No. 6,399,993, the disclosure of which is incorporated herein by reference in its entirety. In one embodiment, the registration method described herein uses bifurcation points (if present) and inflection points (if present) as "anchors" for matching anatomical positions between different frames. Once a set of matching "anchor points" is obtained, registration is based on interpolating for matching positions based on relative geodesic distances measured along the arc lengths of the centerlines. Various distance metrics can be used as appropriate.
[0108] Furthermore, "anchor point" fusion can be used to generate an estimate of three-dimensional cardiac motion and deformation. A pair of angiographic images (2D projections) from the same cardiac phase can be used to obtain a three-dimensional reconstruction of the vascular tree. These 3D vasculature reconstructions at multiple phases of the cardiac cycle are useful for understanding cardiac motion in 3D. The displacement of these anchor points at each view along the image sequence guides the computation of motion estimates in the 3D vasculature. Further details on how to perform "anchor point" matching for frame-to-frame registration of the vascular tree are described below and elsewhere in this paper.
[0109] Anatomical feature detection - branching point extraction / detection and grouping FIG. 14 illustrates an exemplary process flow 630 suitable for registering points related to the cardiac system, such as a vascular tree, between first and second angiography frames. This process flow can be used to detect anatomical features and use them for cross-frame / interframe registration. Cross-frame registration can also be achieved by other anatomical features or landmarks found along the vessels. The method 630 can be used to perform interframe registration of the vascular tree as shown.
[0110] In an embodiment, the process of detecting anatomical features, such as branching points, e.g., the division of an artery into first and second vessels or bends for a given vascular tree, and the associated process of grouping such points, can be implemented using various data transformation and image processing stages. Initially, the method determines a sequence of X-ray images and associated centerlines for processing, such as by user selection or other criteria (stage C1).
[0111] The centerlines can be determined as described herein. These X-ray images are subjected to a pre-processing step C2. An example of such a pre-processed image 640 is shown in FIG. 15A. In one embodiment, as a pre-processing step, a skeleton image is generated around each centerline of interest. As a result of brightening the peripheral features and darkening the arterial features as shown in FIG. 15A, the various arterial branches and associated bends and separations and their angles are evident and detectable.
[0112] 15B and 15C are respectively an original angiography image and a skeleton of one of the vessels and its surroundings in that image after application of image processing and data analysis according to an exemplary embodiment of the present disclosure. The skeleton image in FIG. 15C corresponds to the output of step C3. With continued reference to FIG. 14, the method includes steps that can be grouped into two processing paths or categories. In an embodiment, the two processing paths or categories can relate to a first anatomical feature or anatomical landmark, one related to bifurcations and one related to inflection points.
[0113] In one embodiment, the branching related part of the method includes detecting branches in all or a subset of all frames (step C4) and grouping the branches by clustering (step C5). The bend related part of the method includes detecting "bending" point detection in all or a subset of all frames (step C6) and grouping the detected bend points (step C7). These groupings of branches and bend points are then used to perform a frame-to-frame registration of a vascular tree or other vascular structure or a subset thereof (step C8). In general, any group of first and second anatomical structures can be grouped or clustered as described herein and then used to perform a frame-to-frame registration of a vascular tree or other vascular structure or a subset thereof (step C8).
[0114] Branch detection - a feature related to feature extraction In one embodiment, performing feature extraction for bifurcations as described with respect to step C4 of FIG. 14 includes various steps. In one embodiment, detecting bifurcations, such as by performing feature extraction, includes applying a shadow removal filter to the original image. In one embodiment, this filtering is performed using a bottom-hat morphological operation with a large structuring element. This reduces the effect of the shadow of the heart on the diaphragm and further enhances the blood vessels. The output from the previous steps can be processed using a Hessian-based filter image.
[0115] In one embodiment, the output from the Hessian filtering is adaptively thresholded to produce a binary image. In one embodiment, a skeleton generation algorithm is applied to the binary image to obtain a skeleton image. The skeleton image can then be improved by erasing small components from the skeleton image. In one embodiment, the small components are less than about 5 pixels. In one embodiment, the small components are less than about 10 pixels. In one embodiment, the small components are less than about 15 pixels. In one embodiment, the small components are less than about 20 pixels. In one embodiment, the small components are less than about 25 pixels. In one embodiment, the small components are less than about 30 pixels.
[0116] In an embodiment, after the generation of the skeleton image and subsequent refinement, if any, the method can include detecting a set of bifurcations on each frame, for example on the centerline of each frame. In an embodiment, bifurcations are identified as meeting points in the skeleton. To obtain accurate results of bifurcations, removal of spurious, bifurcation-like features, such as vessel crossings and ribs, is required. For that purpose, a series of one or more filters are applied, as follows:
[0117] In one embodiment, a rib filter is applied to reduce any contribution that may be presented by the rib cage or individual ribs in the image and may be mistaken as part of the vascular tree structure. The ribs are mostly static compared to the faster moving blood vessels. Any filter configured to reduce static elements from the image can be used here to image the ribs or other static elements. In one embodiment, a temporal filter is used. Such a filter operates to take every single pixel in the entire image sequence and filter it as if it were a 1D signal. Furthermore, in some embodiments, it is desirable to filter the collection of these signals with a high pass filter to eliminate the static background. An average image is calculated from multiple frames and subtracted from the frame of interest. In this way, static elements such as ribs can be removed or ignored when performing analysis and image processing on the frames of X-ray image data.
[0118] In one embodiment, a vessel crossing filter or detector is applied to the x-ray image frames. In a given frame, vessel crossings may appear as two adjacent branches splitting in opposite directions. Branch separation angles relative to the main centerline are used to address such vessel crossing occurrences and the associated potential for co-registration errors due to vessel crossings. Vessel crossings are also addressed by filtering out branching branches that are located on different sides of the main centerline and that meet the following conditions: adjacent separation locations along the main centerline with an absolute angle difference close to 180°.
[0119] The association or grouping process of anatomical features such as bifurcations is based on clustering. This clustering or grouping process corresponds to step C5 from FIG. 14. In this context, a cluster refers to a single bifurcation extracted from multiple frames. If the same bifurcation is detected in multiple frames, even at slightly different positions and orientations, the set of representative bifurcations across frames should form a cluster indicating that it is the same bifurcation imaged at different times in different frames. In an embodiment, one or more links or vectors can be identified and measured between the clusters. The distance metric between two clusters takes into account the difference between the bifurcation features or whatever features are evaluated and compared. These features can include: angle relative to the main vessel centerline, normalized arc length of the bifurcation and average image intensity along the bifurcation branch, bifurcation scale (or width), absolute angle in angiographic images, and bifurcation lumen shape. Thus, the above features can be used to identify differences and generate distance metrics to evaluate and compare the clustering behavior of a given feature.
[0120] In one embodiment, a branch descriptor space or model is generated using various features. The features include one or more of the mean image intensity value of the branches (I value), the absolute angle of separation of the branches (A value) and the normalized arc length of the branches (S value). Similarity measures and / or distance measures / metrics can be used to relate data points in the feature space. An example of such a metric is the Euclidean metric D(C i ,C j ) can be.
[0121] D(C i ,C j )=sqrt((I i -I j ) 2 +(A i -A j ) 2 +(S i -S j )2 ) D(C i ,C j ), where I represents the average image intensity, A represents the absolute angle of the isolated branch, and S represents the normalized arc length of the branch. The indices i and j correspond to different angiography frames i and j.
[0122] Clusters of branch data sets that represent the same branch in multiple frames are detected and / or selected. The detection / selection process can be performed using feature extraction. The use of feature extraction is beneficial given the presence of image noise and missing information in the clusters. In one embodiment, a branch is detected on multiple image frames, and this set of detections across frames can be clustered together, helping to verify that the branch is the same across different frames. Feature extraction, in one embodiment, includes filtering out extraneous data such as image noise. Additionally, feature extraction can include filling in missing information in one or more clusters, such as by interpolation or other processes.
[0123] Clusters of a certain size (large clusters, small or medium sized clusters) are selected for the next processing stage in one embodiment. Clusters are identified as being of a suitable large size for selection and further processing based on their number of representatives compared to the set of angiography frames captured during the OCT pullback. Once selected, the clusters are used in the cluster merging stage. The process of merging clusters produces a set of clusters, each with a single representative from each frame of interest.
[0124] FIG. 16A shows a subset of angiographic images with a bifurcation formed from two branches B1 and B2. FIG. 16B shows a representation of that portion of a vascular system showing the bifurcation as part of a vascular tree and the associated angle β between the vascular branch and its underlying (parent) vessel, according to an exemplary embodiment of the present disclosure. As shown, in FIG. 16B, the angle of the bifurcation relative to the (parent) vascular branch B1 is shown as angle β in the schematic representation of the bifurcation. Angle β is formed from the confluence of branches B1 and B2, and is shown by an arrow as the opening from B1 to B2. These branches are plotted or otherwise grouped based on angle and arc length in an embodiment shown in FIG. 17A.
[0125] Cluster integration FIG. 17A shows multiple clusters generated from plotting normalized arc length (vertical axis) against angular measure (horizontal axis). Three clusters of interest are labeled 1, 2, and 3. Cluster 1 is to the left of cluster 2 and contains a larger number of data points. The data points are shown as circular regions. Cluster 3 is above cluster 1 and has a larger number of overlapping data points along a rectangular region that spans the cluster from top to bottom. In one embodiment, a redundancy elimination step is performed.
[0126] For example, for clusters containing multiple representatives from the same frame, a single point closest to the cluster centroid is selected. FIG. 17B shows the corresponding locations for each cluster on the frames of angiography data analyzed to generate the clusters. It is desirable to have a single cluster representative from each frame. Thus, if a cluster lacks a representative from a frame or frames, an interpolated value based on the nearest frames is used to complete the cluster as a substitute for the missing information. Any type of interpolation (linear, cyclic, spline-based, curve fitting, etc.) can be implemented to complete a cluster or define the boundaries of such a cluster. The elements of a cluster are, in one embodiment, the basis for treating such elements as the same elements across images.
[0127] Cluster Selection Each branch cluster is assigned a quality grade that is used for cluster selection. The following factors go into the grade: arc length standard deviation, normalized arc length standard deviation, angle difference standard deviation, proximity to other branch clusters (based on the distance between the centroids of different clusters), average number of redundant branch records per frame, average number of missing branch records per frame. A weighted average including these various factors can be used to generate the grade. The clusters with the best grades are finally selected.
[0128] <Vascular tortuosity feature extraction / detection and features> As shown in FIG. 14, the method also includes a bend-related pass, including steps C6 and C7. Further details regarding these steps are provided below. A set of anatomical "anchors" are extracted as features based on the location of vascular bends. A vascular bend is defined as a point where a vessel changes its direction, creating a corner-like structure that exhibits high curvature. At each frame, a multi-scale curvature or curvature analog is extracted from the vessel centerline described above.
[0129] In one embodiment, the method uses a tracing or shortest path algorithm, such as the Viterbi algorithm, to determine the location of each bend in every frame or in a sample of frames. In various embodiments, feature extraction is used to detect anatomical and other features or landmarks in a given image frame. In one embodiment, shortest path features are extracted from all frames of interest by optimizing a cost criterion based on the bend angle size, the bend location along the vessel centerline using arc length or normalized arc length, and the difference in angular deviation between two bends in consecutive frames. In one embodiment, a collection of bends and branches across various frames is used to identify a path or otherwise perform registration between angiographic frames.
[0130] After the cost metrics are calculated from multiple starting points, solutions are extracted based on the ranking of the associated cost metrics for a given solution. After all the bend candidates from multiple frames are extracted, a filtering step may be applied to eliminate solutions that originate from small bends or bends that display inconsistent positions along the vessel centerline.
[0131] FIG. 18A and B depict two angiographic frames of multiple vascular trees. In each frame 700, 705, three bends were detected from the two X-ray frames. Centerline detection was performed for each X-ray frame. The resulting detected centerlines are depicted by white lines superimposed on each frame. Bend detection was also performed for each X-ray frame. Bend locations are indicated by white diamonds. In each of the left image frame 700 and the right image frame 705, three bends are shown, numbered 1, 2, 3 from bottom to top. Each vascular centerline traces a path through the artery along bends 3, 2, 1. The location in all angiographic frames of a particular bend can be used as an anatomical anchor or reference point to accurately identify the frame transverse location.
[0132] Further supporting details on skeleton and vessel centerline generation Furthermore, in an embodiment, anatomical feature detection is performed as part of the pre-processing of the angiographic image. In an embodiment, this can be performed to generate some a priori information related to the path the probe will take in the blood vessel. The generation of lines, such as through a skeleton generation process, can be used for feature detection. In an embodiment, the skeleton is a static object, such as one or more lines, that is generated to help trace the blood vessels of the subject being imaged.
[0133] The use of a skeleton or line-segment-based approach to generate candidate paths through vessels for a data collection probe that can be used to inform centerline generation and marker tracking offers several advantages over not using such approaches. For example, a skeleton-based approach can prevent the generation of certain vessel centerlines that would otherwise pass through branches or the imaging probe catheter.
[0134] Generating a skeleton provides a method to determine initial candidates for the geometry of the imaged vessels and branches and other vessels as a map or framework to facilitate centerline generation. Generating a skeleton allows for extraction of points of interest such as branching points and vessel segments to stabilize tracking of markers and vessel centerlines and to verify tracking quality through frames of angiographic image data.
[0135] In one embodiment, a process of generating a skeleton to detect anatomical features such as branching and vessel geometry is implemented during pre-processing of the angiographic image. The skeleton can be used to detect anatomical features such as major branching and extrapolation points. Furthermore, the skeleton can be used to detect and generate smooth vessel centerlines. For example, the skeleton can be used with shortest path algorithms such as Viterbi, Deijkstra algorithms or other algorithms to facilitate centerline generation. The skeleton can be generated based on the pre-processed Hessian image. User-selected points on the angiographic image related to the guidewire position can be used to reduce noise and facilitate skeleton generation. In another embodiment, this can be implemented by selecting points based on image features.
[0136] In one embodiment, one or more software modules are used to generate and track a vascular centerline for a given frame of angiographic data. In one embodiment, the vascular centerline, also referred to herein as a centerline, is a model or simulation that is generated based on interactively evaluating each candidate subset of frames of angiographic data for marker bands associated with optical or acoustic sensors or other imaging or data collection sensors deployed during angiographic data acquisition.
[0137] In one embodiment, a dynamic program software module, such as a software module implementing one or more stages of any suitable shortest path or optical path determination algorithm, such as the Viterbi algorithm, is used for tracking of the radiopaque markers. Centerline generation and tracking is typically handled by other algorithms or combinations thereof. Centerline tracking can be enhanced by using feature detection, such as guidewire or landmark detection, to define centerline endpoints. Defining centerline endpoints advantageously increases confidence in frame cross registration and centerline determination.
[0138] The use or absence of directional arrows in a given figure is not intended to limit or dictate the direction in which information may flow. For a given connector, such as the arrows and lines shown connecting elements shown in Figure 1, information may flow in one or more directions, or in only one direction, as suitable for a given embodiment. The connections may include various suitable data transmission connections, such as optical, wired, power, wireless, or electrical connections.
[0139] Non-Limiting Software Features and Embodiments for Implementing Angiography and Intravascular Data Collection Methods and Systems The following description is intended to provide an overview of apparatus hardware and other operating components suitable for carrying out the disclosed methods described herein. This description is not intended to limit the applicable environment or scope of the present disclosure. Similarly, the hardware and other operating components may be suitable as part of the above-described apparatus. The present disclosure may be practiced with other system configurations, including personal computers, multiprocessor systems, microprocessor-based or programmable electronic devices, network PCs, minicomputers, mainframe computers, and the like.
[0140] Some portions of the detailed description are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations may be used by those skilled in the art of computers and software. In one embodiment, an algorithm is here, and generally, conceived to be a self-consistent sequence of operations leading to a desired result. The operations described herein as method steps or otherwise performed involve physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, transformed, compared, and otherwise manipulated.
[0141] Unless otherwise noted, and as will be apparent from the discussion below, discussions throughout this document utilizing terms such as "processing" or "computing" or "calculating" or "comparing" or "arc-length measuring" or "detecting" or "tracing" or "masking" or "sampling" or "clustering" or "feature extraction" or "adaptive thresholding" or "operating" or "generating" or "determining" or "displaying" or "discovering" or "extracting" or "filtering" or "avoiding" or "excluding" or "interpolating" or "optimizing" and the like, are understood to refer to the actions and processes of a computer system or similar electronic computing apparatus that manipulates data represented as physical (electronic) quantities in the registers or memory of the computer system, and similarly converts data into other data represented as physical quantities in the computer system memory or registers or other such information storage, transmission or display device.
[0142] The present disclosure also relates in some embodiments to apparatus for performing the operations herein, which apparatus may be specially constructed for the required purposes, or it may be a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer.
[0143] The algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will be apparent from the description below.
[0144] Embodiments of the present invention may be implemented in many different forms, including, but in no way limited to, computer program logic for use with a processor (e.g., a microprocessor, microcontroller, digital signal processor, or general purpose computer), programmable logic for use with a programmable logic device (e.g., a field programmable gate array (FPGA) or other PLD), discrete components, integrated circuits (e.g., an application specific integrated circuit (ASIC)), or any other means including combinations thereof. In exemplary embodiments of the present disclosure, some or all of the processing of data collected using OCT probes, FFR probes, angiography systems, and other imaging and subject monitoring devices and processor-based systems is implemented as a set of computer program instructions that are converted into a computer executable format and stored as such in a computer readable medium and executed by a microprocessor under the control of an operating system. In this manner, user interface instructions and triggers based on pullback completion or co-registration requests are converted into processor understandable instructions suitable for generating OCT data and performing image processing using the various and other features and embodiments described above.
[0145] Computer program logic implementing all or a portion of the functionality described above may be embodied in a variety of forms, including but not limited to source code form, computer executable form, and various intermediate forms (such as those generated by an assembler, compiler, linker, or locator). The source code may include a series of computer program instructions implemented in any of a variety of programming languages (e.g., object code, assembly language, or high level languages such as Fortran, C, C++, JAVA, or HTML) for use with a variety of operating systems or operating environments. The source code may define and use various data structures and communication messages. The source code may be in a computer executable form (e.g., via an interpreter) or the source code may be converted into a computer executable form (e.g., via a translator, assembler, or compiler).
[0146] The computer program may be fixed, permanently or temporarily, in any form (source code form, computer executable form or intermediate form) on a tangible storage medium, such as a semiconductor memory device (e.g., RAM, ROM, PROM, EEPROM or flash-programmable RAM), a magnetic memory device (e.g., diskette or fixed disk), an optical memory device (e.g., CD-ROM), a PC card (e.g., PCMCIA card) or other memory device. The computer program may be fixed in any form of signal that can be transmitted to a computer using any of a variety of communication technologies, including, but not limited to, analog, digital, optical, wireless (e.g., Bluetooth), networking and internetworking technologies. The computer program may be distributed in any form on a removable storage medium (e.g., shrink-wrapped software), preloaded into a computer system (e.g., in a system ROM or fixed disk) together with accompanying printed or electronic documentation, or distributed over a communication system (e.g., the Internet or World Wide Web) from a server or electronic bulletin board.
[0147] Hardware logic (including programmable logic for use with programmable logic devices) implementing all or a portion of the functionality described hereinabove may be designed using traditional manual methods, or may be designed, captured, simulated or documented electronically using a variety of tools such as computer-aided design (CAD), hardware description languages (e.g., VHDL or AHDL), or PLD programming languages (e.g., PALASM, ABEL or CUPL).
[0148] The programmable logic may be fixed, either permanently or temporarily, in a tangible storage medium, such as a semiconductor memory device (e.g., RAM, ROM, PROM, EEPROM, or flash-programmable RAM), a magnetic memory device (e.g., diskette or fixed disk), an optical memory device (e.g., CD-ROM), or other memory device. The programmable logic may be fixed in a signal that can be transmitted to a computer using any of a variety of communication technologies, including, but not limited to, analog, digital, optical, wireless (e.g., Bluetooth), networking, and internetworking technologies. The programmable logic may be distributed on a removable storage medium (e.g., shrink-wrapped software) along with accompanying printed or electronic documentation, preloaded into a computer system (e.g., in a system ROM or fixed disk), or distributed over a communication system (e.g., the Internet or World Wide Web) from a server or electronic bulletin board.
[0149] Various examples of suitable processing modules are discussed in more detail below. As used herein, a module refers to software, hardware, or firmware suitable for performing a particular data processing or data transmission task. In one embodiment, a module refers to a software routine, program, or other memory resident application suitable for receiving, converting, routing, and executing feature extraction and processing instructions or various types of data. The data may be, for example, angiography data, OCT data, IVUS data, frame cross-section data, pixel coordinates, clusters, clouds, opacity regions, centerlines, shadows, pixels, clusters, distance metrics, intensity patterns, anatomical features, anatomical landmarks, bifurcations, bends, and other information of interest as described herein.
[0150] The computers and computer systems described herein may include operatively associated computer-readable media, such as memory, for storing software applications used in acquiring, processing, storing and / or communicating data, it being understood that such memory may be internal, external, remote or local with respect to the operatively associated computer or computer system.
[0151] Memory may include any means for storing software or other instructions, including, for example, without limitation, a hard disk, an optical disk, a floppy disk, a DVD (digital versatile disk), a CD (compact disk), a memory stick, flash memory, a ROM (read only memory), a RAM (random access memory), a DRAM (dynamic random access memory), a PROM (programmable ROM), an EEPROM (extended erasable programmable read only memory), and / or other similar computer readable medium.
[0152] In general, computer-readable memory media applied in connection with the embodiments of the present disclosure described herein may include any memory medium capable of storing instructions executed by a programmable device. Where applicable, the methods described herein may be embodied or executed as instructions stored on a computer-readable memory medium or memory media. These instructions may be software embodied in various programming languages, such as C++, C, Java, and / or various other types of software programming languages that may be adapted to generate instructions according to the embodiments of the present disclosure.
[0153] The aspects, embodiments, features, and examples of the present disclosure are considered in all respects to be illustrative and are not intended to limit the present disclosure. The scope of the present disclosure is defined only by the claims. Other embodiments, modifications, and uses will be apparent to those skilled in the art without departing from the spirit and scope of the claimed disclosure.
[0154] The use of headings and sections in this application is not intended to limit the disclosure: each section may apply to any aspect, embodiment, or feature of the disclosure.
[0155] Throughout this application, when a composition is described as having, including, or having particular components, or a process is described as having, including, or having particular process steps, it is contemplated that the composition of the teachings herein consists essentially of or consists of the described components, and that the process of the teachings herein consists essentially of or consists of the described process steps.
[0156] It should be understood that when an element or component is described herein as being included in and / or selected from a list of described elements or components, the element or component can be any one of the described elements or components, or can be selected from a group consisting of two or more of the described elements or components. Moreover, it should be understood that the elements and / or features of the compositions, devices or methods described herein can be combined in various ways, whether explicitly stated or implicitly stated herein, without departing from the spirit and scope of the teachings of the present application.
[0157] Use of the terms "including," "having," or "having" should generally be understood as open ended, without limitation, unless otherwise specified.
[0158] The use of the singular in this document includes the plural (and vice versa) unless expressly stated otherwise. Furthermore, the singular includes the plural unless the context clearly dictates otherwise. Furthermore, when the term "about" is used before a quantitative value, the teachings of this application also include the specific quantitative value itself, unless expressly stated otherwise. As used herein, the term "about" refers to a ±10% variation from the nominal value.
[0159] It should be understood that the order of steps or order for performing certain actions is immaterial so long as the teachings of the present application remain operable. Moreover, two or more steps or actions may be conducted simultaneously.
[0160] When a range or list of values is given, each intermediate value between the upper and lower limits of that range or list of values is individually contemplated and encompassed in the disclosure as if each value were specifically recited herein. Additionally, intermediate, smaller ranges that include the upper and lower limits of a given range are contemplated and encompassed within the disclosure. A list of exemplary values or ranges does not disclaim other intermediate values or ranges that include the upper and lower limits of a given range.
[0161] It should be understood that various aspects of the claimed disclosure are directed to subsets and substeps of the techniques disclosed herein. Moreover, the terms and expressions employed herein are used as terms of description rather than of limitation, and the use of such terms and expressions is not intended to exclude any equivalents of the features shown and described, or portions thereof. It is recognized that various modifications are possible within the scope of the claimed disclosure. Thus, what is desired to be protected by Letters Patent is the disclosure as defined and distinguished in the appended claims, including all equivalents.
[0162] Several aspects will be described. [Aspect 1] 1. A computer device based method for detecting one or more regions of interest in one or more x-ray images, comprising: storing in an electronic memory device a set of angiographic image frames obtained during a first time period that includes one or more time periods during which contrast agent is in a blood vessel; detecting a plurality of candidate contrast cloud regions in one or more angiography frames; and combining the plurality of candidate contrast cloud regions to generate a fused contrast cloud region, the fused contrast cloud region having a cloud boundary. method. [Aspect 2] 2. The method of embodiment 1, further comprising excluding angiographic image data located within the fused contrast cloud region from one or more subsequent data processing methods. [Aspect 3] The method of embodiment 2, wherein the one or more subsequent data processing methods include generating vascular centerlines. Aspect 4 The method of embodiment 3, comprising generating one or more vascular centerlines for one or more angiography image frames. Aspect 5 3. The method of embodiment 2, wherein the one or more subsequent data processing methods include cross-frame registration of two or more angiographic image frames of the set of angiographic image frames. Aspect 6 The method of embodiment 5, further comprising performing cross-frame registration of two or more angiographic image frames of said set of angiographic image frames. Aspect 7 7. The method of embodiment 6, further comprising generating a set of cross-frame locations. Aspect 8 3. The method of embodiment 2, wherein the one or more subsequent data processing methods include a process selected from the group consisting of: anchor point extraction; arc length interpolation; and frame cross position generation. Aspect 9 The method of embodiment 2, further comprising expanding one or more candidate regions or the fused contrast cloud region to provide an expanded exclusion zone, and further comprising excluding angiography image data located within the expanded exclusion zone from one or more subsequent data processing methods. Aspect 10 The method of embodiment 2, further comprising co-registering the OCT image data and a plurality of said angiography image frames. Aspect 11 2. The method of embodiment 1, further comprising applying a smoothing filter to one or more angiography frames such that elements having a particular desired scale are preserved. Aspect 12 12. The method of embodiment 11, further comprising adaptively thresholding the smoothed angiography frame from which small-scale elements have been removed to generate a binary image, wherein in the binary image, one or more pixels within a contrast cloud region are a first value. Aspect 13 13. The method of embodiment 12, further comprising selecting a contrast cloud neighborhood to include a cloud region having the first value in the binary image and overlaying the neighborhood on one or more angiography frames. Aspect 14 14. The method of embodiment 13, further comprising counting or scoring the number of pixels in said neighborhood that contain said first value. Aspect 15 2. The method of embodiment 1, further comprising adaptively thresholding pixels that include a first value to remove pixels from each neighborhood that include values other than the first value. Aspect 16 The method of embodiment 1, further comprising generating a mask and using the mask to detect stable anatomical anchor points that serve as proximal endpoints of the vascular centerline. Aspect 17 2. The method of embodiment 1, further comprising the steps of: detecting a plurality of anatomical features in the set of angiography image frames; generating clusters of the detected anatomical features; and performing cross-frame registration between the angiography frames using the clusters, where a cluster refers to a single anatomical feature extracted from a plurality of frames. Aspect 18 2. The method of embodiment 1, further comprising grouping a plurality of anatomically associated inflection points using a shortest path finding algorithm to identify likely paths through the associated inflection points. Aspect 19 The method of embodiment 18, wherein the likely path is identified in response to one or more criteria selected from the group consisting of bend angle change, curvature, curvature analog, bend position along the centerline, and difference in bend angle deviation between consecutive frames. Aspect 20 The method of embodiment 1, further comprising applying an image processing transform to one or more frames to remove or modify features in at least one frame and generate a mask for performing feature extraction during one or more subsequent image processing stages. Aspect 21 2. The method of claim 1, further comprising co-registering the plurality of frames of angiography image data and the plurality of frames of optical coherence tomography data using a co-registration table, the co-registration table including angiography image frames, a plurality of frame-by-frame OCT timestamps, a plurality of frame-by-frame angiography timestamps, and optical coherence tomography image frames, and including a score measurement for each co-registered position. Aspect 22 22. The method of embodiment 21, further comprising displaying, in a user interface using the mutual registration table and a computing device, a representation of the stent in the OCT image and the angiography image. Aspect 23 The method of embodiment 21, further comprising identifying branches in one or more OCT or angiography images using the co-registration table and a user interface configured to display the branches. Aspect 24 2. The method of embodiment 1, further comprising displaying a plurality of frame transverse registered angiographic images using a diagnostic system, wherein the plurality of frame transverse registered angiographic images are selected from the set. Aspect 25 The method of embodiment 1, wherein one or more steps of the method are implemented using a diagnostic system having an input for receiving the set from an angiography system, one or more electronic memory devices for storing the set, one or more computing devices in electrical communication with the input and the one or more memory devices, and instructions, image filters and image processing software modules executable by the one or more computing devices to perform one or more steps of the method. Aspect 26 The method of embodiment 17, wherein the anatomical feature is a branch or inflection point in a vascular tree.
Claims
1. receiving, by one or more processors, vascular data, intravascular image data, and angiography image data; identifying a contrast cloud region in the angiography image data; the one or more processors determining a subset of the angiographic image data that excludes regions of the angiographic image data that include the identified contrast cloud regions; the one or more processors co-registering the intravascular image data and a subset of the angiography image data; determining, by the one or more processors, blood flow data based on pressure measurements based on the received vascular data; the one or more processors determining at least one of a pressure-based measurement or a blood flow measurement at a plurality of points along a blood vessel based on the blood flow data; providing, as output, the one or more processors, a subset of the co-registered intravascular image data and the co-registered angiographic image data and an indication of at least one of the pressure-based measurement or the blood flow measurement, the indication being provided on at least one of the subset of the intravascular image data or the angiographic image data; The method includes:
2. The step of co-registering the intravascular image data and the subset of the angiography image data comprises: generating a score indicative of a level of confidence in a co-registration between the frame of intravascular image data and the subset of angiographic image data; generating a co-registration table including a plurality of angiographic image frames, a plurality of intravascular image timestamps for each frame, a plurality of angiographic image timestamps for each frame, and a plurality of intravascular image frames including the scores, the one or more processors; The method of claim 1 , comprising:
3. The step of co-registering the intravascular image data and the subset of the angiography image data comprises: generating a centerline of the subset of angiographic image data, the one or more processors; detecting a marker in the subset of angiographic image data by the one or more processors; the one or more processors tracking a position of the marker along the centerline; co-registering the intravascular image data and a subset of the angiography image data based on the tracked locations, The method of claim 1 , comprising:
4. The method of claim 1 , wherein the intravascular image data is at least one of optical coherence tomography (OCT) image data or intravascular ultrasound (IVUS) image data.
5. The method of claim 1 , wherein the perfusion data is fractional flow reserve (FFR) data.
6. The method of claim 1 , wherein receiving the vascular data comprises the one or more processors receiving pressure measurements from a pressure wire.
7. The method of claim 6 , further comprising the one or more processors co-registering the pressure measurements using the intravascular image data and a subset of the angiography image data.
8. The method of claim 1 , further comprising the one or more processors providing as an output a longitudinal representation of the blood vessel.
9. The method of claim 1 , further comprising the one or more processors providing as output a stent representation in at least one of the subset of the intravascular image data or the angiography image data.
10. The method of claim 9 , wherein the stent representation is a pre-stent deployment representation or a post-stent deployment representation.
11. receiving vascular data, intravascular image data, and angiography image data; Identifying contrast cloud regions in the angiography image data; determining a subset of the angiographic image data that excludes regions of the angiographic image data that include the identified contrast cloud regions; co-registering the intravascular image data with a subset of the angiography image data; determining blood flow data based on the pressure measurements based on the received vascular data; determining at least one of a pressure-based measurement or a blood flow measurement at a plurality of points along a blood vessel based on the blood flow data; providing as output the co-registered intravascular image data and a subset of the co-registered angiographic image data and an indication of at least one of the pressure-based measurement or the blood flow measurement, the indication being provided on at least one of the subset of the intravascular image data or the angiographic image data.
23. A system comprising one or more processors, configured to:
12. When co-registering the intravascular image data and the subset of the angiography image data, the one or more processors: generating a score indicative of a level of confidence in the co-registration between the frame of intravascular image data and the subset of angiographic image data; generating a co-registration table including a plurality of angiographic image frames, a plurality of intravascular image timestamps for each frame, a plurality of angiographic image timestamps for each frame, and a plurality of intravascular image frames including the scores; The system of claim 11 further configured to:
13. When co-registering the intravascular image data and the subset of the angiography image data, the one or more processors: generating a centerline of the subset of angiographic image data; detecting a marker in the subset of angiographic image data; Tracking the position of the marker along the centerline; Co-registering the intravascular image data and the subset of the angiography image data based on the tracked locations. The system of claim 11 further configured to:
14. The system of claim 11 , wherein the intravascular image data is at least one of optical coherence tomography (OCT) image data or intravascular ultrasound (IVUS) image data.
15. The system of claim 11 , wherein the perfusion data is fractional flow reserve (FFR) data.
16. The system of claim 11 , wherein receiving the vascular data includes the one or more processors receiving pressure measurements from a pressure wire.
17. 17. The system of claim 16, wherein the one or more processors are further configured to co-register the pressure measurements using the intravascular image data and a subset of the angiography image data.
18. The system of claim 11 , wherein the one or more processors are further configured to provide a longitudinal representation of the blood vessel.
19. The system of claim 11 , wherein the one or more processors are further configured to provide as output a stent representation in at least one of the subset of the intravascular image data or the angiography image data.
20. When executed by one or more processors, receiving vascular data, intravascular image data, and angiography image data; Identifying contrast cloud regions in the angiography image data; determining a subset of the angiographic image data that excludes regions of the angiographic image data that include the identified contrast cloud regions; co-registering the intravascular image data with a subset of the angiography image data; determining blood flow data based on the pressure measurements based on the received vascular data; determining at least one of a pressure-based measurement or a blood flow measurement at a plurality of points along a blood vessel based on the blood flow data; providing as output a representation of the co-registered intravascular image data and a subset of the co-registered angiographic image data and at least one of the pressure-based measurement or the blood flow measurement, the representation being provided on at least one of the intravascular image data or the subset of the angiographic image data; 23. A computer-readable storage medium comprising instructions for causing the one or more processors to execute a
Citation Information
Patent Citations
Automated method and automated device for analyzing region affected by embolism in organ, and computer readable medium in which computer program code for analyzing region affected by embolism in organ is stored
JP2006081906A
X-ray diagnostic apparatus and image processing apparatus
JP2010131371A
Dynamic normalization of data for perfusion comparison and quantification
JP2014511713A
Temporal anatomical target tagging in angiography
JP2015519987A
Method of delivering cell therapy to a target site
US20100226903A1
