Vascular selection from images

Abstract

To provide methods and systems for manually assisted determination of vascular features.SOLUTION: In some embodiments, a vascular center line identified with image analysis is divided into segments. Optionally, one or more candidate paths extending to a root position of a blood vessel along the continuous segments is generated for each of a plurality of vascular end points. The anatomical path selection of the blood stream is based on the optionally manual confirmation. Optionally, the path is manually determined on the basis of, for example, selection of a plurality of way points. Optionally, the path is edited by the use of a dynamic profile method that enables the relatively rough display of correction to the path change that strictly matches the geometric shape of a nearby vascular system.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure generally relates to the field of anatomical segmentation, and more particularly to manually assisted segmentation of the anatomical structure of bifurcated blood vessels.

Background Art

[0002] Segmentation of blood vessels and identification of features are preliminary steps in image-based measurements of the state of blood vessels. Many steps of blood vessel segmentation and feature identification can be performed preliminarily based on automatic analysis, but the features of the related images are often of low contrast and / or are incorporated into a complex environment containing ambiguous geometric and irrelevant feature elements. For example, human supervision may be introduced into the workflow to help make corrections and ensure the quality of the results, leading to semi-automatic processing, such as with Livewire and related procedures (discussed in Non-Patent Document 1).

[0003] Additional background art includes the papers of Non-Patent Document 2, Non-Patent Document 3, and Non-Patent Document 4.

Prior Art Documents

Non-Patent Documents

[0004]

Non-Patent Document 1

Non-Patent Document 2

[0005] According to some exemplary embodiments, a method is provided for segmenting a blood vessel image for determining a blood flow path into a blood vessel path. The method includes determining terminal regions of first and second target blood vessel paths within the blood vessel image, identifying positions of portions of blood vessels within the blood vessel image, and automatically generating a plurality of blood vessel path options from the identified portions of blood vessels, each blood vessel path option being characterized by determining a potential blood vessel path extending between the terminal regions of the first and second target blood vessel paths. The method may also include displaying the plurality of blood vessel path options registered with the blood vessel image for selection by a user, each of the displayed blood vessel path options including the first and second target blood vessel path terminal regions. The exemplary method may further include receiving a path option selected by the user for determination of the blood flow path.

[0006] According to some embodiments, a plurality of paths are automatically generated based on a first set of criteria, and the path options selected by the user are selected based on a second set of criteria.

[0007] According to some embodiments, pre - determining includes ranking the vascular path options in order based on an evaluation of the likelihood that each vascular path option corresponds to the actual path of blood flow within the blood vessels imaged in the vascular image.

[0008] According to some embodiments, pre - determining includes applying a cost function that assigns a numerical cost to one or more features associated with the vascular path options.

[0009] According to some embodiments, the cost function assigns a numerical cost based on the centerline and the characteristics of the centerlines of the plurality of vascular segments to which the vascular path option is connected from the centerline.

[0010] According to some embodiments, the characteristics of the centerlines of the plurality of vascular segments include one or more of the group consisting of the direction of the centerline, the centerline offset, and the number of centerlines extending from the node region.

[0011] According to some embodiments, the cost function assigns a numerical cost based on the characteristics of the vascular image on which the vascular path option extends.

[0012] According to some embodiments, the characteristics of the vascular image include one or more of the group consisting of the continuity of the vascular segment image intensity, the continuity of the width of the vascular segment image, and the position of the relative change in vascular intensity with respect to the node region, where three or more vascular segments extend from the node region.

[0013] According to some embodiments, pre-determining includes applying a cost function that assigns a numerical cost based on an estimated relative position of a vascular segment image in depth with respect to an axis extending perpendicular to the plane of the vascular image.

[0014] According to some embodiments, displaying includes presenting the plurality of vascular path options in a sequential order determined by a selection order.

[0015] According to some embodiments, displaying includes presenting the plurality of vascular path options simultaneously, and the order of selection corresponds to the order in which the vascular path options are presented as active for selection.

[0016] According to some embodiments, each vascular path option determines a vascular path that extends through an image region between terminal regions of the first and second target vascular paths and terminates at a vascular region of an image that is closest to one of the terminal regions of the first and second target vascular paths.

[0017] According to some exemplary embodiments, a method for editing a vascular path is provided to more accurately depict the segmentation of a blood vessel in a vascular image. The exemplary method includes receiving an indication of a selected region along the segmentation of the blood vessel, determining an energy functional determined as a function of a position along the segmentation of the blood vessel, and setting a non-zero region of the energy functional based on the position of the selected region. The method may also include moving the segmented region in accordance with minimization of energy within the range of the non-zero region of the energy functional.

[0018] According to some embodiments, the energy functional values in the non-zero region are set based on features of the underlying vascular image.

[0019] According to some embodiments, the energy functional values in the non-zero region are set based on a movement of an indication of a position controlled by a user.

[0020] According to some embodiments, the indication of the position controlled by the user includes an indication of a selection area.

[0021] According to some exemplary embodiments, a user interface for semi - automatic segmentation of a vascular path is provided, the user interface comprising at least one interface module operable to present a default vascular path automatically generated that extends between two target end points, and at least one interface module operable to present at least one additional automatically generated vascular path that extends between the two target end points.

[0022] According to some embodiments, the user interface further comprises at least one interface module configured to be able to determine at least one way point and operable to present an automatically generated vascular path that extends between the two target end points via the at least one way point.

[0023] According to some embodiments, the user interface further comprises at least one interface module operable to modify a pre - determined vascular path by dragging the position of the vascular path to a new location.

[0024] Unless defined otherwise, all technical and / or scientific terms used herein have the same meaning as commonly understood by one of ordinary skill in the art to which this disclosure pertains. The systems, methods, devices and / or computer program products described herein, or their similar or equivalent counterparts, can be used in the practice or testing of embodiments of the disclosure, but exemplary systems, methods, devices and / or computer program products are described below. In case of conflict, the patent specification, including definitions, will control. Additionally, the systems, methods, devices, computer program products and examples are for illustrative purposes only and are not necessarily intended to be limiting.

[0025] As will be apparent to those skilled in the art, aspects of the present disclosure may be embodied as a system, method or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an embodiment combining software and hardware aspects. Furthermore, some embodiments of the present disclosure may take the form of a computer program product embodied in one or more computer-readable media having computer-readable program code embodied therein. The realization of the methods and / or systems of some embodiments of the present disclosure may include performing and / or completing selected tasks manually, automatically, or a combination thereof. Further, according to the actual instrumentation and equipment of some embodiments of the methods and / or systems of the present disclosure, some selected tasks may be implemented by hardware, by software, or by firmware and / or a combination thereof, such as the use of an operating system.

[0026] For example, the hardware for performing selected tasks according to some embodiments of the present disclosure may be implemented as a chip or a circuit. As software, selected tasks according to some embodiments of the present disclosure may be implemented as a plurality of software instructions executed by a computer using any suitable operating system. In exemplary embodiments of the present disclosure, according to some exemplary embodiments of the methods and / or systems described herein, one or more tasks are performed by a data processor such as a computer platform for executing a plurality of instructions. Optionally, the data processor includes a volatile memory for storing instructions and / or data and / or a non-volatile storage medium for storing instructions and / or data, such as a magnetic hard disk and / or removable media. Optionally, a network connection may also be provided. A display and / or a user input layer value such as a keyboard or a mouse may also be provided.

[0027] Any combination of one or more computer-readable media may be used for some embodiments of the present disclosure. A computer-readable media may be a computer-readable signal media or a computer-readable storage media. The computer-readable storage media may be, but is not limited to, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples (a non-exhaustive list) of the computer-readable storage media include electrical connections having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the context of this document, a computer-readable storage device can be any tangible media that can include or store a program for use by or in connection with a system, apparatus, or device that executes instructions.

[0028] A computer-readable signal media may include, for example, a propagated data signal having computer-readable program code embedded therein, either in baseband or as part of a carrier wave. Such a propagated signal may take any form, including, but not limited to, electromagnetic waves, optical, or any combination thereof. A computer-readable signal media is not a computer-readable storage media, but may be any computer-readable media that can communicate, propagate, or transmit a program for use by or in connection with a system, apparatus, or device that executes instructions.

[0029] Program code embodied on a computer-readable medium and / or data used thereby may be transmitted using any appropriate medium including, but not limited to, wireless, wired, fiber optic cable, RF, etc., or any suitable combination thereof.

[0030] Computer program code for carrying out operations for some embodiments of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages such as Java®, Smalltalk, C++, etc., and conventional procedural programming languages such as the "C" programming language or similar programming languages. The program code may execute entirely on the user's computer, partly on the user's computer, as a stand-alone software package, partly on the user's computer and partly on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet service provider).

[0031] Some embodiments of the present disclosure will be described below with reference to the flowchart diagrams and / or block diagrams of methods, apparatuses (systems), and computer program products according to embodiments of the present disclosure. It is understood that each block of the flowchart diagrams and / or block diagrams, and combinations of blocks in the flowchart diagrams and / or block diagrams, can be implemented by computer program instructions. These computer program instructions may be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing apparatus to generate means for performing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0032] These computer program instructions may be stored in a computer-readable medium such that the instructions stored in the computer-readable medium generate a manufacture including instructions for performing the functions / operations specified in the blocks of the flowchart and / or block diagram, and the instructions can direct a computer, other programmable data processing apparatus, or other devices to function in a particular manner.

[0033] These computer program instructions may be loaded onto a computer, other programmable data processing apparatus, or other devices to cause a series of operational steps to be performed on the computer, other programmable data processing apparatus, or other devices to provide a process for realizing the functions / operations specified in the blocks of the flowchart and / or block diagram.

[0034] Additional features and advantages of the systems, methods, and apparatuses of the present disclosure will be described in the following detailed description and drawings and will become apparent therefrom.

[0035] Some embodiments of the exemplary systems, methods, and / or computer program products are described herein by way of example only with reference to the accompanying drawings. Here, referring specifically to the particular drawings in detail, it is emphasized that the details are by way of example and for purposes of embodiments of the systems, methods, apparatuses, and / or computer program products. In this regard, the description made with the drawings will clarify to those skilled in the art how embodiments of the systems, methods, apparatuses, and / or computer program products can be implemented.

Brief Description of the Drawings

[0036]

Figure 1

Figure 2

Figure 3A

Figure 3B

Figure 4A

Figure 4B

Figure 4C

Figure 4D

Figure 4E

Figure 4F

Figure 5A

Figure 5B

Figure 5C

Figure 5D

Figure 5E

Figure 5F

Figure 6

Figure 7

[0037] In some embodiments of the present disclosure, the present disclosure relates to the field of anatomical segmentation, and in particular, to the manually assisted segmentation of the anatomical structure of branched blood vessels.

[0038] Summary Broad aspects of some embodiments of the present disclosure relate to a combination of manual and automatic segmentation techniques that potentially enable the generation of an effective and reliable vascular tree.

[0039] Vascular segmentation is an initial step in the characterization of vascular anatomy and function from medical image data for various applications, including the study of blood flow. However, automatic segmentation methods generally exhibit performance degradation as requirements become more stringent. In addition, contrast agents have limited concentrations available for use, and injectable contrast agents are rapidly diluted, limiting the available imaging time. Furthermore, for safety reasons, the radiation dose used in some types of imaging is preferably minimized for reliable visualization. Even if high contrast and / or high signal-to-noise can often be achieved for large blood vessels, there is also a need in some applications for analyzing smaller blood vessels. However, imaging quality rapidly degrades as the blood vessel diameter decreases because the signal intensity approaches the limits of background noise or quantum noise inherent in the signal itself. Even leaving aside such technical considerations, the vascular system itself has a very complex shape. This potentially causes ambiguous structures in many cases where it is difficult to determine a specific branching structure by inspection at the level of local features, especially in 2D images. On the other hand, the interpretation of such features depends potentially on the particularity of the constraints applicable to a specific structure and / or imaging method, making it difficult to address the detection of broad features with conventional automatic methods.

[0040] For these and other reasons, in actual cases, often the quality of medical images available for analysis results in being close to or exceeding the limits of current technology for machine vision and / or image processing for at least some of the vascular structures of interest. Even if the boundaries of these limits move over time with technological development, it is expected that interesting segmentation problems will continue to exist where automatic vessel segmentation alone is insufficient.

[0041] Semi-automatic segmentation methods seek to address the limitations of pure automatic segmentation methods by augmenting with human judgment and / or control. However, human intervention is expensive in terms of time, money, and / or availability. From this perspective, the goal in some semi-automatic segmentation methods is to reduce the human time and / or effort spent supervising the automatic segmentation. Considering the problems in optimization, the goal in some embodiments of the present disclosure may be to bring the human intervention in semi-automatic segmentation to the lowest achievable level that is consistent with a result of sufficient quality for the applications for which they are applied. Thus, suitable methods of semi-automatic segmentation may include features specific to their application in a particular problem area.

[0042] Aspects of some embodiments of the present disclosure relate to a cascaded method for semi-automatic vessel segmentation of angiographic images. More specifically, in some embodiments, this aspect relates to a cascaded method for semi-automatic segmentation of angiographic images while working within the constraints of generating results in real time. Optionally, the segmentation is completed while any catheterization procedures that may have been used for image generation are in progress and there is sufficient time remaining for subsequent analysis, such as diagnosis and / or treatment planning.

[0043] In some embodiments, the manual management operations of semi-automatic vessel segmentation are configured to cascade through an increasingly attention-demanding set of user operations and stop cascading once the user is satisfied that a result of sufficient quality has been obtained. There may be one or more routes through this cascade of operations. For example, the order of the selected operations may optionally depend on whether a nearly appropriate result has been obtained early on that requires only minor editing, or whether the user needs to determine a new appropriate route. In some embodiments, the cascade may be configured such that more likely options are presented earlier and / or weighted more heavily, potentially reducing the time and / or effort the user expends in making selections. Optionally, the order of operations is selected to emphasize obtaining a "close enough" result with minimal input while also optionally providing an opportunity to correct errors in the automatically identified results as needed.

[0044] In some embodiments, the method optionally begins with the determination of the terminal regions of two vascular paths (e.g., the regions among some maximum distances from a selected point), and then the most likely and automatically detected path is presented to the user for acceptance or rejection. For the appropriate determination of the "most likely" (e.g., an appropriate cost function), this potentially allows that most or multiple user interventions are limited to the default simple acceptance. Optionally, pairs of terminal regions of multiple vascular paths are first determined, corresponding multiple default options are presented simultaneously, and the user interaction is still further limited to default correction. In some embodiments, the determination of one of the endpoints is simplified by determining at least one endpoint at the root position of the vascular tree and is arranged to be considered at one end of any vascular path leading backward from one of its branches. In some embodiments, the determination of the terminal positions of multiple blood vessels is based on a completely automatic detection (e.g., the position where the vascular skeleton determining the vascular centerline naturally terminates), or a semi-automatic method such as the position crossing a segment of the vascular skeletonization where the selection line swept by the user is automatically detected.

[0045] In some embodiments, if the default path is not acceptable, the automatically proposed paths with reduced likelihood (higher cost function scores) are optionally displayed as available. For example, the user can use a scroll wheel or other operating part to quickly indicate alternative paths extending between some pairs of the target vascular path terminal regions.

[0046] If the discovery of an appropriate path fails with the automatically proposed alternatives, in some embodiments, the user is presented with a user interface tool operable to determine a vascular path based on the determination of one or more additional waypoints.

[0047] Optionally or alternatively, one or more editing tools are optionally provided that enable modification of the presented alternatives that are generally acceptable. For example, a path may be cut, extended, and / or joined to a portion of another existing path. In some embodiments, the path is optionally edited along its length, for example, by re-tracing, re-determining anchor points, and / or dragging an erroneous segment region to the correct position.

[0048] In some embodiments of the present disclosure, the goal of semi-automatic vessel segmentation is the production of one or more vessel paths corresponding to anatomically reasonable paths of blood flow. In some embodiments, the vessel path includes numerically stored consecutive positions corresponding to vessel image positions. In some embodiments, the vessel path includes a vessel centerline. Optionally, the vessel path is determined at one end by a root position located, for example, at the end of the path within the least branched vessel that the path crosses. At the other end, the vessel path is optionally determined by a terminal position located, for example, within the most branched vessel that the path crosses.

[0049] In some embodiments, the vessel path is determined by the connection of one or more vessel segments. Optionally, the vessel segments are determined by one or more methods at one or more levels of fidelity. In some embodiments, the vessel segments may be determined within a skeletonized representation of the vessel image (e.g., a binary pixel array representation extending through the detected vessel structure of one pixel width). Such vessel segments optionally comprise a sequence of pixel positions extending between two pixels marking end points. The terminal positions are arbitrarily selected by any convenient, even arbitrary method (e.g., dividing the skeleton into segments of a maximum length of N pixels). However, preferably the end positions of the vessel segments are determined at bifurcations and / or intersections (e.g., skeleton pixels where a branch extends in at least three directions therefrom) and / or at free ends (e.g., skeleton pixels where only one branch extends therefrom).

[0050] In some embodiments, the vascular paths are determined separately from each other. In some embodiments, the vascular paths are determined by their different extents along a branched vascular tree (e.g., branches of the vascular tree, optionally, by a particular path that traverses a branch of the vascular tree determined as a set of linked vascular segments). In contrast, in some embodiments, the vascular tree is determined by the fusion of a set of vascular paths, e.g., paths that share a common vascular segment are interpreted as also sharing a common root segment.

[0051] Aspects of some embodiments of the present disclosure relate to a method of selecting a vascular path based on the order of presentation of options for an automatically generated vascular path.

[0052] In some embodiments of the present disclosure, a plurality of path routes (e.g., vascular segments determined according to criteria of gradient, curve, and / or relative intensity) extending along a detected vascular segment are generated for a pair of terminal regions. Optionally, the generated path routes reach segment points within some regions, optionally, regions determined as being within some maximum distance, e.g., from an endpoint, from one of the endpoints. This distinction is relevant, for example, when one of the endpoints does not exist on the segment.

[0053] In some embodiments, a plurality of path routes are presented as a range of vascular path options in a default order. Optionally, the default order is configured to rank the path routes according to a cost function (optionally a heuristic criterion) that assigns lower cost values to more likely actual blood paths than to less likely actual blood flow paths between two endpoints. Preferably, the path routes are presented along the original image data, such as graphical overlays on an image. In some embodiments, the image data is presented as an animated sequence of images, such as slight movement of blood vessels and / or images from different angles. Potentially, these differences are more useful for the user's visual capabilities than for the automatic detection algorithm, for example by emphasizing the connectivity between parts of the co-moving vascular system.

[0054] In some embodiments, the criteria of the cost function include aspects of vascular continuity, vascular branch anatomy, criteria based on the three-dimensional shape of the organ perfused by the blood vessel, opacity of the blood vessel image, and / or other criteria.

[0055] In some embodiments, the results of the automatic segmentation do not unambiguously distinguish which of the plurality of vascular paths between two vascular positions corresponds to the actual path of blood flow. Particularly in the case of a 2D image of a 3D vascular structure (e.g., a vascular structure generally visualized as extending through two or more depth layers), some branches of the vascular tree may appear to cross each other. Crossings of the same tree are rarely observed in structures generally imaged within a single focal plane, such as the intraretinal vascular system, but ambiguity may arise when it is difficult to distinguish between the structures of veins and arteries.

[0056] Note that dynamic path extending methods, such as Livewire, can partially reduce the ambiguity of automatic segmentation by determining waypoints that can be set to direct the automatic result towards correct segmentation. However, this potentially consumes more time than simply determining the start and end points. In some embodiments, there is a trade-off between the correct but slow determination of the vascular path by using waypoints and the potentially fast but less accurate determination of the vascular path by selecting from a set of automatically pre-determined vascular path options.

[0057] Aspects of some embodiments of the present disclosure relate to a method of editing a vascular path by dragging a mis-segmented region into alignment with a more accurately segmented position.

[0058] In some embodiments, the method includes receiving an indication of a region selected along the segmentation of a blood vessel. In some embodiments, the indication includes the selection of a screen coordinate point, for example, by pressing a button along with a particular position of a screen cursor and / or by a gesture input to a touch screen. In some embodiments, the indication includes directly selecting a mis-segmented region.

[0059] In some embodiments, the method includes assigning a mobility-sensitive property that enables movement of a selected region to a segmentation path region surrounding the selected region, while preferably maintaining the remaining portions of the segmentation in a state that is not easily affected by movement. In some embodiments, the assignment of sensitivity to movement includes determining an energy functional that is determined as a function of the position along the segmentation of the blood vessel. Optionally, a non-zero region of the energy functional is set based on the position of the selected region. In some embodiments, the method includes moving the region of the segmentation in accordance with minimization of energy within the non-zero region of the energy functional. Optionally, the movement of the mobility-sensitive portion of the segmentation path region cooperates with the movement of the cursor. Optionally, the movement of the mobility-sensitive portion is at least partially integrated by the value of the image intensity and / or the gradient in the vicinity of the selected region such that the selected region behaves as if attracted to the movement of the blood vessel region.

[0060] Prior to detailing at least one embodiment of the present disclosure, it should be understood that the exemplary systems, methods, devices, and / or computer program products illustrated are not necessarily limited in their structure details, component arrangements, and / or applications to the methods shown in the following description and / or figures. The exemplary systems, methods, devices, and / or computer program products can be other embodiments and can be executed or implemented in other ways.

[0061] Method for determining a path along a blood vessel tree Reference is now made to FIG. 1, which is a flowchart showing an overview of a method for determining the path of a blood vessel centerline in an image by angiography according to some exemplary embodiments of the present disclosure.

[0062] In some embodiments, an editing mode of a computer program configured for interactive vascular routing via user interaction through a user interface is launched. In some embodiments, at block 102, a root location (e.g., corresponding to root locations 401, 501 in FIGS. 4A - 4F and 5A - 5F) is determined. In some embodiments, the root location is a vascular location visible in an image that is topologically closest to the region where blood enters or exits the heart. In the coronary arteries, for example, the root location is topologically closest to the region where blood exits the left ventricle into the aorta. Optionally, the root location is determined manually, for example, by clicking or touching (via a user interface device) a point within the image and / or by selecting, moving, and / or confirming an automatically detected root location. Optionally, one or more candidates for the root location are automatically detected, for example, based on the timing and / or location of dye appearance in a time series of images immediately after injection and / or based on morphological criteria such as vessel thickness, branch order, and / or direction.

[0063] In some embodiments, at block 110, as described in connection with FIG. 2 for example, the determined root location is used as an input for the determination of route data including alternative route options and the cost of the route options.

[0064] In some embodiments, at block 112, a vascular route representing the continuous vascular connection between several points in the vascular tree image and the root location is optionally determined (and / or selected, for example, based on a list of alternative route options and the route option costs determined at block 110). A detailed example of the implementation of block 112 is described in connection with FIG. 3A.

[0065] In block 114, the vascular path representing the image of the vascular tree and the path determined prior to continuous vascular connection between several points at the root position is optionally edited. A detailed example of the implementation of block 114 is described in connection with FIG. 3B.

[0066] In some embodiments, in block 104, a determination is made as to whether to continue in the editing mode. If not, the flow exits the flowchart. Otherwise, the flow returns before block 112 (optionally, before block 102 so that the root position can be re-determined).

[0067] Vascular Path and Cost Function Now, refer to FIG. 2, which is a schematic flowchart of a method for generating vascular path options from vascular images according to some exemplary embodiments of the present disclosure.

[0068] In some embodiments, the flowchart begins at block 208, where the vascular skeleton graph 206 branches into branches. In some embodiments, the vascular skeleton graph is obtained from the image processing of angiography images. The images 400 of FIGS. 4A - 4F and the images 500 of FIGS. 5A - 5F are examples of source images from which the vascular skeleton graph is obtained. In some embodiments, the vascular skeleton graph includes a vascular centerline. An example of the extraction of the vascular centerline (using anisotropic diffusion, Frangi filtering, and hysteresis thresholding in some embodiments, followed by binary thinning) is described, for example, in connection with block 20 of FIG. 14 of International Publication No. WO 2004 / 11927 to the applicant filed on Jan. 15, 2014, the content of which is hereby incorporated by reference in its entirety. In some embodiments, the basis of the method used is similar to that introduced by Weickert in "A Scheme for Coherence-Enhancing Diffusion Filtering with Optimized Rotation Invariance" and / or "Anisotropic Diffusion in Image Processing" (Thesis 1996).

[0069] In some embodiments, at block 208, the vascular skeleton graph 206 is branched. Optionally, this includes identifying pixels where three or more skeleton segments converge (e.g., pixels having three or more adjacent pixels, optionally excluding short branches from consideration), and assigning path breaks at these points. Optionally, attention is also paid to where the branches connect (the connections are simply obvious such that the branches may cross each other and / or approach close enough to appear to join). The result of the operation of block 208 is the branch list 210. It should be noted that the "branches" in the branch list 210 may potentially be linked in loops due to the actual underlying anatomy (e.g., the development of shunt vessels), and / or apparent connections arising from nearby vessels, and / or intersections at the provided angles, and / or the resolution of the image.

[0070] In some embodiments, the branches stored in the branch list are only those for which there exists a continuous connection to the root location of the vascular tree (e.g., root locations 401, 501 in FIGS. 4A - 4F and 5A - 5F). Optionally, the root location is determined as described, for example, in relation to block 102 of FIG. 1.

[0071] In some embodiments, at block 212, paths are calculated that extend from each branch to the determined root location, for example, by using a search algorithm. Optionally, other two or more such paths are available, for example, due to intersections, close approaches, or other ambiguities present in the image as described above. Paths that contain the same vascular segment more than once are optionally excluded from this list (to avoid looping and / or paths that double back on themselves). The data structure that is the result of the operation of block 212 includes the path list 214. In some embodiments, the path list 214 includes each identified non - excluded path that extends between the root location and each vascular skeleton segment.

[0072] In some embodiments, at block 216, a path cost is calculated for each path in path list 214 to generate a path cost list 218. The path cost list 218 is configured, in some embodiments, such that members of potentially multiple paths in path list 214 that reach a route from any vascular segment can be ranked in order of likelihood of being the actual path of blood flow. The use of path list 214 and / or path cost list 218 is further described in connection with the operation of block 112 of FIG. 1, as detailed in connection with, for example, FIG. 3A.

[0073] According to the convention adopted herein, a higher cost path is optionally considered less likely to be a candidate for the actual path of blood flow in vascular anatomy. The path cost criteria are optionally evaluated as they receive binary values (e.g., 0 or 1), ranked values, and / or scores on a continuous scale. The path cost criteria are optionally added to, multiplied by, or otherwise incorporated into the overall cost function.

[0074] In some embodiments, the results of machine learning are used to assign a reference cost value and / or contribution to the overall cost relative to other criteria. In some embodiments, machine learning techniques include one or more implementations of decision tree learning, association rule learning, artificial neural network, inductive logic programming, support vector machine, cluster analysis, Bayesian networks, reinforcement learning, representation learning, similarity and metric learning, and / or opportunity learning. Machine learning is optionally based on a set of angiography images in which the morphology of the vascular tree is separately marked and / or on the results of its own learning.

[0075] Cost criteria In some embodiments, the path costs recorded in the path cost list 218 are calculated against one or more of the following determinations and / or criteria

[0076] Node type classification As used herein, a connection point of a vascular skeleton segment is referred to as a node. A node may be determined as one pixel and optionally as a region of relatively large size (e.g., in the range of 2, 3, 4, 5 or more pixels) within which three or more vascular skeleton segments meet. In some embodiments, the overall cost function applicable to a node is at least partially by node type classification.

[0077] If two blood vessel segments appear in the image (e.g., due to being in different planes of an image of the heart), the blood vessel skeleton shows four branches appropriately generated from the region of the central node. The corresponding blood vessel skeleton is optionally analyzed to determine the intersection type node. In some embodiments, when a blood vessel branches, it is common to bifurcate. In such cases, the node is optionally determined as a junction node where three branches of the blood vessel skeleton occur. The criteria of the cost functions available for these two basic node types are optionally assigned differently.

[0078] In some embodiments, there are other junction orders that may appear in the path list. For example, a bifurcation may occur at the same position as the intersection of blood vessels. Another potentially occurring situation is that two junction nodes may appear very close to each other and be analyzed as a four-branched node. In contrast, the skeletonization artifact results in a potentially slight deviation for the same blood vessel on either side of the intersection, making it look more like two adjacent junction-type nodes. A terminal blood vessel segment (i.e., the blood vessel segment that is the last segment visible in the image in that branch) may have a free end and be located at or near the position of other blood vessel segments, in which case three-branched and four-branched nodes may also occur. Optionally, nodes containing five, six or more segments are analyzed as being composed of an appropriate combination of junction nodes, intersection nodes and / or free-end terminations.

[0079] In some embodiments, morphological criteria other than the number of branches are used in node classification for the purpose of cost function assignment. For example, several nearby sequential branches (bifurcations) simply include a pair of segments in a single continuous direction, while at the same time two other segments extend at a relatively acute angle (e.g., about 90°) with respect to each other. In other examples, the branch angle generally (but not exclusively) exhibits an acute angle in the forward direction (the direction of arterial flow). A backward branch angle is more likely to imply an intersection node. In still other examples, the region of vessel intersection may appear as a region of increased radiopacity compared to the region where the vessels branch (due to the cumulative contribution of the thickness of the two overlapping vessels). Conversely, a sudden change in the apparent radiopacity at either end of a node due to a change in the direction of the vessel with respect to an axis perpendicular to the image plane (and a corresponding change in the length of the absorption path) potentially indicates a bifurcating type of node.

[0080] Optionally, for example, for the classification of more complex nodes, r p 3 =r d1 3 +r d2 3 r d3 3 +···+r dn 3 wherein r p 3 is the radius of the parent branch (trunk), and r d1 3 r d2 3 r d3 3 ···r dn 3 are the radii of the respective child branches, applying Murray's principle. Optionally, combinations of vessel segments that satisfy Murray's principle at several nodes are assigned a higher likelihood of indicating a bifurcating type of node, while the remaining segments are assigned a higher likelihood of being free end terminals or participants in the structure of an intersection node.

[0081] In some embodiments, the appropriate cost function for a node depends on which analysis of the node type is correct. In some embodiments, the determination of the node type is a matter of probabilistic assignment, for example, a node is assigned an 80% likelihood of two tubes intersecting, a 15% likelihood of two branches (or a branch into three), and a 5% likelihood of a branch in the region of the free end of a terminal vascular segment.

[0082] In some embodiments, there is no explicit classification in the cost function between junction nodes and intersection nodes (or free ends, or any combination of these types), rather, a heuristic is employed that acts on such basic relative vascular morphologies. This approach may be suitable for certain types of implementations based on machine learning, such as a population neural network. Nevertheless, node correspondence provides a convenient organizational concept for the purpose of explaining further vascular pathway cost function criteria.

[0083] Continuity and / or consistency at an intersection node In some embodiments of the present disclosure, the continuity of the vascular morphology at a node is used as the basis for the assignment of cost function values.

[0084] The convergence of four vascular skeleton branches potentially interpreted as an intersection node is arbitrarily analyzed as determining three empirical options for continuity (exit side) from several approaching fourth directions (inlet side). In some embodiments, one or more criteria related to the continuity of the morphology are applied to determine which exit side branch is the most likely continuous branch from a given inlet side.

[0085] One such criterion is the vessel radius, where the vessel branch on the outlet side with the radius most similar to that on the inlet side correspondingly receives the lowest cost assignment. Optionally, the vessel radius is measured as half of the apparent width of the filled lumen in the image. Optionally, the continuity of the image intensity values (which are partly a function of the radius but not only that), for example by direct comparison of intensity values, is also taken into account. Optionally, the principle of density measurement is applied, for example, the vessel is modeled as a cylinder of a specific radius filled with a substance having a specific density and / or absorption coefficient, and the cost function is jointly based on the continuity of the radius and absorption characteristics to satisfy the intensity values observed in the image.

[0086] In some embodiments, the continuity of direction is the criterion, and the vessel segment on the outlet side most similar in direction to the segment on the inlet side optionally receives the lowest cost assignment as its continuation. Note that based on the vessel directionality in the direction perpendicular to the image plane, there may also be a difference in density measurement, for example, the vessel may appear darker when viewed closer to the end. Thus, the continuity of the vessel intensity values optionally serves as an alternative to the continuity of the vessel direction in this dimension.

[0087] In some embodiments, these criteria include one or more further refinements. For example, the direction criterion applied to the junction optionally includes a cost assignment based on the continuity ratio and / or the direction of the angular change before and after the node (for example, the continuity between two segments with more similar radii of curvature receives a lower cost assignment). This potentially has a particular relevance for vessel segments that loop around the bulge of the 3D heart surface through a part of the vessel image. In some embodiments, the criteria for the cost function prefer a path along which the vessel width automatically changes (for example, increases in the route direction).

[0088] In some embodiments, the paths from segments A, B, C, D meeting at a junction (particularly an intersecting junction), are treated as "entangled" such that a low cost for the corresponding path extending from A→B, reduces the cost of path C→D as compared to the alternative path C→B. Optionally, a user's definitive indication that path A→B is a preferred path of blood vessel flow during some stages of the vascular tree diagram, similarly entangles in reducing the cost assignment of path C→D as compared to path C→B.

[0089] Continuity and / or coherence at a bifurcating node In some embodiments, when a blood vessel bifurcates, it is typically a bifurcation. Optionally, a greater number of bifurcations, such as a trifurcation, is treated as a configuration of adjacent bifurcations. At a bifurcation, a junction node is optionally determined that has two empirical options for the two directions to continue (exit side) from any approaching third direction (inlet side). In some embodiments, a heuristic is employed such that the exit side branch seeking minimal change in direction through the junction node, receives a lower cost assignment.

[0090] Optionally, cost scoring based on blood vessel radius similarity, reduces its own influence on the cost function (e.g., by reducing the weight), to account for the potential size proximity of two thinner branches to each other, as compared to the main portion they share, when a three-way junction node is detected. Additionally or alternatively, the radius-based cost function is assigned differently for a bifurcating node, e.g., the thickest blood vessel segment (generally located in the direction of the root position), is given the lowest cost score for both of the two thinner branches. Optionally, the cost scoring assignment is further based particularly on Murray's principle, such that a path that better satisfies this theoretical relationship, is characterized as being given a corresponding lower cost score.

[0091] Other criteria In some embodiments, paths that use vascular segments running along relatively straight and / or continuous arcing lines are arbitrarily scored at a lower cost. This is a potential advantage for favorable image features of being "more vascular" in terms of features, as opposed to image features that may terminate in the vascular skeleton but are not actually the original blood vessels. The cost is arbitrarily calculated, for example, based on the overall area under (above or below zero) the curve of the angle derived from the direction of local vascular segments.

[0092] In some embodiments, the position of blood vessels in three dimensions is estimated. Optionally, the use of this information is part of node classification and / or is used for cost scoring of continuity / coherence. In some embodiments, it is known a priori that the vascular tree extends across a three-dimensionally curved surface, such as the surface defined by the shape of the myocardium. In some embodiments, the parameters of this curved surface are derived from the image shadow that depicts the boundary line of the following shape. In some embodiments, the shape and / or intensity of the tube itself distinguishes the boundary line of the shape. For example, as blood vessels curve more inward / outward from the image plane, they exhibit a longer absorption cross-sectional area and thus may potentially appear more radiopaque at some boundary limits. Tubes that curve around in three dimensions may first approach and then appear to curve away from some boundary limits in the 2D image. In some embodiments, the shell is modeled as having two parts that generally occupy different depths but are joined to each other at such boundary limits. Optionally, the vascular segment is scored for possible positions on the upper or lower part of the shell while treating the boundary limit as a transition zone. Paths that require a sharp transition between two different shell parts (partially away from the boundary limit) are arbitrarily given a higher cost value.

[0093] Note that, in some embodiments, it may be assumed that vessels of the same type (artery or vein) occupying the same anatomical plane are relatively unlikely to cross each other. Optionally, if an intersection node is identified with a high likelihood (additionally or alternatively, suggesting a vessel crossing with a relatively low coherence and continuity associated path cost), this may be used to increase the assumed likelihood that two different blood vessels occupy two separate shell portions. And other directly contacting vessel segments may further be in the same shell portion and are optionally used information to eliminate the ambiguity in the assignment of path costs across adjacent nodes.

[0094] Selection and / or determination of a blood vessel path Now refer to FIG. 3A, which is a schematic flowchart of a method for selecting and / or determining options for a particular blood vessel path according to some exemplary embodiments of the present disclosure. In some embodiments, the operations of FIG. 3A correspond to the operations occurring within block 112 of FIG. 1. Also refer to FIGS. 4A-4F, which schematically illustrate the selection of blood vessel path options according to some exemplary embodiments of the present disclosure. FIG. 4A shows an angiographic image 400 showing a portion of a blood vessel tree 402 (in the example, the blood vessels of the heart) including a currently determined root location 401. In FIGS. 4B-4F, the contrast of the angiographic image 400 is reduced to make other elements more visible.

[0095] In some embodiments of the present disclosure, one or more manually guided methods are provided for viewing, selecting, and / or determining vascular tree topology. In this context, an "accepted route list" 324 is introduced. The accepted route list 324 optionally includes routes selected from the route list 214, routes edited based on the routes in the route list 214, and / or newly generated routes, for example, based on user input. In some embodiments, the route cost list 214 is used, as needed, in determining the display order of the default selection from the route list 214 and / or additional selections from the route list 214.

[0096] Here, the generation of a data structure including the route list 214 is described in relation to block 212 of FIG. 2. In some embodiments, the route list 214 potentially includes many vascular routes that are "true" vascular routes in the sense that they are isomorphic to the vascular routes through which blood flows in the anatomical structure of an actual patient as they flow between the root location 401 and the distal end of the vascular route. However, there may be many "false" vascular routes within the same route list 214, for example, reflecting ambiguities introduced by vascular intersection points and / or limitations in the quality of the vascular image. Although the route cost list 218 optionally provides a basis for some routes to be preferred over others, the situation may lead to the true vascular route being less preferred. In some embodiments, due to contrast dropout and / or other artifacts that can affect vascular skeletonization, the true vascular route may not even be available in the route list 214.

[0097] In some embodiments, at block 310, the end of the path is indicated (e.g., by the user). In some embodiments, the indication of the end of the path includes a "hover" cursor input event (e.g., moving the cursor 405 near a potential blood vessel path and pausing for a sufficient time for detection, such as a long touch on a touch screen). In some embodiments, other input events, such as a screen tap, indicate the end of the blood vessel. In some embodiments, the user instruction refers to an area (e.g., a pixel) in the nearest available segment shown in the path list 214. Optionally, the user instruction refers to the area of the segment closest to the indicated position. For example, the blood vessel path 407 in FIG. 4B extends towards the end 407A of the path near the cursor 405 at a certain position. When the cursor 405 is moved as shown in FIG. 4C, a different blood vessel path including the blood vessel path 408 added to the blood vessel path 407 is shown instead, and again, it ends at the path end 408A near the cursor 405. Optionally, the user's instruction refers to the end of the segment furthest from the root position 401.

[0098] For the purpose of explanation, the conversion of the shown path ends to paths in the accepted path list 324 is mainly presented from the perspective of a manual operation on one path at a time. However, it should be understood that these explanations can be varied as needed and applied for the processing of multiple and / or automatically selected paths.

[0099] For example, in some embodiments, one or more candidate path ends are automatically detected (e.g., the free ends of the blood vessel skeleton segments that are continuously connected to the root position 401). Additionally or alternatively, multiple path ends are optionally indicated by the user as part of one input action, e.g., dragging out a path that crosses between some candidate path ends (by cursor movement, touch input, or other means), and the indicated ends are received as the positions where the dragged-out path intersects the segments of the blood vessel skeleton.

[0100] If multiple candidate path ends are available, the user is optionally presented with options for stepping through the candidate path ends (e.g., by successive key presses, dragging with a finger on a touch input screen, movement of a scroll wheel, or other input methods), and the selected end becomes the indicated path end. Optionally, an initial path is determined for each of the multiple candidate path ends (and is optionally accepted as a member of the path list accepted by default).

[0101] Upon a first entry to block 311, in some embodiments, the lowest cost available path from the path list 214 among those reaching the indicated path end (based on the path cost list 218) is first selected for presentation to the user. Optionally, the presentation is highlighted and distinguished, e.g., with special coloring and / or thickness, as being active with respect to the current decision. In some embodiments, the path from the path list 214 is optionally the basis of what is shown, but the actual path shown is derived from the input in the path list 214, e.g., by application of an active contour or dynamic "snake" algorithm as described herein, and it should be noted that the points are derived from the input in the path list 214.

[0102] In some embodiments, at block 312, the user determines whether the correct path (suitable for use as the “true” vascular path) is being displayed. If so, the flowchart optionally proceeds at block 316 when the user issues a confirmation input (e.g., a double click, a screen tap, a key press, or other input). Then, the currently selected path is added to the accepted path list 324, and the flowchart exits from block 112 of FIG. 1. Optionally, the accepted path continues to be displayed and is optionally shown as deselected, e.g., by being drawn thinner with lower contrast or by other visual indication. FIG. 4D shows a sequence of paths 407 and 408 depicted as members of the accepted path list 324. Due to the new position of the cursor 405, a corresponding new path option 409 is shown. This sequence continues in FIGS. 4E-4F where new paths 411 and 414 are generated. As shown, each new path is drawn to extend to the root position 401. Alternatively, the display is extended only backward to the first contact with a previously accepted vascular path. Optionally (and independently of the optional display method), the path is stored in the accepted path list 324 as a complete path extending between the end and the root position, as an incremental addition to the vascular tree structure, and / or in other formats.

[0103] Alternatively, if the correct path is not displayed in block 312, the flowchart proceeds, optionally, to an option selection event at block 314. In some embodiments, the option selection event includes movement of a scroll wheel, a touch screen gesture (e.g., a slide gesture), a key press, or other user input event. Optionally, this selection is interpreted in software as indicating that the current path display should move to another candidate (e.g., the next candidate available in the order of costs defined in the path cost list 218). In a variation of the method, multiple vascular path candidates are presented simultaneously for a pair of endpoints, and the selection is used to indicate which candidate should be depicted as being actively selected. A potential advantage of this variation is that it can more quickly materialize that there are no suitable candidates available and / or that the available options can be compared simultaneously.

[0104] In block 316, in some embodiments, the user may determine that a predetermined path in the path list 214 is not appropriate for the current path endpoint being shown and proceed to block 320 to manually determine a new path. Alternatively, in some embodiments, the flowchart returns to block 311, which shows the newly selected path candidate.

[0105] Regarding the iteration through blocks 311, 312, and 314, refer to FIGS. 5A - 5C, which schematically illustrate a selection from among alternative vascular path options according to some exemplary embodiments of the present disclosure. FIG. 5A shows an angiographic image 500 depicting a portion of a vascular tree 502 (in the example, the cardiovascular system) including a currently determined root location 501. In FIGS. 5B - 5F, the contrast of the angiographic image 500 is reduced to make other elements more visible. In FIG. 5B, some pre - determined paths 507 are shown. The cursor 505 is shown directly hovering over a vascular location, but the suggested path 509 ends somewhat away from the cursor over another blood vessel instead. In FIG. 5C, the user indicates that the next candidate vascular path 511 should be shown, but this vascular path may also suggest what the user is trying to add to the accepted path list.

[0106] When it is discovered that the intended path is not available (for each block 316), the user can optionally proceed to block 320. Regarding this block, refer to FIGS. 5D - 5E, which schematically illustrate a method for manually determining vascular path options according to some exemplary embodiments of the present disclosure. FIG. 5 shows a selection made (formed) by consecutive position indications (e.g., clicks made at different positions of the cursor 505) with line segments drawn between each indication. The user generally makes position indications along the intended vascular path. Optionally, the root location 501 is interpreted as part of the position indication.

[0107] In FIG. 5E, the user ends the consecutive position indications (e.g., using a double - click). In some embodiments, this results in the operation of block 322, where the position indications, as described herein in connection with an active - contour algorithm, are used to generate the determination of a new path in the form of a fitting between them. The resulting vascular path is optionally added to the accepted path list, and the flowchart of FIG. 3A exits from block 112.

[0108] In some embodiments, a variation of the Livewire technique is used, where the segment position is arbitrarily live-drawn ( "grows" to satisfy the cursor position from the previous anchor point) between clicks to determine waypoints, rather than all waypoints being determined first and then all at once, as in other methods of manually determining a vascular path.

[0109] Dynamic contour method for vascular path In some embodiments of the present disclosure, a dynamic contour method, also commonly referred to as the "snake" dynamics method (Kaas et al. 1987), is implemented. In this method, the behavior of the contour c(s)=(x(s),y(s)) is governed by simulated internal and external forces applied on the contour. These forces are derived by minimizing an energy functional.

Equation

[0110] The internal energy term is as follows.

Equation

[0111] The external energy term E ext (c(s)) is implicitly defined by the force field.

Equation

[0112] In some embodiments, a GGVF field (Xu and Prince, 1998) on a Frangi filter grayscale image (Frangi et al., 1998) is calculated as the force field, although generally any force field can be used.

[0113] Dynamic contour method for initial path determination When a tube is first determined from the blood vessel path list 214, the initial input may be made from the run of pixels in the blood vessel skeleton segment that connects along the selected path. Optionally, the coordinates actually used within the path are rearranged so as to be at uniform intervals along the blood vessel path length (the distance between the centers of the actual pixels themselves can be non-uniform due to alternating movement in the diagonal and base point directions). And as the active contour method is iterated, the blood vessel path position is drawn into a new position according to the various force and field terms used. Once the path becomes firmly in a sufficiently stable configuration, the force terms are set to zero and the process stops.

[0114] Optionally, for example, as described in relation to FIGS. 5D-5E, when the blood vessel is determined by sparsely provided anchor points, the forces acting on the anchor points are set to zero (for example, α(s), β(s) and γ(s) are set to zero) so that these points continue to be fixed. The points interpolated therebetween (which may initially be provided as a straight line by spline interpolation or other methods) are subject to non-zero forces (for example, α(s), β(s) and γ(s) are set to non-zero values) until they reach a sufficiently stable configuration through a number of iterations that numerically minimize the energy functionals of Equations 2 and 3.

[0115] Active Contour for Path Editing Now refer to FIG. 3B, which is a schematic flowchart of a method for manually editing a blood vessel path according to some exemplary embodiments of the present disclosure. FIG. 3B shows an embodiment of the path editing operation of block 114 in FIG. 1. Also refer to FIGS. 5E-5F, which schematically illustrate the manual editing of blood vessel path options according to some exemplary embodiments of the present disclosure.

[0116] In block 340, in some embodiments, the user makes a position selection along the length s of the displayed vascular path (in some embodiments, the path from the received path list 324). In some embodiments, the position selection includes pressing a button at some selection positions of the cursor, a gesture on the touch screen, or other input methods. In subsequent operations, the position selection is made to determine a drag point, such as drag point 519A in FIG. 5E.

[0117] In block 342, in some embodiments, the energy terms of α(s), β(s) and / or γ(s) are set to be non-zero only for s within a limited surrounding range of the drag point. Optionally, the external force field of Equation 3 is determined to include an attractor at the current cursor position, or at another position determined by further user interface operations as the user drags the drag point to a new position.

[0118] In block 344, in some embodiments, the energy functionals of Equation 2 and Equation 3 are numerically minimized, and as a result, the displayed vascular path (e.g., path 517 in FIGS. 5E-5F) may be moved around the position of the drag point.

[0119] In block 346, in some embodiments, the display of the vascular path 517 is updated.

[0120] In block 348, the system determines, based on, for example, a user input (such as releasing a button or ending a touch screen gesture), whether the drag operation should be ended. If not, in block 350, in some embodiments, the user arbitrarily drags the drag point to a new position, such as position 519B in FIG. 5F. Alternatively, the edited path 517 is updated within the accepted path list 324 and the flowchart ends (optionally, exiting block 114 in FIG. 1). It should be understood that in some embodiments, the accepted path list is dynamically updated additionally or alternatively during the drag operation itself.

[0121] Determination of Vascular Paths from Multiple Displays Referring now to FIG. 6, which schematically illustrates a display 620 of a path determination made simultaneously with a plurality of different vascular image views 400, 500, 600, 610, in accordance with some exemplary embodiments of the present disclosure.

[0122] In some embodiments, the vascular image displays 400, 500, 600, 610 each show a display from a different angle of the same vascular system (e.g., the cardiovascular region). Partially determined groups of vascular paths 620, 622, and 624 are shown for the displays 400, 500, and 600. In some embodiments, a match between features of the vascular images (the matching image features optionally indicate the same anatomical location from different viewpoints) is established in at least a portion of the structure specified in one or more of the vascular skeleton graph 216, branch list 210, path list 214, and / or the received path list 324. In some embodiments, the received paths enable the distinction of matching vascular features without optionally detailed 3D information. For example, based on branch numbers and / or positions, root positions 611, 601, 501, 401, and / or relative vascular lengths. Optionally, basic 3D information (e.g., display angles specified within about 45°, 90°, or other larger or smaller amounts) is useful for identification by correctly setting the general relative direction and / or identifying mirror images. In some embodiments, more complete 3D information can be utilized, for example, based on the position of the vascular centerline reconstructed by stereoprojection into a general 3D frame of a reference example (e.g., as described in International Publication WO2014 / 111930 to the applicant filed on January 15, 2014, the entire content of which is incorporated herein by reference) from two or more images. Optionally, the vascular paths generated for one image display are transformed based on the reconstructed reference example's 3D frame so that they can be overlaid on one or more alternative image displays. In some embodiments, such transformation is performed for the vascular skeleton graph 216, branch list 210, and / or path list 214, for example, to assist in validating the image processing results. For example, missing segment portions in the path and / or skeleton data from one image can be filled from available data in the display obtained from another image.Conversely, the apparent segment portions in one image display are arbitrarily identified, for example, based on their absence and / or lack of connection to the root position in other image displays.

[0123] Refer to FIG. 7, which is a schematic diagram of a data structure 700 and software modules implemented in a system 702 for semi - automatic segmentation of a vascular path according to some exemplary embodiments of the present disclosure.

[0124] In some embodiments, blocks 206, 210, 214, and 218 include data structures generated and / or used by software modules of system 702. These correspond, for example, to the blocks labeled corresponding to FIG. 2. The vascular image 701 is optionally provided from an imaging device 750 that is part of system 702. Optionally, system 702 is stand - alone or remote from the imaging device 750, and the image is provided, for example, via a remote network connection.

[0125] In some embodiments, block 703 includes a vascular skeletonizer configured to generate a skeletonized representation (skeleton graph) of a vascular segment, as described, for example, in connection with block 206 of FIG. 2.

[0126] In some embodiments, block 705 includes a path option manager. In some embodiments, the path option manager includes software functions for implementing the operations of blocks 208, 212, and / or 216 of FIG. 2.

[0127] In some embodiments, editing module 710 includes sub-modules that process the operations described in connection with, for example, FIGS. 1, 3A, and / or 3B. Optionally, optional display module 712 and / or manual routing module 716 implement block 112. Optionally, route editing module 714 implements block 114.

[0128] In some embodiments, system 702 additionally includes display 760 and / or one or more input devices 770.

[0129] As used herein in reference to a quantity or numerical value, the term "about" means "within +10%".

[0130] The terms "comprises", "comprising", "includes", "including", "having" and their conjunctions mean "including but not limited to".

[0131] The term "consisting of" means "including and limited to".

[0132] The term "consisting essentially of" means that a composition, method or structure may include additional ingredients, steps and / or parts, but only if the additional ingredients, steps and / or parts do not materially change the basic and novel characteristics of the composition, method or structure according to the claim.

[0133] Unless the context clearly dictates otherwise, as used herein, the singular forms "a", "an" and "the" include plural references. For example, the term "a compound" or "at least one compound" may include a plurality of compounds including mixtures thereof.

[0134] The words "for example" and "exemplary" are used herein to mean "serving as an example, instance, or illustration." Any embodiment described as "exemplary" or "for example" should not be construed as preferred or advantageous over other embodiments and / or as excluding other embodiments from incorporating their features.

[0135] The word "optionally" is used herein to mean "provided in some embodiments but not in others." Any particular embodiment of the present disclosure may include a plurality of "optional" features as long as such features are not inconsistent.

[0136] As used herein, the term "method" refers to, but is not limited to, methods, means, techniques, and procedures for accomplishing a given task, including those known to, or readily developed from, methods, means, techniques, and procedures known to experts in the fields of chemistry, pharmacology, physiology, biochemistry, and medicine.

[0137] As used herein, the term "treating" includes abrogating, substantially inhibiting, slowing, or reversing the progression of a disease condition, ameliorating clinical or aesthetic symptoms of a disease condition, or substantially preventing the appearance of clinical or aesthetic symptoms of a disease condition.

[0138] Through this application, embodiments of the present disclosure may be presented with reference to a range format. It should be understood that the description in the range format is for convenience and brevity only and should not be construed as a rigid limitation on the scope of the present disclosure. Thus, a description of a range should be understood to specifically disclose not only the individual numerical values within the range but also all possible sub-ranges. For example, a range description such as "from 1 to 6" should be understood to specifically disclose sub-ranges such as "from 1 to 3", "from 1 to 4", "from 1 to 5", "from 2 to 4", "from 2 to 6", "from 3 to 6", etc., and the individual numerical values included in the range, such as 1, 2, 3, 4, 5, and 6, as well. This applies regardless of the width of the range.

[0139] When a numerical range is indicated in this specification (e.g., "10 - 15", "from 10 to 15", or any pair of numbers linked by such other range notations), unless the context clearly indicates otherwise, it is meant to include any number (fractional or integer) within the limits of the indicated range, including the limits of the range. The phrases "range / ranging between / range between" of the first indicated number and the second indicated number, and the phrases "range / ranging from / to / until / between" (or other such terms indicating a range) of the first indicated number to the second number are used interchangeably herein and are meant to include the first and second numbers and all fractions and integers therebetween.

[0140] Although example systems, methods, apparatuses, and / or computer program products have been described in connection with specific embodiments, it is apparent to those skilled in the art that many alternatives, modifications, and variations are possible. Accordingly, it is intended to embrace all such alternatives, modifications, and variations that fall within the spirit and scope of the appended claims.

[0141] All publications, patents, and patent applications mentioned in this specification are hereby incorporated by reference in their entirety as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated herein by reference. In addition, the citation or identification of any document in this specification should not be construed as an admission that such citation is available as prior art against this disclosure. To the extent that section headings are used, they should not necessarily be construed as limiting.

[0142] In the context of individual embodiments, it is recognized that some features of the exemplary systems, methods, devices, and / or computer program products described for clarity may be provided in combination in a single embodiment. Conversely, various features of the systems, methods, devices, and / or computer program products described for brevity in the context of a single embodiment may be provided separately, or in any suitable sub-combination, or in any other described embodiment of the present disclosure as being suitable. Specific features described in the context of various embodiments should not be considered essential features of those embodiments unless the embodiments cannot operate without those elements.

[0143] [Appendix] [Appendix 1] A method for segmenting a vascular image into a vascular path for determining a blood flow path, the method comprising: receiving, within a processor, the vascular image; determining, via the processor, terminal regions of first and second target vascular paths within the vascular image; segmenting, via the processor, the vascular image to identify positions of portions of blood vessels within the vascular image; ​Automatically generate a plurality of vascular path options that, via the processor, determine potential vascular paths extending between the end regions of the first and second target vascular paths from the identified portion of the blood vessel. Via the processor, display, for user selection, the plurality of vascular path options registered in the vascular image, each of which includes the end regions of the first and second target vascular paths, and Receive, within the processor, the path option selected by the user for determining the path of blood flow. A method comprising the above.

[0144] [Appendix 2] A plurality of the paths are automatically generated based on a first set of criteria. The path option selected by the user is selected based on a second set of criteria. The method according to Appendix 1, characterized in that the above is the case.

[0145] [Appendix 3] Further comprising pre-determining the order of selection for the vascular path options. The pre-determining comprises ranking the vascular path options in order based on an evaluation of the likelihood that each of the vascular path options corresponds to the actual path of blood flow within the imaged blood vessels in the vascular image. The method according to Appendix 1, characterized in that the above is the case.

[0146] [Appendix 4] The pre-determining includes applying a cost function that assigns a numerical cost to one or more characteristics associated with the vascular path options. The method according to Appendix 3, characterized in that the above is the case.

[0147] [Appendix 5] The cost function assigns a numerical cost based on the characteristics of the centerlines of a plurality of vascular segments that are centerlines and to which the vascular path options are connected from the centerlines. The method according to appended note 4, characterized in that

[0148] [Appended note 6] The features of the centerlines of the plurality of vascular segments include one or more of the group consisting of the direction of the centerline, the centerline offset, and the number of centerlines extending from the node region. The method according to appended note 5, characterized in that

[0149] [Appended note 7] The cost function assigns a numerical cost based on the features of the vascular image on which the vascular path option extends. The method according to appended note 3, characterized in that

[0150] [Appended note 8] The features of the vascular image include one or more of the group consisting of the continuity of the vascular segment image intensity, the continuity of the width of the vascular segment image, and the position of the relative change in vascular intensity with respect to the node region, where three or more vascular segments extend from the node region. The method according to appended note 7, characterized in that

[0151] [Appended note 9] The pre-determining includes applying a cost function that assigns a numerical cost based on the estimated relative position of the vascular segment image in depth with respect to an axis extending perpendicular to the plane of the vascular image. The method according to appended note 3, characterized in that

[0152] [Appended note 10] The displaying includes presenting a plurality of the vascular path options in a consecutive order determined by the selection order. The method according to appended note 3, characterized in that

[0153] [Appended note 11] The displaying includes simultaneously showing a plurality of the vascular path options, and the order of selection corresponds to the order in which the vascular path options are displayed as active for selection. The method according to appended note 3, characterized in that...

[0154] [Appended note 12] Each vascular path option determines a vascular path that terminates in the vascular region of the image closest to one of the terminal positions of the first and second target vascular paths and extends through the image region between the terminal regions of the first and second target vascular paths. The method according to appended note 1, characterized in that...

[0155] [Appended note 13] A method for editing a vascular path to more accurately depict the segmentation of blood vessels in a vascular image, the method comprising: receiving, in a processor, an indication of a selected region along the segmentation of the blood vessel; determining, via the processor, an energy functional determined as a function of a position along the segmentation of the blood vessel, the non-zero region of the energy functional being set based on the position of the selected region; and moving, via the processor, the region of the segmentation according to the minimization of energy within the range of the non-zero region of the energy functional. The method as described above. The energy functional values in the non-zero region are set based on the characteristics of the basic vascular image.

[0156] [Appended note 14] The method according to appended note 13, characterized in that... The energy functional values in the non-zero region are set based on the movement of an indication of a position controlled by the user.

[0157] [Appended note 15] The method according to appended note 13, characterized in that... The indication of the position controlled by the user includes the indication of the selected region.

[0158] [Appended note 16] The indication of the position controlled by the user includes the indication of the selected region. The method according to appended claim 15, characterized in that...

[0159] [Appended claim 17] A user interface for semi - automatic segmentation of a vascular path, wherein the user interface comprises at least one interface module operable to present a default vascular path that extends between two target end points and is automatically generated; and at least one interface module operable to present at least one additional automatically generated vascular path that extends between the two target end points; characterized by comprising the above.

[0160] [Appended claim 18] further comprising at least one interface module configured to be able to determine at least one way point and operable to present an automatically generated vascular path that extends between the two target end points via the at least one way point; The user interface according to appended claim 17, characterized in that...

[0161] [Appended claim 19] further comprising at least one interface module operable to modify a pre - determined vascular path by dragging the position of the vascular path to a new location; The user interface according to appended claim 17 or 18, characterized in that...

Claims

1. A user interface for semi-automatic segmentation of vascular pathways, comprising: at least one interface module operable to present an automatically generated default vascular path extending between two target endpoints of the vascular image; at least one interface module operable to accept user input reflecting one or more position indications following the two target endpoints, where an updated vascular path is generated based on fitting the default vascular path through the one or more position indications and based on a geometric shape corresponding to one or more blood vessels included in the vascular image; and Equipped with A user interface for semi-automatic segmentation of vascular pathways.

2. The updated vascular route is presented simultaneously with the default vascular route. The user interface of claim 1 .

3. The position indication is related to the one or more blood vessels. The user interface of claim 1 .

4. The fitting of the default vascular path is based on an active contour algorithm. The user interface of claim 1 .

5. The one or more position indications include a plurality of position indications, and the updated vascular route is generated subsequent to the user input. The user interface of claim 1 .

6. The one or more position indications include a first position indication, and the updated vascular route is generated to extend the default vascular route to the first position indication. The user interface of claim 1 .

7. The user input reflecting the first position indication includes a click. The user interface of claim 6.

8. Following generation of the updated vascular path, user input reflecting a second position indication is accepted, and the updated vascular path is extended to the second position indication. The user interface of claim 6.

9. The method of claim 8, wherein the user input includes dragging an end of the default vascular route. The user interface of claim 1 .

10. A method performed by a system of one or more processors, comprising: presenting an automatically generated default vascular path extending between two target endpoints of the vascular image; accepting user input reflecting one or more position indications following the two target endpoints; generating an updated vascular path based on fitting the default vascular path through the one or more location indications and based on a geometry corresponding to one or more blood vessels included in the vascular image; Including, method.

11. The updated vascular route is presented simultaneously with the default vascular route. The method of claim 10.

12. The position indication relates to the one or more blood vessels. The method of claim 10.

13. The fitting of the default vascular pathway is based on an active contour algorithm. The method of claim 10.

14. The one or more position indications include a plurality of position indications, and the updated vascular route is generated following the user input. The method of claim 10.

15. The method according to claim 1, wherein the one or more position indications include a first position indication, and the updated vascular route is generated to extend the default vascular route to the first position indication. The method of claim 10.

16. The user input reflecting the first position indication includes a click. The method of claim 15.

17. Following generation of the updated vascular path, user input reflecting a second position indication is accepted, and the updated vascular path is extended to the second position indication. The method of claim 15.

18. The method of claim 17, wherein the user input includes dragging an end of the default vascular route. The method of claim 15.

19. A system comprising one or more processors and a computer storage medium having stored thereon instructions that, when executed by the one or more processors, cause the one or more processors to: presenting an automatically generated default vascular path extending between two target endpoints of the vascular image; accepting user input reflecting one or more position indications following the two target endpoints; generating an updated vascular path based on fitting the default vascular path through the one or more location indications and based on a geometric shape corresponding to one or more blood vessels included in the vascular image; system.

20. The updated vascular route is presented simultaneously with the default vascular route.

20. The system of claim 19.

Images