How to select blood vessels from an image

Abstract

To provide methods and systems for manually assisted determination of vascular features.SOLUTION: In some embodiments, a vascular center line identified with image analysis is divided into segments. Optionally, one or more candidate paths extending to a root position of a blood vessel along the continuous segments is generated for each of a plurality of vascular end points. The anatomical path selection of the blood stream is based on the optionally manual confirmation. Optionally, the path is manually determined on the basis of, for example, selection of a plurality of way points. Optionally, the path is edited by the use of a dynamic profile method that enables the relatively rough display of correction to the path change that strictly matches the geometric shape of a nearby vascular system.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] TECHNICAL FIELD This disclosure relates generally to the field of anatomical segmentation, and more particularly to manually assisted segmentation of branched vascular anatomy. [Background technology]

[0002] Vascular segmentation and feature identification are preliminary steps in image-based measurement of vascular status. Although many steps of vascular segmentation and feature identification can be performed preliminarily based on automated analysis, the relevant image features are often low contrast and / or embedded in a complex environment with ambiguous geometric and irrelevant feature elements. Human supervision may be introduced into the workflow to help make corrections and ensure the quality of the results, leading to semi-automatic processing, such as with Livewire and related procedures (discussed, for example, in "Vascular Surgery and Imaging: A Case Study of Vascular Surgery and Imaging," vol. 1, no. 1, pp. 1171-1175).

[0003] Additional background art includes the articles in Non-Patent Document 2, Non-Patent Document 3, and Non-Patent Document 4. [Prior art documents] [Non-patent literature]

[0004] [Non-Patent Document 1] Ryan Dickie, et al.; Live-vessel: Interactive vascular image segmentation with simultaneous extraction of optimal medial and boundary paths. Technical report TR 2009-23, School of Computing Science, Simon Fraser University, Burnaby, BC, Canada, November 2009 [Non-Patent Document 2] "Snakes: Active contour models", by M. Kass, A. Witkin, and D. Terzopoulos, published in Int. J. Comput. Vis. (1987), 1:321-331 [Non-Patent Document 3] "Multiscale vessel enhancement filtering", by AF Frangi, WJ Niessen, KL Vincken, MA Viergever, published in Medical Image Computing and Computer- Assisted Intervention-MICC' 98 [Non-Patent Document 4] "Snakes, Shapes, and Gradient Vector Flow", by C. Xu and JL Prince, published in IEEE Transactions on Image Processing (1998), 7:359-369 Summary of the Invention [Means for solving the problem]

[0005] According to some exemplary embodiments, a method for segmenting a vascular image into vascular pathways for determining a blood flow pathway is provided. The method includes determining end regions of first and second target vascular pathways in the vascular image, identifying locations of portions of a blood vessel in the vascular image, and automatically generating a plurality of vascular pathway options from the identified portions of the blood vessel, each vascular pathway option defining a potential vascular pathway extending between end regions of the first and second target vascular pathways. The method may also include displaying a plurality of vascular pathway options registered in the vascular image for selection by a user, each respective displayed vascular pathway option including the first and second target vascular pathway end regions. The exemplary method may further include receiving a pathway option selected by the user for determining the blood flow pathway.

[0006] According to some embodiments, a plurality of routes are automatically generated based on a first set of criteria and a user selected route option is selected based on a second set of criteria.

[0007] According to some embodiments, pre-determining includes ranking the vascular route options in order based on an assessment of the likelihood that each vascular route option corresponds to an actual route of blood flow in the blood vessel imaged in the vascular image.

[0008] According to some embodiments, pre-determining includes applying a cost function that assigns a numerical cost to one or more features associated with the vascular routing options.

[0009] According to some embodiments, the cost function assigns a numerical cost based on characteristics of the centerline of the vascular segments from which the vascular route options connect.

[0010] According to some embodiments, the centerline characteristics of the plurality of vessel segments include one or more of the group consisting of centerline orientation, centerline offset, and number of centerlines extending from the nodal region.

[0011] According to some embodiments, the cost function assigns a numerical cost based on the characteristics of the vascular image over which the vascular path option extends.

[0012] According to some embodiments, the vascular image features include one or more of the group consisting of: continuity of vascular segment image intensity, continuity of vascular segment image width, and location of relative change in vascular intensity relative to a nodal region, the nodal region having three or more vascular segments extending therefrom.

[0013] According to some embodiments, the pre-determining includes application of a cost function that assigns a numerical cost based on an estimated relative position of the vessel segment image in depth with respect to an axis extending perpendicular to the plane of the vessel image.

[0014] According to some embodiments, displaying includes presenting the plurality of vascular routing options in a sequential order determined by a selection order.

[0015] According to some embodiments, displaying includes showing multiple vascular route options simultaneously, with the order of selection corresponding to the order in which the vascular route options are displayed as active for selection.

[0016] According to some embodiments, each vascular path option determines a vascular path that extends through an image area between the end regions of the first and second target vascular paths, terminating in a vascular area of ​​the image closest to one of the end regions of the first and second target vascular paths.

[0017] According to some exemplary embodiments, a method for editing a vascular path to more accurately depict a vascular segmentation in a vascular image is provided, the exemplary method comprising receiving an indication of a selected region along the vascular segmentation, determining an energy functional determined as a function of a position along the vascular segmentation, and setting a non-zero region of the energy functional based on the position of the selected region. The method may also include moving the region of segmentation within the non-zero region of the energy functional according to energy minimization.

[0018] According to some embodiments, the energy functional value in the non-zero regions is set based on features of the underlying vascular image.

[0019] According to some embodiments, the energy functional value in the non-zero regions is set based on the movement of the user-controlled position indication.

[0020] According to some embodiments, the indication of the user controlled position includes an indication of a selection area.

[0021] According to some exemplary embodiments, a user interface for semi-automatic segmentation of a vascular pathway is provided, the user interface comprising at least one interface module operable to present an automatically generated default vascular pathway extending between two target termination points, and at least one interface module operable to present at least one additional automatically generated vascular pathway extending between the two target termination points.

[0022] According to some embodiments, the user interface further comprises at least one interface module operably configured to determine at least one way point and to present an automatically generated vascular route extending between the two target end points via the at least one way point.

[0023] According to some embodiments, the user interface further comprises at least one interface module operable to modify the pre-determined vascular route by dragging the position of the vascular route to a new location.

[0024] Unless otherwise defined, all technical and / or scientific terms used herein have the same meaning as commonly understood by those skilled in the art to which this disclosure pertains. Although the systems, methods, devices and / or computer program products described herein, or similar or equivalents thereof, can be used to practice or test the disclosed embodiments, exemplary systems, methods, devices and / or computer program products are described below. In case of conflict, the patent specification, including definitions, will prevail. In addition, the systems, methods, devices, computer program products and examples are merely illustrative and are not necessarily intended to be limiting.

[0025] As will be apparent to those skilled in the art, aspects of the present disclosure may be embodied as a system, method, or computer program product. Accordingly, aspects of the present disclosure may take the form of an entirely hardware embodiment, an entirely software embodiment (including firmware, resident software, microcode, etc.), or an implementation of a combination of software and hardware aspects, all of which may be generally referred to herein as a "circuit," "module," or "system." Furthermore, some embodiments of the present disclosure may take the form of a computer program product embodied in one or more computer readable mediums having computer readable program code embodied therein. Implementation of the methods and / or systems of some embodiments of the present disclosure may include performing and / or completing selected tasks manually, automatically, or a combination thereof. Furthermore, depending on the actual instrumentation and equipment of some embodiments of the methods and / or systems of the present disclosure, some selected tasks may be implemented by hardware, software, or firmware and / or a combination thereof, such as using an operating system.

[0026] For example, hardware for performing selected tasks according to some embodiments of the present disclosure may be implemented as a chip or circuit. As software, selected tasks according to some embodiments of the present disclosure may be implemented as a number of software instructions executed by a computer using any suitable operating system. In an exemplary embodiment of the present disclosure, according to some exemplary embodiments of the methods and / or systems described herein, one or more tasks are performed by a data processor, such as a computer platform for executing a number of instructions. Optionally, the data processor includes a volatile memory for storing instructions and / or data and / or a non-volatile storage medium, such as a magnetic hard disk and / or a removable media, for storing instructions and / or data. Optionally, a network connection may also be provided. A display and / or a user input layer value, such as a keyboard or mouse, may also be provided.

[0027] Any combination of one or more computer readable media may be used for some embodiments of the present disclosure. The computer readable medium may be a computer readable signal medium or a computer readable storage medium. The computer readable storage medium may be, for example, but not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any suitable combination thereof. More specific examples (non-exhaustive list) of computer readable storage media include an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In the context of this document, a computer readable storage device may be any tangible medium that can contain or store a program for use by or in connection with a system, apparatus, or device that executes instructions.

[0028] A computer-readable signal medium may include a propagated data signal having computer-readable program code embedded therein, for example in baseband or as part of a carrier wave. Such a propagated signal may take any form, including but not limited to electromagnetic, optical, or any combination thereof. A computer-readable signal medium is not a computer-readable storage medium, but may be any computer-readable medium that can communicate, propagate, or transmit a program for use by or in connection with a system, apparatus, or device that executes instructions.

[0029] Program code embodied on a computer readable medium and / or data used thereby may be transmitted using any suitable medium, including but not limited to wireless, wired, fiber optic cable, RF, etc., or any suitable combination thereof.

[0030] Computer program code for carrying out the operations of some embodiments of the present disclosure may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, Smalltalk, C++, etc., and conventional procedural programming languages, such as the "C" programming language or similar programming languages. The program code may run entirely on the user's computer, partially on the user's computer, as a standalone software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In the latter scenario, the remote computer may be connected to the user's computer through any type of network, including a local area network (LAN) or a wide area network (WAN), or the connection may be made to an external computer (e.g., through the Internet using an Internet Service Provider).

[0031] Some embodiments of the present disclosure are described below with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems) and computer program products according to embodiments of the present disclosure. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be realized by computer program instructions. The computer program instructions may be provided to a processor of a general-purpose computer, a processor of a special-purpose computer, or other programmable data processing apparatus to manufacture a machine, such that these computer program instructions, executing via a processor of a computer or programmable data processing apparatus, generate means for performing the functions / operations specified in the blocks of the flowchart illustrations and / or block diagrams.

[0032] These computer program instructions may be stored on a computer readable medium, which instructions stored in the computer readable medium may instruct a computer, other programmable data processing apparatus, or other device to function in a particular manner to generate an article of manufacture including instructions that perform the functions / acts identified in the flowchart and / or block diagram blocks.

[0033] Computer program instructions may be loaded onto a computer, other programmable data processing apparatus, or other device for causing a sequence of operational steps to be performed on the computer, other programmable apparatus, or other device to produce a computer-implemented process, such that the instructions executing on the computer or other programmable apparatus provide a process implementing the functions / acts identified in the flowchart and / or block diagram blocks.

[0034] Additional features and advantages of the disclosed systems, methods, and apparatus are described in, or will be apparent from, the following detailed description and drawings.

[0035] Some embodiments of exemplary systems, methods and / or computer program products are described herein, by way of example only, with reference to the accompanying drawings. Reference will now be made in detail to certain drawings, where it is emphasized that the details are by way of example and are for system, method, apparatus and / or computer program product embodiments. In this regard, the description taken together with the drawings will make apparent to those skilled in the art how embodiments of the systems, methods, apparatus and / or computer program products may be implemented. [Brief description of the drawings]

[0036] [Figure 1] FIG. 1 is a flowchart of a method for determining a vascular centerline path according to some exemplary embodiments of the present disclosure. [Diagram 2] FIG. 2 is a schematic flow chart of a method for generating vascular routing options from a vascular image according to some exemplary embodiments of the present disclosure. [Figure 3A] FIG. 3A is a schematic flow chart of a method for selecting and / or determining a particular vascular routing option according to some exemplary embodiments of the present disclosure. [Figure 3B] FIG. 3B is a schematic flow chart of a method for manually editing a vascular pathway according to some exemplary embodiments of the present disclosure. [Figure 4A] FIG. 4A is a diagram outlining selection of vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 4B] FIG. 4B is a diagram outlining selection of vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 4C] FIG. 4C is a diagram outlining selection of vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 4D] FIG. 4D is a diagram outlining selection of vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 4E]FIG. 4E is a diagram outlining selection of vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 4F] FIG. 4F is a diagram outlining selection of vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 5A] FIG. 5A is a diagram outlining selection from alternative vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 5B] FIG. 5B is a diagram outlining selection from alternative vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 5C] FIG. 5C is a diagram outlining selection from alternative vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 5D] FIG. 5D is a diagram outlining a method for manually determining vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 5E] FIG. 5E is a diagram outlining a method for manually determining vascular routing options according to some exemplary embodiments of the present disclosure. [Figure 5F] FIG. 5F is a diagram outlining manual editing of vascular route options according to some exemplary embodiments of the present disclosure. [Figure 6] FIG. 6 is a diagram illustrating an overview of displaying route determinations made simultaneously on multiple different vascular image displays according to some exemplary embodiments of the present disclosure. [Figure 7] FIG. 7 is a schematic diagram of software modules and data structures implemented in a system for semi-automatic segmentation of vascular pathways according to some exemplary embodiments of the present disclosure. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0037] The present disclosure, in some embodiments thereof, relates to the field of anatomical segmentation, and in particular to manually assisted segmentation of branched vascular anatomical structures.

[0038] overview A broad aspect of some embodiments of the current disclosure relates to a combination of manual and automatic segmentation techniques, potentially enabling the generation of an effective and reliable vascular tree.

[0039] Vascular segmentation is an initial step in the characterization of vascular anatomy and function from medical imaging data for various applications, including blood flow studies. However, automated segmentation methods generally show a decline in performance as demands become more demanding. In addition, contrast agents are limited in the concentrations at which they can be used, and injectable contrast agents are quickly diluted, limiting the available imaging time. Furthermore, for safety reasons, it is preferable that the radiation dose used in some types of imaging be kept to the minimum necessary for reliable visualization. Even though high contrast and / or high signal-to-noise can often be achieved for large vessels, there is also a need in some applications to analyze smaller vessels. However, imaging quality rapidly declines with decreasing vessel diameter, as the signal intensity approaches the limit of background noise or quantum noise inherent in the signal itself. Apart from such technical considerations, the vasculature itself is a very complex shape. This potentially leads to many cases of ambiguous structure, especially in 2D images, where it is difficult to determine the specific branching structure by inspection at the level of local features. However, the detection of a wide range of features is difficult to address using conventional automated methods, since the interpretation of such features potentially depends on the specificity of the particular structure and / or constraints applicable to the imaging method.

[0040] For these and other reasons, practical cases often end up with the quality of medical images available for analysis approaching or exceeding the limits of current techniques for machine vision and / or image processing, for at least some vascular structures of interest. Even if the boundaries of these limits move over time due to technological developments, it is expected that there will continue to be interesting segmentation problems for which automatic vessel segmentation alone is insufficient.

[0041] Semi-automatic segmentation methods seek to address the limitations of purely automatic segmentation methods by augmenting them with human judgment and / or control. However, human intervention is costly in terms of time, funds, and / or availability. In view of this, a goal in some semi-automatic segmentation methods is to reduce the human time and / or effort spent supervising the automatic segmentation. Given the optimization problem, a goal in some embodiments of the present disclosure may be to approach the lowest achievable level of human intervention in semi-automatic segmentation that is consistent with results of sufficient quality for the application to which they are applied. Thus, suitable methods of semi-automatic segmentation may include features specific to their application in a particular problem domain.

[0042] Aspects of some embodiments of the present disclosure relate to a cascaded method for semi-automatic vessel segmentation of angiographic images. More particularly, in some embodiments, this aspect relates to a cascaded method for semi-automatic segmentation of angiographic images, working within the constraints of producing results in real time. Optionally, the segmentation is completed while a catheter procedure that may have been used to generate the image is ongoing, leaving sufficient time for subsequent analysis, e.g., analysis leading to a diagnosis and / or treatment plan.

[0043] In some embodiments, the manual management of the semi-automatic vessel segmentation is configured to cascade through an increasingly demanding set of user operations, with the cascade stopping once the user is satisfied that a result of sufficient quality has been obtained. There may be one or more routes through this cascade of operations. For example, the order of operations selected may optionally depend on whether a near-good result was obtained earlier, requiring only minor editing, or whether the user needs to determine a new suitable path. In some embodiments, the cascade may be configured such that more likely options are presented earlier and / or given more weight, potentially reducing the time and / or effort the user spends making a selection. Optionally, the order of operations is selected to emphasize obtaining a "near enough" result with minimal input, while also providing an opportunity to correct errors in the automatically identified result if necessary.

[0044] In some embodiments, the method optionally begins with the determination of two vascular path end regions (e.g., regions within some maximum distance from a selected point), after which the most likely automatically detected path is presented to the user for acceptance or rejection. Due to the appropriate determination of the "most likely" (e.g., an appropriate cost function), this potentially allows most or some user intervention to be limited to simple acceptance of defaults. Optionally, multiple vascular path end region pairs are first determined and corresponding multiple default options are simultaneously presented, with user interaction still further limited to correction of the defaults. In some embodiments, the determination of one of the endpoints is simplified by determining at least one endpoint at a root position of the vascular tree, positioned such that it is considered to be at one end of any vascular path leading back from one of its branches. In some embodiments, determination of the terminal positions of multiple blood vessels is based on fully automatic detection (e.g., the position where the vascular skeleton naturally terminates, determining the vascular centerline), or semi-automatic methods, such as the position where a selection line swept by the user crosses a segment of the vascular skeletonization that is automatically detected.

[0045] In some embodiments, if the default route is unacceptable, automatically suggested routes of decreasing probability (higher cost function scores) are optionally displayed as available. For example, the user can use a scroll wheel or other control to quickly indicate alternative routes that extend between several pairs of the target vascular route end regions.

[0046] If the automatically suggested alternative guidance fails to find a suitable route, in some embodiments, the user is presented with a user interface tool operable to determine a vascular route based on determining one or more additional waypoints.

[0047] Additionally or alternatively, one or more editing tools are optionally provided that allow the generally acceptable displayed alternatives to be modified. For example, the path may be cut, extended, and / or merged with other existing path portions. In some embodiments, the path is optionally edited along its length, for example, by retracing, redetermining anchor points, and / or dragging erroneous segment regions to their correct positions.

[0048] In some embodiments of the present disclosure, the goal of semi-automatic vessel segmentation is the production of one or more vessel paths corresponding to anatomically plausible paths of blood flow. In some embodiments, the vessel path comprises numerically stored successive positions corresponding to vessel image positions. In some embodiments, the vessel path comprises a vessel centerline. Optionally, the vessel path is determined at one end by a root position, e.g., located at the end of the path that is within the least branched vessel that the path crosses. At the other end, the vessel path is optionally determined by a terminal position, e.g., located within the most branched vessel that the path crosses.

[0049] In some embodiments, the vascular path is determined by the concatenation of one or more vascular segments. Optionally, the vascular segments are determined by one or more methods, at one or more levels of fidelity. In some embodiments, the vascular segments may be determined within a skeletonized representation of the vascular image (e.g., a binary pixel array representation extending through an extent of the detected vascular structure that is one pixel wide). Such vascular segments optionally comprise a sequence of pixel locations extending between two pixels that mark end points. The end locations are optionally selected by any convenient, even arbitrary method (e.g., dividing the skeleton into segments of up to N pixels in length). However, preferably, the end locations of the vascular segments are determined at bifurcations and / or crossings (e.g., skeleton pixels from which branches extend in at least three directions) and / or at free ends (e.g., skeleton pixels from which only one branch extends).

[0050] In some embodiments, the vascular paths are determined in isolation from one another. In some embodiments, the vascular paths are determined by their different extents along a branched vascular tree (e.g., by specific paths traversing the branches of the vascular tree, optionally determined as a set of linked vascular segments). In contrast, in some embodiments, the vascular tree is determined by amalgamation of a set of vascular paths, e.g. paths sharing a common vascular segment are interpreted as also sharing a common root segment.

[0051] An aspect of some embodiments of the present disclosure relates to a method for selecting a vascular route based on a presentation order of automatically generated vascular route options.

[0052] In some embodiments of the present disclosure, multiple path routes extending along the detected vessel segments (e.g., vessel segments determined according to criteria of slope, curvature and / or relative strength) are generated for a pair of end regions. Optionally, the generated path routes reach segment points that are within some region, optionally determined, for example, as a region that is within some maximum distance from the end points. This distinction is relevant, for example, when one of the endpoints is not on the segment.

[0053] In some embodiments, the multiple path routes are presented as a range of vascular path options in a predefined order. Optionally, the predefined order is based on a cost function configured to rank the path routes according to a criterion (optionally a heuristic criterion) that assigns a lower cost value to more likely actual blood paths between two end points than to less likely actual blood flow paths. Preferably, the path routes are presented along with the original image data, e.g., as graphical overlays on the image. In some embodiments, the image data is presented as an animated sequence of images, e.g., slight movements of the blood vessels and / or images viewed from different angles. Potentially, these differences benefit the visual capabilities of the user more specifically than for an automatic detection algorithm, e.g., by highlighting connectivity between parts of the vasculature that move together.

[0054] In some embodiments, the criteria of the cost function include aspects of vascular continuity, vascular branch anatomy, criteria based on the three-dimensional shape of the organ perfused by the vessel, vascular image opacity, and / or other criteria.

[0055] In some embodiments, the results of the automatic segmentation potentially do not uniquely distinguish which of the multiple vascular paths between two vascular locations corresponds to the actual path of blood flow. In particular in the case of 2D images of 3D vascular structures (e.g., vascular structures that are typically visualized as extending through two or more layers of depth), some vascular tree branches may appear to cross each other. Although the same tree crossing is rarely observed in structures that are typically imaged within one focal plane, such as the inner retinal vasculature, ambiguity may arise when it is difficult to distinguish between venous and arterial structures.

[0056] It should be noted that dynamic path extending methods such as Livewire partially reduce the ambiguity of automatic segmentation by being able to determine waypoints that can be set to steer the automatic results towards a correct segmentation. However, this is potentially more time consuming than simply determining a start point and an end point. In some embodiments, there is a trade-off between determining a correct but slower vascular path by using waypoints and determining a less accurate but potentially faster vascular path by choosing from an option of automatically pre-determined vascular paths.

[0057] An aspect of some embodiments of the present disclosure relates to a method for editing a vessel path by dragging an incorrectly segmented region into alignment with a more accurately segmented location.

[0058] In some embodiments, the method includes receiving an indication of a selected region along the vessel segmentation. In some embodiments, the indication includes selection of a screen coordinate point, for example by pressing a button along with a specific position of a screen cursor and / or by gesture input to a touch screen. In some embodiments, the indication includes directly selecting the incorrectly segmented region.

[0059] In some embodiments, the method includes assigning motion-sensitive properties to segmentation path regions surrounding the selected region that allow the selected region to move, preferably while keeping the remainder of the segmentation insensitive to motion. In some embodiments, the assignment of motion sensitivity includes determining an energy functional determined as a function of position along the vessel segmentation. Optionally, non-zero regions of the energy functional are set based on the position of the selected region. In some embodiments, the method includes moving the region of the segmentation according to an energy minimization within the non-zero regions of the energy functional. Optionally, the movement of the motion-sensitive portion of the segmentation path region is coordinated with the movement of the cursor. Optionally, the movement of the motion-sensitive portion is at least partially integrated by values ​​and / or gradients of image intensities in the vicinity of the selected region such that the selected region behaves as if attracted to the movement of the vessel region.

[0060] Before describing at least one embodiment of the present disclosure in detail, it is to be understood that the illustrated systems, methods, apparatus, and / or computer program products are not necessarily limited in their application to the details of construction, the arrangement of components, and / or methods set forth in the following description and / or drawings, as the illustrated systems, methods, apparatus, and / or computer program products are capable of other embodiments and of being practiced or carried out in other ways.

[0061] Method for determining a path along a vascular tree Reference is now made to FIG. 1, which is a flow chart outlining a method for determining a vascular centerline path in an angiographic image, according to certain exemplary embodiments of the present disclosure.

[0062] In some embodiments, an edit mode of a computer program configured for interactive vessel path determination via user interaction through a user interface is initiated. In some embodiments, in block 102, a root location (e.g., corresponding to root locations 401, 501 in Figs. 4A-4F and 5A-5F) is determined. In some embodiments, the root location is the vessel location visible in the image that is topologically closest to the area where blood enters or exits the heart. In a coronary artery, for example, the root location is topologically closest to the area where blood exits the left ventricle into the aorta. Optionally, the root location is determined manually, for example, by clicking or touching (via a user interface device) a point in the image, and / or by selecting, moving, and / or confirming an automatically detected root location. Optionally, an automatic detection of one or more candidates for the root location is performed, for example, based on the timing and / or location of dye appearance in an image time series immediately after injection, and / or based on morphological criteria such as vessel thickness, branch order, and / or direction.

[0063] In some embodiments, in block 110, the determined root location is used as an input for determining route data including alternative route options and costs of the route options, as described, for example, in connection with FIG. 2.

[0064] In some embodiments, in block 112, a vascular path representative of a path of successive vascular connections between several points in the vascular tree image and a root location is determined arbitrarily (and / or selected, for example, based on a list of alternative path options and path option costs determined in block 110). A detailed implementation example of block 112 is described in connection with FIG. 3A.

[0065] In block 114, the vascular path representative of the previously determined path of successive vascular connections between several points in the image of the vascular tree and the root location is optionally edited. An example of a detailed implementation of block 114 is described in relation to FIG. 3B.

[0066] In some embodiments, at block 104, a decision is made whether to continue in edit mode, and if not, the flow chart is exited, otherwise the flow returns before block 112 (optionally before block 102 to allow redetermining of the root position).

[0067] Vascular Paths and Cost Functions Reference is now made to FIG. 2, which is a schematic flow chart of a method for generating vascular routing options from a vascular image according to some exemplary embodiments of the present disclosure.

[0068] In some embodiments, the flow chart begins at block 208, where the vascular skeleton graph 206 splits into branches. In some embodiments, the vascular skeleton graph is obtained from image processing of angiographic images. Images 400 of FIGS. 4A-4F and 500 of FIGS. 5A-5F are examples of source images from which the vascular skeleton graph is obtained. In some embodiments, the vascular skeleton graph includes vascular centerlines. Examples of vascular centerline extraction (in some embodiments using anisotropic diffusion, Frangi filtering, and hysteresis thresholding followed by binary thinning) are described, for example, in conjunction with block 20 of FIG. 14 of International Publication No. WO 2004 / 11927, filed Jan. 15, 2014 to the applicant, the contents of which are incorporated herein by reference in their entirety. In some embodiments, the basis of the method used is similar to that introduced by Weickert in "A Scheme for Coherence-Enhancing Diffusion Filtering with Optimized Rotation Invariance" and / or "Anisotropic Diffusion in Image Processing" (Thesis 1996).

[0069] In some embodiments, in block 208, the vascular skeleton graph 206 is divided into branches. Optionally, this includes noting pixels where three or more skeleton segments converge (e.g., pixels with three or more neighboring pixels, optionally excluding short branches from consideration) and assigning path breaks to these points. Optionally, attention is also given to where the branches connect (connections are simply apparent, as branches may cross one another and / or approach closely enough to appear to join). The result of the operation of block 208 is a branch list 210. It should be noted that the "branches" in the branch list 210 are potentially linked into loops due to the actual underlying anatomy (e.g., development of shunt vessels) and / or apparent connections resulting from closely approaching vessels and / or intersections at the angles provided and / or due to image resolution.

[0070] In some embodiments, the branches stored in the branch list are only those that have a continuous, connected route to a root location in the vascular tree (e.g., root locations 401, 501 in images 4A-4F and 5A-5F), optionally determined as described in connection with block 102 of FIG.

[0071] In some embodiments, in block 212, paths extending from each branch to the determined root location are calculated, for example by using a search algorithm. Optionally, two or more other such paths are available, for example due to intersections, close proximity, or other ambiguities present in the image as described above. Paths that include the same vascular segment more than once within them are optionally excluded from this list (to avoid paths that loop and / or back on themselves). The data structure resulting from the operation of block 212 includes a path list 214. In some embodiments, the path list 214 includes each identified non-excluded path extending between the root location and each vascular skeleton segment.

[0072] In some embodiments, at block 216, a path cost is calculated for each path in the path list 214 to generate a path cost list 218. The path cost list 218 is configured in some embodiments such that members of potentially multiple paths in the path list 214 that reach from any vessel segment to a root can be ranked in order of likelihood of being an actual path of blood flow. Use of the path list 214 and / or the path cost list 218 is further described in connection with the operation of block 112 of FIG. 1, e.g., as detailed in connection with FIG. 3A.

[0073] According to the convention adopted herein, higher cost pathways are optionally considered less likely candidates for being actual pathways of blood flow in the vascular anatomy. The pathway cost criteria are optionally evaluated to receive a binary value (e.g., 0 or 1), a ranked value, and / or a continuous scale score. The pathway cost criteria are optionally added to, multiplied with, or otherwise incorporated into the overall cost function.

[0074] In some embodiments, the results of the machine learning are used to assign criterion cost values ​​and / or contributions to an overall cost relative to other criteria. In some embodiments, the machine learning techniques include one or more implementations of techniques related to decision tree learning, association rule learning, artificial neural networks, inductive logic programming, support vector machines, cluster analysis, Bayesian networks, reinforcement learning, representation learning, similarity and metric learning, and / or machine learning. The machine learning is optionally based on a set of angiographic images in which the morphology of the vascular tree is separately marked and / or based on the results of its own learning.

[0075] Cost Standards In some embodiments, the route costs recorded in the route cost list 218 are calculated against one or more of the following criteria and / or criteria:

[0076] Node Type Classification Herein, the connection points of the vascular skeleton segments are referred to as nodes. A node may be determined as a single pixel, or optionally, as a region of relatively large size (e.g., 2, 3, 4, 5 or more pixels in extent) within which three or more vascular skeleton segments meet. In some embodiments, the overall cost function applicable to a node depends, at least in part, on the node type classification.

[0077] When two vessel segments meet in the image (e.g., due to being in different planes of the image of the heart), the vessel skeleton shows four branches that are suitably generated from the area of ​​the central node. The corresponding vessel skeleton is optionally analyzed as determining a cross-type node. In some embodiments, when a vessel branches, it is common for it to bifurcate. In such a case, the node is optionally determined as a junction node from which three vessel skeleton branches emerge. The cost function criteria available for these two basic node types are optionally assigned differently.

[0078] In some embodiments, there are other junction orders that may appear in the path list. For example, bifurcations may occur at the same location as a vessel intersection. Another situation that may potentially occur is when two junction nodes appear very close to each other and are analyzed as being a quad-furcation node. In contrast, skeletonization artifacts may potentially result in a slight shift for the same vessel on either side of the intersection, making it look better as two adjacent junction-type nodes. Tri- and quad-furcation nodes may also occur when a terminal vessel segment (i.e., the vessel segment that is the last segment of its branch visible in the image) has a free end that is located at or near the location of another vessel segment. Optionally, nodes containing five, six or more segments are analyzed as consisting of an appropriate combination of junction nodes, intersection nodes and / or free-end terminations.

[0079] In some embodiments, morphological criteria other than the number of branches are used in node classification for the purpose of cost function assignment. For example, some closely sequential branches (bifurcations) simply include a pair of segments in one consecutive direction while the other two segments extend at a relatively acute angle (e.g., about 90°) relative to each other. In other examples, the branching angle typically (but not exclusively) indicates an acute angle in the forward direction (direction of arterial flow). A branching angle to the rear is more likely to indicate a cross-type node. In yet other examples, the region of vessel crossing may appear as an area of ​​increased radiopacity (due to the cumulative contribution of the thickness of the two overlapping vessels) compared to the region where the vessels branch. Conversely, a sudden change in apparent radiopacity appearing at either end of the node due to a change in vessel orientation (and a corresponding change in absorbing path length) relative to an axis perpendicular to the image plane is indicative of a potentially bifurcated node.

[0080] Optionally, for more complex classification of nodes, e.g., r p 3 =r d1 3 +r d2 3 r d3 3 +···+r dn 3 In the case of r p 3 is the radius of the parent branch (trunk), and r d1 3 , r d2 3 , r d3 3 ···r dn 3 We apply Murray's principle, characterized in that x is the radius of each child branch. Optionally, combinations of vessel segments that satisfy Murray's principle at some nodes are assigned a higher probability of representing a branched node, and the remaining segments are assigned a higher probability of being free ends or participants in a cross-node structure.

[0081] In some embodiments, the appropriate cost function for a node depends on which analysis of the node type is correct. In some embodiments, the determination of the node type is up to a probabilistic assignment, for example, a node is assigned an 80% chance of being a crossing of two vessels, a 15% chance of being a bifurcation (or a bifurcation into three), and a 5% chance of being a bifurcation in the region of the free end of a terminal vessel segment.

[0082] In some embodiments, there is no explicit classification of cost functions between junction-type nodes and cross-type nodes (or free ends, or any combination of these types), but rather a heuristic is employed that operates on the underlying relative vessel morphology as such. This approach may be preferable for certain types of implementations based on machine learning, such as artificial neural networks. Nevertheless, node correspondence provides a convenient organizing concept for the purposes of describing further vessel path cost function criteria.

[0083] Continuity and / or consistency at cross-type nodes In some embodiments of the present disclosure, the continuity of the vessel morphology at the nodes is used as the basis for assigning a cost function value.

[0084] The concentration of four vascular skeleton branches, interpreted as potentially cross-type nodes, is optionally analyzed to determine three a priori options for directional continuation (outlet side) from some nearby fourth direction (inlet side). In some embodiments, one or more criteria related to morphological continuity are applied to determine which outlet side branch is the most likely continuation branch from a given inlet side.

[0085] One such criterion is the vessel radius, characterized in that the vessel branch at the outlet side with the most similar radius to the inlet side receives the correspondingly lowest cost allocation. Optionally, the vessel radius is measured as half the apparent width of the filled lumen in the image. Optionally, the continuity of the image intensity values ​​(which are partly, but not exclusively, a function of radius) is also taken into account, for example by direct comparison of the intensity values. Optionally, the principle of densitometry is applied, for example, the vessel is modeled as a cylinder of a certain radius filled with a substance having a certain concentration and / or absorption coefficient, and the cost function is jointly based on the radius and the continuity of the absorption properties to satisfy the intensity values ​​observed in the image.

[0086] In some embodiments, directional continuity is the criterion, and the outlet vessel segment that is most similar to the inlet segment orientation optionally receives the lowest cost assignment as its continuation. Note that there may also be differences in density measurements based on the vessel's orientation in the direction perpendicular to the image plane, e.g., vessels appear darker when viewed closer to the edge. Thus, continuity of vessel intensity values ​​is optionally a proxy for vessel orientation continuity in this dimension.

[0087] In some embodiments, these criteria include one or more further refinements. For example, the directional criteria applied to junctions may optionally include a cost assignment based on the continuity ratio and / or the direction of the angle change before and after the node (e.g., continuity between two segments with a more similar radius of curvature receives a lower cost assignment). This potentially has particular relevance for vessel segments that go through a portion of the vessel image and around a bulge in the 3D heart surface. In some embodiments, the cost function criteria favors paths where the vessel width changes automatically along the path (e.g., increasing in the route direction).

[0088] In some embodiments, paths from segments A, B, C, D that meet at a junction (particularly a cross-type junction) are treated as "entangled" such that a lower cost for the path extending from the corresponding A→B lowers the cost of path C→D compared to the alternative path C→B. Optionally, a definitive indication by the user during some stage of the vascular tree diagram that path A→B is the preferred path of vascular flow will likewise be confounded in lowering the cost assignment of path C→D compared to path C→B.

[0089] Continuity and / or consistency at branching nodes In some embodiments, when a vessel branches, it is typically a bifurcation. Optionally, higher bifurcations, such as trifurcation, are treated as adjacent bifurcation configurations. At the bifurcation, a junction node is optionally determined, with a priori options for two directions to continue (outlet side) from an approaching optional third direction (entrance side). In some embodiments, a helical approach is employed, where the outlet branch that requires the least change in direction through the junction node receives the lower cost assignment.

[0090] Optionally, the cost scoring based on vessel radius similarity reduces its own influence (e.g., by lowering its weight) on the cost function when a three-way junction node is detected to account for the fact that two thin branches are potentially closer in size to each other than to their common main body. Additionally or alternatively, the radius-based cost function is assigned differently for branched nodes, for example, the thickest vessel segment (generally located toward the root location) is given the lowest cost score for both of the two thinner branches. Optionally, the cost score assignment is further specifically based on Murray's rule, characterized in that paths that better satisfy this theoretical relationship are given a correspondingly lower cost score.

[0091] Other criteria In some embodiments, paths using vessel segments that run in relatively straight lines and / or continuous arcing lines are optionally scored with a lower cost. This is a potential advantage for preferred image features that are "more vascular" in nature versus image features that may terminate in the vessel skeleton but are not actually native vessels. The cost is optionally calculated based on, for example, the total area under the curve (above or below zero) of the derived angle of the local vessel segment direction.

[0092] In some embodiments, the location of the vessels in three dimensions is estimated. Optionally, the use of this information is part of node classification and / or used for continuity / consistency cost scoring. In some embodiments, the vascular tree is known a priori to extend across a three-dimensionally curved surface, such as the surface defined by the shape of the myocardium. In some embodiments, parameters of this curved surface are derived from image shadows that follow the shape boundary. In some embodiments, the shape and / or intensity of the vessels themselves distinguish the shape boundary. For example, as vessels curve further in / out from the image plane, they exhibit a longer absorption cross section and therefore potentially begin to appear more radiopaque at some boundary limits. A vessel going around a curve in three dimensions may first appear to curve closer to and away from some boundary limits in a 2D image. In some embodiments, the shell is modeled as having two parts that generally occupy different depths but are joined together at such boundary limits. Optionally, vessel segments are scored for possible locations on the upper or lower parts of the shell, treating the boundary limits as transition zones. Paths that require an abrupt transition between two different shell parts (partially away from the boundary limits) are optionally given a higher cost value.

[0093] In contrast, it should be noted that vessels of the same type (artery or vein) that occupy the same anatomical plane may in some embodiments be estimated to be relatively unlikely to cross one another. Optionally, if a cross-type node is identified with a high probability (additionally or alternatively indicating a vessel crossing with a relatively low associated path cost with consistency and continuity), this may be used to increase the estimated likelihood that two different vessels occupy two separate shell parts. And other vessel segments that directly contact them are even more likely to be in the same shell part, information that is optionally used to disambiguate the assignment of path costs across adjacent nodes.

[0094] Selection and / or determination of vascular route Reference is now made to FIG. 3A, which is a schematic flow chart of a method for selecting and / or determining a particular vascular path option, according to some exemplary embodiments of the present disclosure. In some embodiments, the operations of FIG. 3A correspond to the operations occurring in block 112 of FIG. 1. Reference is also made to FIGS. 4A-4F, which generally illustrate the selection of a vascular path option, according to some exemplary embodiments of the present disclosure. FIG. 4A shows an angiographic image 400 illustrating a portion of a vascular tree 402 (in the example, a cardiac vessel) including a currently determined root position 401. In FIGS. 4B-4F, the contrast of the angiographic image 400 has been reduced to make other elements more visible.

[0095] In some embodiments of the present disclosure, one or more manually guided methods of identifying, selecting, and / or determining a vascular tree topology are provided. In this context, an "accepted routes list" 324 is introduced. The accepted routes list 324 may include routes optionally selected from the routes list 214, routes edited based on routes in the routes list 214, and / or newly generated routes based on user input, for example. In some embodiments, the route cost list 214 is used in determining the display order of the default selection from the routes list 214 and / or additional selections from the routes list 214, as appropriate.

[0096] We will now describe the generation of the data structure including the path list 214 in relation to block 212 of FIG. 2. In some embodiments, the path list 214 potentially includes many vascular paths that are "true" vascular paths in the sense that they are isomorphic to the vascular paths through which blood flows in the actual patient anatomy as they flow between the root location 401 and the far end of the vascular path. However, there may be many "false" vascular paths within the same path list 214, for example reflecting ambiguities introduced by vascular crossing points and / or limitations in vascular image quality. The path cost list 218 optionally provides a basis for making some paths preferred over others, nevertheless, circumstances may lead to true vascular paths being less preferred. In some embodiments, true vascular paths may not even be available in the path list 214 due to contrast dropout and / or other artifacts that may affect vascular skeletonization.

[0097] In some embodiments, at block 310, the end of the path is indicated (e.g., by a user). In some embodiments, the indication of the end of the path includes a "hover" cursor input event (e.g., a long touch on a touch screen that moves the cursor 405 to the vicinity of a potential vascular path and holds it still long enough for detection for processing). In some embodiments, other input events, such as a screen tap, indicate the end of the path. In some embodiments, the user indication refers to an area (e.g., pixels) in the closest available segment shown in the path list 214. Optionally, the user indication refers to an area of ​​the closest segment to the indicated location. For example, the vascular path 407 in FIG. 4B extends toward the path end 407A in the vicinity of the cursor 405 at a location. When the cursor 405 is moved as shown in FIG. 4C, a different vascular path is instead shown, including a vascular path 408 added to the vascular path 407, again terminating at a path end 408A in the vicinity of the cursor 405. Optionally, the user instruction refers to the end of the segment furthest from the root position 401 .

[0098] For purposes of explanation, the conversion of the indicated route endpoints to routes in the accepted routes list 324 is presented primarily in terms of manual operations for one route at a time, however, it should be understood that these descriptions may be modified, as necessary, to apply for the processing of multiple and / or automatically selected routes.

[0099] For example, in some embodiments, one or more candidate path ends are automatically detected (e.g., free ends of vascular skeleton segments that are contiguously connected to the root location 401). Additionally or alternatively, multiple path ends are optionally indicated by a user as part of a single input action, e.g., by dragging out a path (by cursor movement, touch input, or other method) that crosses between several candidate path ends, and the indicated end is taken to be the location where the dragged-out path intersects with a segment of the vascular skeleton.

[0100] If multiple candidate path termini are available, the user is optionally presented with the option to step through the candidate path termini (e.g., by successive key presses, dragging a finger on a touch input screen, moving a scroll wheel, or other input method), and the selected terminus becomes the presented path terminus. Optionally, an initial path is determined for each of the multiple candidate path termini (and optionally accepted as being a member of a default accepted path list).

[0101] Upon first entry into block 311, in some embodiments, the lowest cost available route from route list 214 (based on route cost list 218) between those reaching the indicated route end is first selected for presentation to the user. Optionally, the presentation is highlighted, for example with a special coloring and / or thickness, to differentiate as active for the current decision. It should be noted that in some embodiments, the route from route list 214 is optionally the basis for what is shown, but the actual route shown is derived from the entries in route list 214, for example by application of active contours or dynamic "snake" algorithms as described herein.

[0102] In some embodiments, at block 312, the user determines whether the correct route (suitable for use as a "true" vascular route) is displayed. If yes, the flowchart optionally proceeds with the user issuing a confirming input (e.g., double-click, screen tap, key press, or other input) at block 316. The currently selected route is then added to the accepted route list 324, and the flowchart exits from block 112 of FIG. 1. Optionally, the accepted route continues to be displayed and is optionally indicated as deselected, for example by being drawn lighter with lower contrast, or by other visual indication. FIG. 4D shows a sequence of routes 407 and 408 that have been drawn to be members of the accepted route list 324. Due to the new position of the cursor 405, a corresponding new route option 409 is shown. This sequence continues in FIGS. 4E-4F where new routes 411 and 414 are created. As shown, each new route is drawn to extend to the root position 401. Instead, the display extends only backwards to the first contact with a previously accepted vascular path. Optionally (and optionally independent of the display method), paths are stored in the accepted paths list 324 as complete paths extending between the distal and proximal locations, as incremental additions to the vascular tree structure, and / or in other formats.

[0103] Alternatively, if the correct route is not displayed at block 312, the flowchart optionally continues at block 314 with an option selection event. In some embodiments, the option selection event includes a scroll wheel movement, a touch screen gesture (e.g., a sliding motion), a key press, or other user input event. Optionally, the selection is interpreted by the software as an indication that the current route display should move to another candidate (e.g., the next available candidate in cost order as defined in the route cost list 218). In a variation of this method, multiple vascular route candidates are presented simultaneously for a pair of endpoints, and the selection is used to indicate which candidate should be depicted as being actively selected. A potential advantage of this variation is that it may more quickly realize that no suitable candidate is available and / or allow for a simultaneous comparison of available options.

[0104] At block 316, in some embodiments, the user may determine that the pre-determined route in the route list 214 is not suitable for the currently displayed route end and proceed to block 320 to manually determine a new route, or in some embodiments, the flowchart returns to block 311 showing the newly selected route candidate.

[0105] With regard to iterating through blocks 311, 312 and 314, reference is now made to FIGS. 5A-5C, which illustrate diagrammatically the selection among alternative vascular route options, according to some exemplary embodiments of the present disclosure. FIG. 5A shows an angiographic image 500 illustrating a portion of a vascular tree 502 (in the example, the cardiovascular system) including a currently determined root location 501. In FIGS. 5B-5F, the contrast of the angiographic image 500 is reduced to make other elements more visible. In FIG. 5B, several pre-determined routes 507 are shown. A cursor 505 is shown hovering directly over a vascular location, but a suggested route 509 ends instead some distance away from the cursor on another vascular value. In FIG. 5C, the user indicates that a next candidate vascular route 511 should be shown, but this vascular route fails to indicate what the user intends to add to the accepted route list.

[0106] If it is found (per block 316) that the intended route is not available, the user optionally proceeds to block 320. For this block, reference is now made to Figs. 5D-5E, which illustrate, in accordance with some exemplary embodiments of the present disclosure, a method for manually determining vascular route options. Fig. 5 shows a selection made (formed) by successive position indications (e.g., clicks made at different positions of cursor 505) with line segments drawn between each indication. The user indicates positions generally along the intended vascular route. Optionally, root position 501 is interpreted as part of the position indication.

[0107] In Figure 5E, the user terminates the sequence of position indications (e.g., with a double click). In some embodiments, this results in the operation of block 322, where the position indications are used to generate a new route determination by way of a fitting between them, for example as described herein in connection with the active contour algorithm. The resulting vascular route is optionally added to the accepted routes list, and the flowchart of Figure 3A is exited at block 112.

[0108] In some embodiments, other methods of manually determining the vascular route are used, such as a variation of the Livewire technique, where segment positions are optionally live-drawn ("growing" to fill the cursor position from the previous anchor point) between clicks to determine waypoints, rather than all waypoints being determined first and then all at once.

[0109] Active contour method for blood vessel pathways In some embodiments of the present disclosure, an active contour method, also commonly referred to as the "snake" dynamics method (Kaas et al. 1987), is implemented. In this method, the behavior of the contour, c(s)=(x(s),y(s)), is governed by simulated intrinsic and extrinsic forces applied on the contour. These forces are guided by the minimization of an energy functional.

number

[0110] The internal energy term is:

number

[0111] External Energy Term E ext (c(s)) is implicitly defined by the force field.

number

[0112] In some embodiments, the GGVF field (Xu and Prince, 1998) on Frangi filtered grayscale images (Frangi et al., 1998) is calculated as the force field, although in general any force field can be used.

[0113] Active contour method for initial route determination When a vessel is first determined from the vessel path list 214, the initial input may be made from the string of pixels in the vessel skeleton segments that connect along the selected path. Optionally, the coordinates actually used in the path are relocated to be evenly spaced along the vessel path length (the center-to-center distance of the actual pixels themselves may be non-uniform due to alternating diagonal and cardinal motion). Then, as the active contour method is iterated, the vessel path position is pulled to a new position according to the various force and field terms used. Once the path has consolidated into a sufficiently stable configuration, the force terms are set to zero and the process stops.

[0114] Optionally, once a vessel has been determined by sparsely provided anchor points, e.g., as described in connection with Figures 5D-5E, the forces acting on the anchor points are set to zero (e.g., α(s), β(s) and γ(s) are set to zero) so that these points remain fixed. Points interpolated between them (which may be initially provided as straight lines, by spline interpolation or otherwise) are subjected to non-zero forces (e.g., α(s), β(s) and γ(s) are set to non-zero values) through multiple iterations of numerically minimizing the energy functionals of Equations 2 and 3, until a sufficiently stable configuration is reached.

[0115] Active contours for route editing Reference is now made to Fig. 3B, which is a schematic flow chart of a method for manually editing a vascular route, according to certain exemplary embodiments of the present disclosure. Fig. 3B illustrates an embodiment of the route editing operation of block 114 of Fig. 1. Reference is also made to Figs. 5E-5F, which are schematic illustrations of manual editing of vascular route options, according to certain exemplary embodiments of the present disclosure.

[0116] At block 340, in some embodiments, a user makes a location selection along the length s of the displayed vascular route (in some embodiments, a route from the accepted route list 324). In some embodiments, the location selection includes pressing a button with a cursor at some selected location, a gesture on a touch screen, or other input method. In a subsequent operation, the location selection is made to determine a drag point, such as drag point 519A in FIG. 5E.

[0117] In block 342, in some embodiments, the energy terms α(s), β(s) and / or γ(s) are set to nonzero only for s within a limited perimeter of the drag point. Optionally, the external force field of Equation 3 is determined to include attractors to the current cursor position, or to other positions determined by further user interface actions by the user dragging the drag point to a new position.

[0118] At block 344, in some embodiments, the energy functionals of Equation 2 and Equation 3 may be numerically minimized, resulting in the displayed vascular path (e.g., path 517 in Figures 5E-5F) being shifted around the location of the drag point.

[0119] At block 346, in some embodiments, the display of the vascular pathway 517 is updated.

[0120] At block 348, the system determines whether the drag operation should be terminated, for example, based on user input (e.g., a button release or end of a touchscreen gesture). If not, at block 350, in some embodiments, the user optionally drags the drag point to a new location, such as to location 519B in FIG. 5F. Alternatively, the edited path 517 is updated in the accepted paths list 324, and the flowchart ends (optionally exiting block 114 of FIG. 1). It should be appreciated that in some embodiments, the accepted paths list may additionally or alternatively be updated dynamically during the drag operation itself.

[0121] Determining Vascular Pathways from Multiple Views Reference is now made to FIG. 6, which illustrates a schematic diagram of a display 620 of routing decisions made simultaneously on a number of different vascular image views 400, 500, 600, 610, according to some exemplary embodiments of the present disclosure.

[0122] In some embodiments, vascular image views 400, 500, 600, 610 each show a view of the same vasculature (e.g., cardiovascular region) from different angles. Partially determined vascular pathways 620, 622, and 624 are shown for views 400, 500, and 600. In some embodiments, matches between vascular image features (matching image features showing the same anatomical location, optionally from different perspectives) are established at least in part on structures identified in one or more of the vascular skeleton graph 216, the branch list 210, the route list 214, and / or the accepted route list 324. In some embodiments, the accepted routes allow for differentiation of matching vascular features, optionally without detailed 3D information, e.g., based on branch number and / or location, root location 611, 601, 501, 401, and / or relative vessel length. Optionally, basic 3D information (e.g., specified viewing angles within about 45°, 90°, or other larger or smaller amounts) aids in the identification by correctly setting the general relative orientation and / or by identifying mirror images. In some embodiments, more complete 3D information can be utilized, for example, the positions of vascular centerlines reconstructed based on stereographic projections from two or more images into a general 3D frame of a reference (e.g., as described in International Publication WO2014 / 111930 to the applicant, filed on January 15, 2014, the contents of which are incorporated herein by reference in their entirety). Optionally, vascular paths generated for one image representation are transformed based on the 3D frame of the reconstructed reference such that they can be superimposed on one or more alternative image representations. In some embodiments, such transformations are performed on the vascular skeleton graph 216, the branch list 210, and / or the path list 214, for example, to aid in the validation of image processing results. For example, missing segment portions in the path and / or skeleton data from one image can be filled in from available data in the representation obtained from another image.Conversely, spurious segment portions in one image representation are arbitrarily identified, for example, based on their absence and / or lack of connection to root locations in the other image representation.

[0123] Reference is now made to FIG. 7, which is a schematic illustration of data structures 700 and software modules implemented in a system 702 for semi-automatic vascular pathway segmentation, according to certain exemplary embodiments of the present disclosure.

[0124] In some embodiments, blocks 206, 210, 214, and 218 include data structures generated and / or used by software modules of system 702. These correspond, for example, to correspondingly labeled blocks in Figure 2. Vascular images 701 are provided from an imaging device 750 that is optionally part of system 702. Optionally, system 702 is standalone or separate from imaging device 750, and images are provided, for example, via a remote network connection.

[0125] In some embodiments, block 703 includes a vascular skeletonizer configured to generate a skeletonized representation (skeleton graph) of vascular segments, for example as described in connection with block 206 of FIG. 2.

[0126] In some embodiments, block 705 includes a route option manager 705. In some embodiments, the route option manager includes software functionality for implementing the operations of blocks 208, 212, and / or 216 of FIG.

[0127] In some embodiments, the editing module 710 includes sub-modules that handle operations such as those described in connection with Figures 1, 3A and / or 3B. Optionally, the display options module 712 and / or the manual route determination module 716 implement block 112. Optionally, the route editing module 714 implements block 114.

[0128] In some embodiments, the system 702 additionally includes a display 760 and / or one or more input devices 770 .

[0129] As used herein in reference to a quantity or numerical value, the term "about" means "within a range of +10%."

[0130] The words "comprises," "comprising," "includes," "including," "having" and their conjunctions mean "including but not limited to."

[0131] The term "consisting of" means "including, but not limited to."

[0132] The term "consisting essentially of" means that a composition, method, or structure may contain additional ingredients, steps, and / or parts, but only if the additional ingredients, steps, and / or parts do not materially alter the basic and novel characteristics of the claimed composition, method, or structure.

[0133] As used herein, the singular forms "a," "an," and "the" include plural references unless the context clearly excludes otherwise. For example, the term "a compound" or "at least one compound" can include a plurality of compounds, including mixtures thereof.

[0134] The words "for example" and "exemplary" are used herein to mean "serving as an example, instance, or illustration." Any embodiment described as "example" or "exemplary" is not to be construed as preferred or advantageous over other embodiments and / or to exclude the incorporation of features of the other embodiments.

[0135] The word "optionally" is used herein to mean "provided in some embodiments, but not provided in other embodiments." Any particular embodiment of the present disclosure may include multiple "optional" features, so long as such features are not inconsistent.

[0136] The term "method" as used herein refers to methods, means, techniques, and procedures for accomplishing a given task, and includes, but is not limited to, methods, means techniques, and procedures that are known to those skilled in the art of chemistry, pharmacology, physiology, biochemistry, and medicine, or that can be readily developed from known methods, means, techniques, and procedures by those skilled in the art of chemistry, pharmacology, physiology, biochemistry, and medicine.

[0137] As used herein, the term "treating" includes negating, substantially inhibiting, slowing, or reversing the progression of a condition, ameliorating the clinical or cosmetic symptoms of a condition, or substantially preventing the appearance of clinical or cosmetic symptoms of a condition.

[0138] Throughout this application, embodiments of the present disclosure may be described with reference to a range format. It should be understood that the description in range format is merely for convenience and brevity and should not be construed as an inflexible limitation on the scope of the present disclosure. Thus, the description of a range should be construed as specifically disclosing all possible subranges, as well as the individual numerical values ​​within the range. For example, a description of a range such as "1 to 6" should be construed as specifically disclosing subranges such as "1 to 3", "1 to 4", "1 to 5", "2 to 4", "2 to 6", "3 to 6", etc., as well as the individual numerical values ​​contained within the range, e.g., 1, 2, 3, 4, 5, and 6. This applies regardless of the breadth of the range.

[0139] Whenever a numerical range is given herein (e.g., any pair of numbers linked by "10-15," "10 to 15," or these other such range designations), it is meant to include any number (fractional or integer) within the limits of the given range, including the limits of the range, unless the context clearly indicates otherwise. The phrases "range / in the range / between" a first given number and a second given number, and "range / in the range / between" a first given number and a second given number "to," "up to," "up to," or "between" (or other such range designation terminology) are used interchangeably herein and are meant to include the first and second numbers and all fractional and integer numbers therebetween.

[0140] While the example system, method, apparatus and / or computer program product has been described in conjunction with specific embodiments, it is evident that many alternatives, modifications and variations will be apparent to those skilled in the art. Accordingly, it is intended to embrace all such alternatives, modifications and variations that fall within the spirit and broad scope of the appended claims.

[0141] All publications, patents, and patent applications mentioned in this specification are herein incorporated by reference in their entirety to the same extent as if each individual publication, patent, or patent application was specifically and individually indicated to be incorporated by reference herein. In addition, citation or identification of any document in this specification should not be construed as an admission that such citation is available as prior art to the present disclosure. To the extent that section headings are used, they should not be construed as necessarily limiting.

[0142] It is recognized that some features of example systems, methods, apparatus and / or computer program products that are described for clarity in the context of separate embodiments may also be provided in combination in a single embodiment. Conversely, various features of systems, methods, apparatus and / or computer program products that are described for brevity in the context of a single embodiment may also be provided separately or in any suitable subcombination or as suitable in any other described embodiment of the present disclosure. Particular features described in the context of various embodiments should not be considered essential features of those embodiments, unless the embodiment is inoperable without those elements.

[0143] [Note] [Appendix 1] 1. A method for segmenting a vascular image into vascular paths for determining blood flow paths, comprising: The method comprises: receiving, into a processor, the vascular image; determining, via the processor, termination regions of first and second target vascular paths within the vascular image; segmenting, via the processor, the vascular image to identify locations of portions of a vessel within the vascular image; automatically generating, via the processor, a plurality of vascular pathway options from the identified portions of the blood vessel, each of which defines a potential vascular pathway extending between terminal regions of the first and second target vascular pathways; displaying, via the processor, the plurality of vascular pathway options registered to the vascular image, each displayed including a termination region of the first and second target vascular pathways, for selection by a user; and receiving, into the processor, a routing option selected by the user for routing blood flow; A method comprising:

[0144] [Appendix 2] the plurality of paths are automatically generated based on a first set of criteria; the route option selected by the user is selected based on a second set of criteria. The method according to claim 1,

[0145] [Appendix 3] and predetermining an order of selection for the vascular route options. said predetermining comprising ranking the vascular route options in order based on an assessment of the likelihood that each of the vascular route options corresponds to an actual route of blood flow in an imaged blood vessel in the vascular image. The method according to claim 1,

[0146] [Appendix 4] and determining in advance includes applying a cost function that assigns a numerical cost to one or more features associated with the vascular routing options. The method according to claim 3,

[0147] [Appendix 5] the cost function assigns a numerical cost based on centerline characteristics of a plurality of vascular segments from which the vascular route options are connected; The method according to claim 4,

[0148] [Appendix 6] the characteristics of the centerlines of the plurality of vascular segments include one or more of the group consisting of centerline orientation, centerline offset, and number of centerlines extending from a nodal region; The method according to claim 5, characterized in that

[0149] [Appendix 7] the cost function assigns a numerical cost based on features of the vascular image over which the vascular path option extends; The method according to claim 3,

[0150] [Appendix 8] The features of the blood vessel image include one or more of the group consisting of: continuity of blood vessel segment image intensity, continuity of blood vessel segment image width, and location of a relative change in blood vessel intensity relative to a nodal region, the nodal region having three or more blood vessel segments extending therefrom. 8. The method according to claim 7,

[0151] [Appendix 9] said predetermining including applying a cost function that assigns a numerical cost based on an estimated relative position of the vessel segment image in depth with respect to an axis extending perpendicular to a plane of the vessel image. The method according to claim 3,

[0152] [Appendix 10] and displaying the vascular route options includes presenting the vascular route options in a sequential order determined by a selection order. The method according to claim 3,

[0153] [Appendix 11] and wherein the displaying includes simultaneously showing a plurality of the vascular route options, the order of selection corresponding to the order in which the vascular route options are displayed as active for selection. The method according to claim 3,

[0154] [Appendix 12] Each vascular path option determines a vascular path that extends through an image region between the end regions of the first and second target vascular paths and terminates in a vascular region of the image that is closest to one of the end locations of the first and second target vascular paths. The method according to claim 1,

[0155] [Appendix 13] 1. A method for editing vascular paths to more accurately depict vessel segmentation in a vascular image, comprising: The method comprises: receiving, at a processor, an indication of a selected region along the segmentation of the blood vessel; determining, via the processor, an energy functional determined as a function of position along the segmentation of the blood vessel, the energy functional having non-zero regions set based on the position of the selected region; and via the processor, moving regions of the segmentation within the non-zero regions of the energy functional according to an energy minimization; A method comprising:

[0156] [Appendix 14] the energy functional value in the non-zero regions is set based on features of the underlying vascular image; 14. The method according to claim 13,

[0157] [Appendix 15] and setting the energy functional value in the non-zero region based on a user-controlled position indication movement. 14. The method according to claim 13,

[0158] [Appendix 16] the indication of the user controlled position includes the indication of the selection area. 16. The method according to claim 15,

[0159] [Appendix 17] 1. A user interface for semi-automatic segmentation of vascular pathways, comprising: The user interface includes: at least one interface module operable to present an automatically generated default vascular route extending between two target termination points; at least one interface module operable to present at least one additional automatically generated vascular pathway extending between the two target termination points; A user interface comprising:

[0160] [Appendix 18] and at least one interface module operatively configured to determine at least one way point and present an automatically generated vascular route extending between the two target end points via the at least one way point. 18. The user interface of claim 17,

[0161] [Appendix 19] and at least one interface module operable to modify the pre-determined vascular path by dragging the position of the vascular path to a new location. 19. The user interface according to claim 17 or 18.

Claims

1. 1. A method implemented by one or more processors, comprising: estimating characteristics of a plurality of vessel segments depicted in at least one vessel image, each vessel segment including a plurality of portions extending between two end points depicted in the vessel image; estimating a vascular path from at least a subset of the plurality of vascular segments depicted in the at least one vascular image; displaying an interactive user interface, the interactive user interface comprising: displaying a graphical representation of at least a portion of the estimated vascular segments forming the vascular pathway; and displaying a plurality of vascular route options for the vascular route based at least on the estimated direction of blood flow; The method includes the steps of: A method comprising:

2. the interactive user interface is configured to sequentially display the plurality of vascular routing options based on a ranking of the vascular routing options. The method of claim 1.

3. the interactive user interface is configured to respond to user input and cause the one or more processors to correct errors in the display of the graphical representation. The method of claim 1.

4. the interactive user interface is configured to respond to user input and cause the one or more processors to adjust at least one vessel segment of the graphical representation. The method of claim 1.

5. the estimated blood flow direction is based in part on a directional continuity criterion. The method of claim 1.

6. An apparatus for defining a vascular path from a vascular image, comprising: A processor; A memory storing non-transitory computer readable instructions that, when executed, cause the processor to: estimating characteristics of a plurality of vessel segments depicted in at least one vessel image, each vessel segment including a plurality of portions extending between two end points depicted in the vessel image; estimating a vascular path from at least a subset of the plurality of vascular segments depicted in the at least one vascular image; and Displaying an interactive user interface; and a memory for causing the The interactive user interface includes: displaying a graphical representation of at least a portion of the estimated vascular segments forming the vascular pathway; and displaying a plurality of vascular route options for the vascular route based at least on the estimated direction of blood flow; The apparatus is configured so as to

7. the interactive user interface is configured to sequentially display the plurality of vascular routing options based on a ranking of the vascular routing options.

7. The apparatus of claim 6.

8. the interactive user interface is configured to respond to user input and cause the processor to correct errors in the display of the graphical representation; 7. The apparatus of claim 6.

9. the interactive user interface is configured to respond to a user input and cause the processor to adjust at least one vessel segment of the graphical representation.

7. The apparatus of claim 6.

10. the estimated blood flow direction is based in part on a directional continuity criterion.

7. The apparatus of claim 6.

Images