Systems and methods for machine learning based sensor analysis and vascular tree segmentation

Abstract

A method for automatically identifying vascular features is described. In some embodiments, one or more machine learning (ML)-based vascular classifiers are used, whose results are combined with the results of at least one other vascular classifier to generate a final result. Potential advantages of this approach include the ability to combine certain strengths of ML classifiers with segmentation approaches based on more classical ("mathematical formula-based") methods. These strengths include, in particular, the identification of anatomically identified targets that are intermixed within images that show visually similar but anatomically distinct targets.

Description

[Technical field]

[0001] (CROSS REFERENCE TO RELATED APPLICATIONS) All applications for which a foreign or domestic priority claim is identified in the Application Data Sheet filed with this application are hereby incorporated by reference under 37 CFR 1.57.

[0002] This application claims priority to U.S. Provisional Patent Application Serial No. 63 / 308550, entitled "Vascular Tree Segmentation," filed February 10, 2022, the disclosure of which is incorporated herein by reference.

[0003] (Technical field) The present disclosure, in some embodiments thereof, relates to the field of vascular imaging, and more particularly, but not exclusively, to computer modeling and segmentation of blood vessels. [Background technology]

[0004] Arterial stenosis is one of the most serious forms of arterial disease. Its severity can be determined by estimating vascular geometric measurements or flow rates, usually through invasive procedures. However, by using images of blood vessels to create computer models of blood vessels, it is possible to determine the severity of arterial stenosis without the need for invasive procedures. Vascular imaging provides the characteristics of the vessel location required to generate computer models of the vessels. However, vascular imaging requires vessel segmentation and feature identification as a preliminary step to image-based measurements of vascular status.

[0005] Currently, many stages of vessel segmentation and feature identification can be performed using automated analysis. Automatically identifying vessel locations is a potential advantage because it reduces the time, effort, skill, and / or attention required by a human operator to fully manually identify these locations, even if the user must subsequently manually verify or correct the vascular tree.

[0006] Despite the ability to perform automated analysis, the relevant image features generated from vessel segmentation and feature identification often have low contrast and may be embedded in complex environments containing ambiguous geometries and extraneous features, making the output error-prone. [Brief description of the drawings]

[0007] Throughout the drawings, reference numbers are reused to indicate correspondence between referenced elements. The drawings are provided to illustrate embodiments of particular subject matter described herein and are not intended to limit the scope thereof.

[0008] [Figure 1] FIG. 1 is a block diagram of an optimal image determination system that outputs an optimal angiographic image based on an input image sequence. [Diagram 2] FIG. 2 is a schematic diagram of different algorithm components from which a vascular identifier may be selected, according to some embodiments of the present disclosure. [Figure 3A] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 3B] FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 3C] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 3D] FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 3E] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 3F] FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 3G] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 3H] FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 4] FIG. 13 illustrates superimposed two-pass ML-based vessel identifier output according to some embodiments of the present disclosure. [Figure 5A] FIG. 13 is a diagram that illustrates a schematic representation of a method for vascular identification and repair using a mixed-path and masked ML-based vascular classifier, according to some embodiments of the present disclosure. [Figure 5B] FIG. 13 is a diagram that illustrates a schematic representation of a method for vascular identification and repair using a mixed-path and masked ML-based vascular classifier, according to some embodiments of the present disclosure. [Figure 6A] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 6B] FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 6C] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 6D] FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 6E] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 6F] FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 6G] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 6H]FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 6J] FIG. 13 illustrates superimposed two-pass ML-based vessel identifier output according to some embodiments of the present disclosure. [Figure 6K] 6A, 6B, 6D, 6F, 6H, and 6J show segmented results generated based on the use of various correct and partially erroneous information obtained from the ML-based vascular identifier output shown in FIGS. 6B, 6D, 6F, 6H, and 6J, in accordance with some embodiments of the present disclosure. [Figure 7A] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 7B] FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 7C] FIG. 1 illustrates an X-ray angiogram image, according to some embodiments of the present disclosure. [Figure 7D] FIG. 13 illustrates an ML-based vessel identifier output for different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. [Figure 7E] FIG. 13 illustrates a path-based ML-based vessel identifier output overlaid on the images of FIGS. 7A-7C, according to some embodiments of the present disclosure. [Figure 7F] 7A, 7B, and 7E show segmented results generated based on the use of various correct and partially erroneous information obtained from the ML-based vascular identifier output shown in FIGS. 7B, 7D, and 7E and modified by further constraints, in accordance with some embodiments of the present disclosure. [Figure 8A] FIG. 13 is a schematic diagram illustrating the exchange of vessel direction information from a pass-type ML output to a mask-type ML-based vessel identifier output according to some embodiments of the present disclosure. [Figure 8B]FIG. 13 is a schematic diagram illustrating the exchange of vessel direction information from a pass-type ML output to a mask-type ML-based vessel identifier output according to some embodiments of the present disclosure. [Figure 8C] FIG. 13 is a schematic diagram illustrating the exchange of vessel direction information from a pass-type ML output to a mask-type ML-based vessel identifier output according to some embodiments of the present disclosure. [Figure 8D] FIG. 13 is a schematic diagram illustrating the exchange of vessel direction information from a pass-type ML output to a mask-type ML-based vessel identifier output according to some embodiments of the present disclosure. [Figure 8E] FIG. 13 is a schematic diagram illustrating the exchange of vessel direction information from a pass-type ML output to a mask-type ML-based vessel identifier output according to some embodiments of the present disclosure. [Figure 9A] 1A-1C are schematic diagrams illustrating a method for performing vessel subtraction between mask vessel identifier outputs in accordance with some embodiments of the present disclosure. [Figure 9B] 1A-1C are schematic diagrams illustrating a method for performing vessel subtraction between mask vessel identifier outputs in accordance with some embodiments of the present disclosure. [Figure 9C] 1A-1C are schematic diagrams illustrating a method for performing vessel subtraction between mask vessel identifier outputs in accordance with some embodiments of the present disclosure. [Figure 9D] 1A-1C are schematic diagrams illustrating a method for performing vessel subtraction between mask vessel identifier outputs in accordance with some embodiments of the present disclosure. [Figure 9E] 1A-1C are schematic diagrams illustrating a method for performing vessel subtraction between mask vessel identifier outputs in accordance with some embodiments of the present disclosure. [Figure 9F] 1A-1C are schematic diagrams illustrating a method for performing vessel subtraction between mask vessel identifier outputs in accordance with some embodiments of the present disclosure. [Figure 9G] 1A-1C are schematic diagrams illustrating a method for performing vessel subtraction between mask vessel identifier outputs in accordance with some embodiments of the present disclosure. [Figure 10A] FIG. 1 is a block diagram illustrating an example of an optimal image determination system for filtering angiographic images according to cardiac phase. [Figure 10B]FIG. 1 is a block diagram illustrating an example of an optimal image determination system that performs an additional filtering process on a mask score associated with a segmentation mask. [Figure 10C] FIG. 1 is a block diagram illustrating an example of an optimal image determination system for generating optimal angiographic images. [Figure 11] 1 is a flow chart illustrating an example of a process for determining an optimal angiographic image. [Figure 12] 1 is a flow chart illustrating an example of a process for selecting an optimal angiographic image from an image sequence based on a contrast score. [Figure 13] FIG. 1 illustrates an example of a system that can be used to implement the techniques described herein. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] introduction The present application describes techniques for selecting vascular images from a sequence of vascular images that provide, among other things, improved contrast, improved visibility of vessels, improved image quality, etc. As described below, the selected vascular images can be used for further analysis, such as forming a three-dimensional model of at least a portion of the patient's heart. The selected vascular images can also be used as part of an automated process or workflow that allows a user to identify vessels in the vascular images or to adjust the automated determination of vessels in the vascular images. The vascular images described herein may represent angiographic images in some embodiments, where each sequence of vascular images represents a portion of the patient's heart from a particular perspective. As can be seen, combining vascular images from different perspectives can generate a three-dimensional view of the portion. Currently, such three-dimensional views are prone to inaccuracies, with the vascular images being adversely affected by motion errors, differences in cardiac phase during image acquisition, etc. The present disclosure describes techniques for optimizing or enhancing the selection of a subset of these vascular images to reduce such inaccuracies.

[0010] As described below, the system described herein may implement one or more machine learning models, optionally in combination with classical computer vision techniques, to determine an optimal image from among a number of input vascular images. The system may be, for example, the optimal image determination system 100 described below. For example, the machine learning model may include a neural network (e.g., a deep learning model, a convolutional neural network, etc.). In this example, the system may calculate a forward pass through the machine learning model and generate an output that is utilized in the decision. In some embodiments, the neural network may output segmentation masks for at least some of the received vascular images. These segmentation masks segment the vessels depicted in the vascular images.

[0011] The system can analyze the segmentation mask described above and, for example, determine a size or length score associated with the depicted vessels. Using these scores, the system can select an upper threshold number of images. The system can analyze a threshold number of images to identify the optimal image. For example, the system may calculate a contrast score indicating the contrast or other image quality measure of the vessels depicted in the image. In some embodiments, the optimal image can represent the image with the highest score or combination of scores. This automated selection of the optimal image can improve the accuracy of resulting downstream workflows, such as three-dimensional model generation, while reducing the need for and time for manual adjustments by a human operator after analysis.

[0012] Introduction – Background Angiographic images are often used to provide a detailed view of a patient's heart. These images can be obtained, for example, by injecting a radiopaque contrast agent into the patient's blood vessels and obtaining X-ray images (e.g., by fluoroscopy). Although these images provide a detailed view, because of the use of X-rays, medical professionals may prefer to limit the scope of angiographic image acquisition. Therefore, it is paramount that technologies that accurately leverage existing angiographic images be employed in downstream processes.

[0013] An example of a downstream process includes generating a three-dimensional model, or vascular tree, of a portion of a patient's heart. For example, angiographic images of different views of the portion may be obtained. In this example, the different views may depict different views of the blood vessels. As known to those skilled in the art, these vessels may be correlated to identify unique vessels depicted in the diagram. For example, automated or semi-automated techniques may be utilized to enable such correlation. The views may be combined to generate a three-dimensional view of these vessels. The views may also be combined to provide an easy-to-understand schematic representation of the vascular tree showing vessels upstream and / or downstream of one another.

[0014] Typically, an image sequence of angiographic images is acquired while an imaging system (e.g., a C-arm) is aimed at a portion of the patient's heart. For example, 5, 10, 20, etc. angiographic images are acquired. As can be appreciated, these angiographic images may differ in their usefulness with respect to the downstream processes described above. For example, a subset of angiographic images may include errors due to motion or shaking of the imaging system. As another example, a subset may have errors related to contrast agents. As another example, the angiographic images may depict portions of the heart when the heart is in different cardiac phases. In this example, certain cardiac phases may be associated with reduced vessel detail, size, etc. in the resulting angiographic images.

[0015] Additionally, errors and ambiguities may relate to: The use of angiographic images, which are two-dimensional projections of a complex three-dimensional vasculature, gives vascular connections that do not actually exist a false appearance in a single image. Vessels can overlap and obscure one another. Contrast agents used to reveal the vasculature may incompletely, unevenly, weakly, and / or sporadically fill the blood vessels of a subject. Images may contain indications of other body structures and sometimes obvious artifacts, resulting in image "noise" that may interfere with identification of the target of interest. There may be technical and / or safety limitations in image resolution and exposure to radiant energy which may, for example, limit image detail and / or contrast. There is also the problem of choosing which 3D shapes of blood vessels to reconstruct, as cardiac vessels move with the heartbeat and / or breathing, and the patient himself may also move. Typically, a user of the software tool views the images in an image sequence and selects the image that appears to best depict the blood vessel. For example, the user can zoom in on the image to identify obvious potential errors in the image. Some software tools may be able to use limited automated techniques to uncover potential errors. For example, classical computer vision techniques may be used to detect motion artifacts.

[0016] In contrast, the techniques of this disclosure leverage machine learning models that are specifically trained to deal with such errors and ambiguities.

[0017] Introduction - Choosing the Best Image The system includes one or more machine learning models, such as convolutional neural networks, that the system uses to output a particular angiographic image from an image sequence for use in downstream processes, the particular angiographic image being referred to herein as an optimal angiographic image or image frame.

[0018] For example, the system receives an image sequence from an angiography imaging tool or system (e.g., a C-arm). In this example, the imaging tool or system may be rotated such that the angiography images depict or include a portion of the patient's heart from a particular viewpoint. Thus, the image sequence may include angiography images taken at different times from a particular viewpoint.

[0019] A patient's heart beats and transitions between cardiac phases. Thus, the angiographic images in an image sequence may depict a portion of a patient's heart in various cardiac phases. For example, a first image may depict the heart in systole. As another example, a second image may depict the heart in diastole. As known to those skilled in the art, the second image may provide a clearer or more advantageous view of the blood vessels of the portion of the heart. For example, the portion may be depicted in a more relaxed state, such that it appears longer, in the second image, as compared to the first image.

[0020] As described below, the system may utilize machine learning techniques to identify images in an image sequence depicting the heart at a particular cardiac phase. In some embodiments, the particular cardiac phase may be the end-diastolic phase. The machine learning techniques may include a convolutional neural network trained to label or otherwise indicate values ​​that depict images depicting the heart at a particular cardiac phase. Although a convolutional neural network is described, it should be apparent that other machine learning models may be used. For example, fully connected networks, recurrent neural networks, attention-based networks, etc. may be used.

[0021] The system then uses machine learning techniques, or optionally classical computer vision techniques (such as applying a Franzi filter), to output a segmentation mask of the identified image. For example, the segmentation mask may have pixel values ​​assigned based on whether the pixel forms part of a blood vessel. In this example, the segmentation mask may include binary colors (such as black and white) with a first color indicating pixels that do not form part of a blood vessel and, as can be appreciated, a second color indicating pixels that form part of a blood vessel. Although binary colors are described, each pixel may be assigned a value or likelihood that indicates that the pixel forms part of a blood vessel.

[0022] The segmentation mask is then analyzed to identify a size or length metric (hereinafter referred to as a "mask score") associated with the vessel. For example, the mask score may indicate a length associated with the vessel. In this example, the length may indicate a length associated with a centerline from a first end of the vessel to a second end of the vessel. As another example, the mask score may indicate a fill score (e.g., area) associated with the vessel. In this example, the fill score indicates the number of pixels that form the vessel or the estimated area enclosed by the vessel. The mask score may be used to remove images in which the contrast agent fills the patient's vessels incompletely, unevenly, weakly, and / or sporadically. For example, these images may include segmentation masks that include vessels that appear as disconnected segments or vessels that are reduced in length or size. A threshold number of segmentation masks may be maintained, e.g., 3, 5, 7, 9, 10, 12, etc., as having the largest mask scores.

[0023] The system may then analyze the angiographic images used to generate the threshold number for the segmentation mask. For example, the system may determine a quality or clarity score for the angiographic images. The quality or clarity score is described herein as a contrast score and is used to indicate the image with the highest contrast. Although contrast scores are described herein, additional quality or clarity scores may be used and are within the scope of the disclosed technology. For example, scores associated with one or more of sharpness, focus, motion artifacts, image artifacts, etc. may be used. An optimal image may then be selected from the analyzed angiographic images based on the quality or clarity score.

[0024] As shown in Figure 1, the optimal image may be presented to the end user in a user interface. In some embodiments, the end user may view the optimal image and use it for downstream processes. For example, the end user may mark or otherwise identify blood vessels in the optimal image. The end user may also select another image from the image sequence.

[0025] In some embodiments, different machine learning models may be used based on whether the image sequence depicts the left or right side of the patient's heart, as described below. For example, the left image may depict the left anterior descending artery (LAD) and the circumflex artery (LCX), while the right image may depict the right anterior descending artery (LAD) and the left circumflex artery (LCX). Meanwhile, the right image may depict the right coronary artery (RCA). In some embodiments, a machine learning model (e.g., a convolutional neural network) may be used to indicate whether the image sequence depicts the left or right side. For example, the machine learning model may output a classification.

[0026] Thus, the disclosed technology addresses technical problems and provides technical solutions to these problems. For example, intelligent selection of angiographic images can avoid the need for unnecessary increases in image resolution and radiation energy exposure during X-ray imaging. As another example, machine learning models can be used to accurately classify and determine information to ensure optimal image selection from an image sequence. Without such technology, information derived from the resulting 3D model of the heart and angiographic images may contain errors and inaccuracies, which may adversely affect patient outcomes.

[0027] In this disclosure, the term "ML-based vascular identifier" refers to the output of machine learning. The term "vascular identifier" more generally encompasses both ML-based vascular identifiers and computation-based methods (e.g., formula-based methods such as classical computer vision techniques) that identify vascular regions in an image. Exemplary classical computer vision techniques include kernel filters (such as skeletonization filters and Gaussian filters), affine transformations (such as combinations of translation, scaling, rotation, inversion, and / or shear), bit masks, component analysis techniques (such as principal component analysis), and domain transformations (such as Fourier transforms between frequency and spatial domains). In particular, some embodiments of the present disclosure use a version of a filter type known in the art as a Frangi filter to help detect the generally tubular structures that make up the vasculature.

[0028] In this application, by way of example, Figures 1 and 10A-12 may relate to techniques for determining an optimal image frame from an image sequence. Figures 2-9G may relate to techniques for determining vessels in a vascular image based on a segmentation mask and / or addressing errors in determining vessels and / or segmentation masks. In some embodiments, the optimal image may be analyzed via the techniques described in Figures 2-9G to accurately identify vessels associated with a vascular tree or artery (e.g., left artery or right artery), address errors, etc. Thus, the systems described herein may determine an optimal image and analyze the image via, for example, a segmentation mask or other ML-based vessel identifier output as shown in Figures 2-9G.

[0029] These and other features are now described in detail.

[0030] Block Diagram FIG. 1 illustrates a block diagram of a system 100 for determining optimal images. The system 102 may represent, for example, a system of one or more computers or processors implementing the techniques described herein. In some embodiments, the system may be in communication with an imaging system or tool that acquires vascular images of a patient. As described herein, the vascular images may be angiographic images. In some embodiments, the vascular images may be computed tomography images or scans.

[0031] In the illustrated embodiment, the optimal image determination system 100 receives an image sequence 102 including angiographic images 104A-104N. The image sequence 102 may depict a portion of a patient's heart from an angle or perspective. In some embodiments, the angiographic images 104A-104N are obtained by user input. For example, a user may input a DICOM file including a series of angiographic images, or a video from which angiographic images can be extracted. The images 104A-104N may be captured at a particular frequency (e.g., 5 Hz, 7 Hz, 10 Hz, 30 Hz, etc.) over a period of time, resulting in a potentially large number of images.

[0032] Images 104A-104N may be taken during periods of vascular contrast injection and washout. As already explained, ambiguities in vascular structure may arise due to vessel segments crossing and / or approaching each other within a single image. Because the heart is hollow and x-rays image the entire heart, some of these ambiguities arise from vessels that are actually in completely different 3D planes (e.g., on opposite sides of the heart). As a result, vessels may show different patterns of movement, inconsistent with actually being in the same tissue bed, even if they appear to be close to each other in a static 2D plane. This inconsistency may be seen as potential information regarding vascular connectivity. Another form of temporal information arises from vascular filling dynamics. Vessels that are connected to each other should be highly correlated in their vessel filling (darkening) and vessel emptying (brightening) dynamics when vascular contrast is injected and when the contrast is washed out. Vessels that are not connected may be less correlated in their vessel filling dynamics, even if they appear to be crossing on the image.

[0033] Given these ambiguities, a particular angiographic image in the sequence 102 may be able to more clearly and accurately depict the blood vessels in the patient's heart. Thus, the optimal image determination system 100 may determine an optimal image 110 from the image sequence 102 that represents the best image from a particular angle or perspective. As described below, for example, with respect to at least FIGS. 10A-12, the optimal image may be identified using machine learning techniques, optionally in combination with classical computer vision techniques. For example, the optimal image may include an optimal or overall best combination of features, such as the size of the depicted blood vessels, image clarity, and blood vessel connectivity. With respect to connectivity, the optimal image may depict blood vessels as being substantially continuous without separation.

[0034] The optimal image 110 may be used by an end user as medical data for downstream processes, etc. For example, the system 100 may identify the locations or boundaries of blood vessels in the optimal image 110. In this example, the end user may adjust these locations or boundaries and, optionally, identify the locations or boundaries of other blood vessels in the image 110. As another example, the end user may obtain multiple (e.g., 2, 3, 5, etc.) optimal images from different image sequences. In this example, the end user may generate a three-dimensional model or vascular tree. Thus, the optimal images may represent the images from the respective image sequences that are determined to be most advantageous for use in these downstream processes.

[0035] In FIG. 1, a user interface 120 is shown that includes an optimal image 110. In some embodiments, the user interface 120 may be generated by the system 100. For example, the system 100 may execute software that causes the presentation of the user interface 120. The user interface 120 may be presented by a user device of an end user. For example, the system 100 may provide information for inclusion in the user interface 120 that is rendered by the user device. In this example, the system 100 may be in wired or wireless communication with the user device. As an example, the system 100 may be associated with a web application.

[0036] In some embodiments, the user interface 120 may include a button or other selectable object 122 to cause a different optimal image to be selected. For example, an end user may select the button 122 to view other angiographic images in the image sequence 102. In this example, the user interface 120 may be updated to display at least some of these angiographic images. Thus, the end user may override the optimal image 110 to select a different optimal image. In some embodiments, such an override may be used to update the machine learning models described herein.

[0037] The user interface 120 can be used, for example, to identify an image sequence for analysis. For example, an end user can provide user input that causes the user interface 120 to respond to a selection of an image sequence. Once selected, the selection of the image sequence can trigger the system 100 to determine an optimal image, as described herein. The optimal image may be displayed or included in the user interface 120, as shown in FIG. 1. In some embodiments, an animation or movie may be played in the user interface 120, with an animation or movie being formed based on the image sequence. The optimal image may be identified and, for example, highlighted, text may be displayed adjacent to it, or the animation or movie may be paused for a threshold amount of time on the optimal image during playback. In this manner, the end user may view the optimal image within the context of the image sequence. As previously described, the end user may select a different optimal image. For example, the end user may operate the button 122. As another example, the end user may select an image in the middle of an animation or movie of the image sequence.

[0038] Other examples of image locations or image portions that may be included in downstream processing include, for example, an image location being a portion of a blood vessel (a "vascular portion"), an image location being a portion of a specific (anatomically identified) blood vessel, an image location being a more specific part of a path definition that extends along a blood vessel portion (e.g., a general or anatomically identified vascular centerline), an image location being a "root" location of a vascular tree (e.g., the ostium of a coronary artery), and / or an image or image portion being a suitable target for another image processing methodology.

[0039] From an image input (e.g., an image input showing identifiable structural features of a cardiac vessel), the vessel identifier may generate a data structure in which there is some correspondence with locations in the image, such as, for example, a 1:1 correspondence between identifications and pixels (other types of scaled, binned, or otherwise transformed correspondences are not excluded). The data structure can be any form of representation suitable for describing the identification results, such as, for example, a scalar, array, matrix, linked list, or other form of representation. More specifically, the data structure is a mask, e.g., a binary image with one pixel for each pixel of the input image, where the values ​​of the mask pixels are set according to the identification results of the corresponding input image pixel. In some embodiments, the data structure is a list of locations in the input image that have (or do not have) a particular identification information (e.g., information identified as representing a part of a particular vessel centerline). The image locations can be expressed, for example, as absolute locations or relative locations to other locations. For example, a path in the image can be expressed as either a binary image mask or a list of image locations. The vessel identifier can optionally generate a data structure that performs a classification of the image as a whole. For example, the data structure may classify images according to their likelihood of being acquired near a particular phase of the cardiac cycle.

[0040] Type of vessel identifier / Output of vessel identifier 2-9G illustrate example techniques that may be implemented using optimal image determination system 100 described herein. As described below, system 100 may generate an output (e.g., an ML-based vascular identifier, a vascular identifier, etc.) that may be included in a user interface (e.g., user interface 120). In some embodiments, the ML-based vascular identifier may include a segmentation mask generated from the angiogram image. The ML-based vascular identifier, in some embodiments, may also represent an output from a machine learning model, such as a score, value, likelihood, classification, etc.

[0041] Reference is now made to Figure 2, which is a schematic diagram of different algorithm components from which a vascular identifier may be selected in accordance with some embodiments of the present disclosure. The diagram is in the form of a Venn diagram, with regions of the diagram ("type regions") characterized by which labeled closed boundary regions they fall within. Type regions represent groups of algorithm components indicated by labels and with properties as further described herein.

[0042] Type fields 201, 202 represent general types of ML-based and formula-based vessel identifiers, respectively, which in either case should be understood to be suitable vessel identifiers that can be used to identify vessels from vessel images. More specifically, in some embodiments, the vessel identifiers are suitable for use in identifying contrast-filled cardiac vessels in x-ray angiographic images of a living heart.

[0043] In either type region 201, 202, more specifically, a type region 203 of mask-based vessel identifiers and a type region 204 of path-based vessel identifiers may be distinguished (the representation of type regions 203, 204 is split between type regions 201, 202 to reduce visual clutter). A mask-based vessel identifier is understood here to be a vessel identifier that provides an output representing the likelihood (optionally threshold likelihood) of whether a particular image pixel is within a well-defined vessel target or not. Characteristics of the output of a mask-like vessel identifier include: Since there is no directionality, for example, the output of the vessel identifier itself does not indicate the direction of the vessel. Being "blobby", i.e. having widths of any size in any direction from any particular point in the mask.

[0044] In contrast, the output from a path-based vessel identifier is directional, i.e., the identified locations are in the order defined by the output of the vessel identifier. Furthermore, paths are skeletal (not blobby), i.e., there is no representation of width per se.

[0045] It should be noted that the outputs of the masked and path-type vessel identifiers can be interconverted by various processing methods and have the opposite type of directionality / blockiness. For example, blobs can be skeletonized or different widths can be assigned to positions along the longitudinal extent of the path. It is the form of the initial classification that is taken into account for the purpose of type and region assignment here. Note that a vessel identifier can in principle combine both the directionality of the path-type output and any width of the masked output, even within the initial classification (i.e., these two properties are not logically contradictory). It is also not excluded that there exist vessel identifiers that do not share any of the distinguishing properties of the path-type and mask-type vessel identifiers. However, unless otherwise stated, the vessel identifiers used in the examples can be reasonably understood to belong to one of the two types mentioned above.

[0046] Corresponding to the earlier classification types, there is a fundamental difference in how masked and path-based vascular identifiers use the information available to them, which is believed to reveal differences in the underlying models. In particular, path-based vascular identifiers operate on the model assumption that there exists a path, an ordered set of locations that spans between a start location and an end location. From the further knowledge or assumption that two non-adjacent locations are on the path, path-based vascular identifiers rely on the existence of a third location between those locations. Models for masked vascular identifiers may operate on model assumptions that evaluate pixel properties and patterns without relying on such global properties.

[0047] In embodiments of the present disclosure, it is not prohibited for a vascular identifier to use both path-like and pattern-like information (for example) to generate an output, even if the output itself is strictly path-type or mask-type. Indeed, particularly for ML-based vascular identifiers, it may be difficult, and perhaps impossible, to constrain what type of information is used. Even then, however, it will be seen that the use of path-type and mask-type training data tends to affect the training process that generates the ML-based vascular identifier, such that path-like or pattern-like information is emphasized more or less.

[0048] Accordingly, some embodiments of the present disclosure provide for the use of a mask-based vascular identifier. The mask-based vascular identifier may be an ML-based vascular identifier. In some embodiments, the vascular identifier is more specifically characterized by having a classification-based output layer (e.g., as a member of type region 206), as also described above. The mask-based ML-based vascular identifier is trained using mask data that identifies which portions of a vascular image should be considered vascular portions.

[0049] In some embodiments, the masked vessel identifier is a formula-based vessel identifier, such as a Frangi filter-based vessel identifier 207 (A.F. Frangi et al., "Multiscale vessel-enhancing filtering," Medical Image Computing and Computer-Aided Intervention MICCAI'98 Computer Science Lecture Notes 1496 / 1998: 130). Variations of this method have also been described in recent literature. More common examples of formula-based masked vessel identifiers include edge detectors and threshold detectors. Frangi filter-based vessel identifiers (and variations thereof) are characterized as "tube detectors" in that they are designed to enhance image features that correspond to the image appearance of tubes that are characteristic of the vasculature, such as having a longer longitudinal orientation, having a more or less constant or slowly varying diameter, cross-sectional brightness profiles, and / or other characteristics.

[0050] Additionally or alternatively, in some embodiments of the present disclosure, a path-type vessel identifier is used. The path-type vessel identifier may be an ML-based vessel identifier. In some embodiments, the vessel identifier is more specifically characterized by having a recurrent output layer (e.g., as a member of type group 205), as described above. A path-type ML-based vessel identifier is trained using path data, where the path data identifies a location along a vessel portion. In some embodiments, the identified location is a vessel centerline. The centerline is optionally and advantageously identified as representing the geometric center of the vessel, although there is no general restriction requiring this. In some embodiments, additionally or alternatively, the identified path location is an edge of the vessel.

[0051] In some embodiments, the path-type vessel identifier is a mathematical formula-based vessel identifier 208. Such a path-type vessel identifier may, for example, move in one or two directions away from a seed location along the minimum gradient of the local brightness values ​​of the image. If the seed location is within a vessel (e.g., a dark vessel against a light background), the movement direction may tend to be restricted to be approximately parallel to the direction of the vessel itself, as long as the vessel is relatively narrow (making the gradient steep) compared to its length (along which the gradient becomes gentler). The path tends to be towards the vessel center, since the overall intensity level is lower at the vessel center, and therefore the gradient may also be lower. Refinement of this general idea may include different methods of seed location selection, solving stopping conditions, and / or bridging gaps in vessel crossing and / or vessel brightness.

[0052] Apart from its usefulness in describing some embodiments of the present disclosure, the distinction between path-based and mask-based vascular identifiers is relevant to the problem of vascular identification because, more generally, it can be noted that even if two vascular identifiers are both designed to solve the same problem of "identifying vessels" in an image, differences in their underlying models lead to somewhat different solutions.

[0053] In the case of the two ML-based vascular identifiers, this is true even if the vascular identifiers were trained using the same basic foundation images. This is important in practice because a given vascular identifier can make a certain number of mistakes: identifying vessels where none exist, or failing to identify vessels when they should have been identified. Furthermore, the pair of path-based and mask-based vascular identifiers are examples of two different types of vascular identifiers that are prone to different mistakes. Understanding these differences can translate into a solution where multiple vascular identifiers can be used complementary to achieve more accurate results than any individual vascular identifier (or type of vascular identifier) ​​alone.

[0054] The principle of complementarity may be generally applied to other types of distinctions, including those shown in FIG. 2. For example, in some embodiments, one or more ML-based vascular identifiers are trained to identify selective anatomical targets. The selective anatomical targets consist, for example, not of general vessels (as in the case of the generic / "branch vessel" selective ML-based vascular identifier 210), but more specifically of specific anatomically defined vascular targets, such as the LAD, LCX, RCA, and / or their subtree vessels (these are examples of targets that may be designated for the anatomically targeted selective ML-based vascular identifier 212). It should be emphasized that there are no specific technical limitations that prevent the training of ML-based vascular identifiers of general type 210, or of somewhat more selective types, such as general cardiac vessels. However, the inventors have discovered that training ML-based vascular identifiers on an anatomically restricted range of vessels has potential advantages in terms of the accuracy of the identification and the amount of training data required to reach an acceptable level of accuracy. Further advantages may be obtained when images taken from different angles are used to reconstruct the 3D location of the vessels. The problem of matching vessel coordinates may be simplified and / or less error-prone if it were clear in each image which vessel was the LAD or LCX.

[0055] Another potential advantage of such an approach is that whatever the vessel-type specific ML-based vessel identifier identifies as a vessel, will by this fact also be identified as belonging to a particular anatomically defined vessel type. Identification could be relevant, for example, to the analysis of vascular lesions, where the severity of the lesion is determined in part based on which anatomically identified parts of the vasculature are lesioned.

[0056] Additionally, the ability of machine learning techniques to create anatomically selective vascular identifiers with appropriate selection of training data is a potential advantage over using formula-based vascular identifiers. In the case of formula-based vascular identifiers, the parameters available for tuning are not necessarily suitable to make the type of anatomical selection required. For example, flange-based algorithms typically have adjustable free parameters to tune which vessel diameters are detected. However, cardiac vessels of interest such as the LAD and LCX are not particularly distinguishable by a single width, or range of widths, as vessels tend not only to be similar in width but also to vary in width with length. While with sufficient exploration it may be possible to develop discriminatory vascular identifiers, formula-based vascular identifiers do not expose existing “knobs” (parameters) with the same flexibility for tuning training data available in the generation of ML-based vascular identifiers.

[0057] Anatomically selective ML-based vascular identifier Masked ML-based vascular identifier 3A-3H, there are shown x-ray angiogram images (depicted in FIGS. 3A, 3C, 3E, and 3G, respectively) along with ML-based vessel identifier outputs (FIGS. 3B, 3D, 3F, 3H) of different mask types corresponding to different anatomical vessel types, in accordance with some embodiments of the present disclosure. FIG. 3B shows the LAD mask, FIG. 3D shows the LCX mask, FIG. 3F shows the mask including the LAD subtree (LAD and its diagonal or branch), and FIG. 3H shows the mask including the LCX subtree (LCX and its border or branch).

[0058] 3B, 3D, 3F, and 3H were each generated from the same image input using a different ML-based vascular identifier, each trained using a training dataset specific to the identified vessel type. In some embodiments, the training dataset for a particular ML-based vascular identifier consists of a large set of images (e.g., more than 1000 images, more than 10000 images, or another number of images), for each of which a mask is defined that identifies the target anatomically identified vessel type.

[0059] From this set of images, it is easy to see that each of the LCX and LAD can be distinguished from the rest of their respective subtree vessels by observing which vessel territories are only output in the whole subtree vessel identifier and which vessel territories are also included in the output of the corresponding single vessel ML-based vessel identifier. However, the images may require additional algorithmic processing to achieve this distinction. Thus, by applying four different ML-based vessel identifiers to the images, four different vessel types can be identified in a single image.

[0060] It should be noted that the identifications shown in this set of examples are relatively error-free, but note, for example, that Figure 3F shows a small number of disconnected blobs, at least some of which would appear to be better identified as part of an LCX subtree. In the following examples, we discuss different types of detection errors in the ML-based vessel identifier outputs and how they can be addressed by combining vessel identifier outputs together.

[0061] Path-based ML-based vascular identifier 4, there is shown a superimposed two-path ML-based vascular identifier output 401, 402 according to some embodiments of the present disclosure. Path 401 extends along the LCX and path 402 extends along the LAD.

[0062] In this case, the ML-based vessel classifier used was regression-based, and the training data used was paths rather than blob-like masks. Regression-based training proceeds, in part, by "teaching" the ML-based vessel classifier to adjust the position of the output path to reduce the deviation of the position from the paths in the training set. This is a type of regression problem. The trained vessel classifier is expected to produce useful vessel identifications when equipped with image data that was not used as part of the training dataset. The path has an order of positions along the output path, including the two path ends. The model might output the path as a sequence of X and Y coordinates.

[0063] In some embodiments, more specifically, to apply the path-based ML-based vascular identifier, each of the 50 coordinates along the seed starting path is converted into a profile image, and the profile images are stacked to create a 2D image. Each profile image is composed of pixels perpendicular to the vessel direction and passing through the selected coordinate. In the profile images, the true vessel path generally extends along the seed starting path, but is "wavy". The ML-based vessel identifier works by laterally shifting the seed path coordinates to positions that are considered likely to actually be on the vessel path (optionally constrained somewhat by the lateral placement of nearby points to maintain at least some degree of continuity). Optionally, this is performed on an interpolated image, generating, for example, 200 or more (e.g., 210) interpolated points. If the positions of the points along the lateral lines in the original 2D image are known, the laterally shifted positions can be mapped back to 2D image coordinates to obtain the 2D shape of the vessel path. Optionally, the remapping uses fewer points than are available in the interpolated intermediate image. For example, remapping might involve going back to 50 selected anchor points and drawing the vascular pathways between the anchor points, which can be connected by straight lines, spline fits, or in another way.

[0064] Note that the identification of paths 401, 402 is substantially overlapping with the mask identification performed in Figures 3D and 3B, respectively. In some embodiments, pixels in the subtree masks of Figures 3H and 3F are split into main and branch vessel types based on comparison with paths 401, 402 in addition to or instead of comparison with the mask type output of Figures 3D and 3B. This is a form of output cross vessel identifier blend, combining the output of the path type ML-based vessel identifier and the mask type ML-based vessel identifier.

[0065] Cross-type repair using ML-based vessel classifier Reference is now made to Figures 5A-5B, which illustrate generally a method for vessel identification and restoration using a mixed-path and mask ML-based vessel classifier, according to some embodiments of the present disclosure.

[0066] In Fig. 5A, two LCX identifications generated as output of the ML-based vessel identifier are overlaid. Path 501 represents the output of the ML-based vessel identifier of type Path, and Mask 502 is the output of the vessel identifier of type Mask. Each output contains at least one error. Gaps 503 are errors in Mask 502, which may be caused, for example, by lesions interfering with the contrast intensity and / or vessel morphology (the gaps are artificially introduced for illustration purposes).

[0067] Path 501 crosses a gap 503. Path-type ML-based vessel classifiers are relatively tolerant to local irregularities, since (for example) if there are "normal" vessels on both sides of a gap, the process of regression will tend to make the path match those normal regions. To maintain the integrity of the overall path, the intermediate regions are naturally bridged.

[0068] However, a tail region 504 of path 501 diverges from mask 502. In some embodiments, deviations between mask 502 and path 501 near the tail 504 are treated as likely (and optionally inevitable) to be due to errors in path 501. The reason is that, for example, path 501 may be constrained by overlearned features such as limits on how curvilinear the path can be, a relative increased sensitivity to small side branches, and / or a susceptibility to getting stuck in local minima (previous path dependency) that generally tend to propagate along the path when errors occur.

[0069] 5B shows a revised vessel identification combining mask 502 and path 501. Tail regions 504 may be removed, for example, because in some parts (most distal) they detach from part of mask 502 but never rejoin, or (slightly more proximally) they detach from mask 502 and rejoin but rejoin the same self-joined "blob", thus not bridging the gap.

[0070] In contrast, bridges 505 are retained based on, for example, the criteria that two distinct (previously not interconnected) blobs are bridged. Additionally or alternatively, other criteria may be applied, for example, maximum bridge distance and / or directional similarity between the skeletonized version of mask 502 and the region of path 501 that extends along it.

[0071] ML-based vessel classification repair with combined examples Reference is now made to Figures 6A-6H, which show x-ray angiogram images (depicted in Figures 6A, 6C, 6E, and 6G, respectively) along with different masked ML-based vessel identifier outputs (Figures 6B, 6D, 6F, 6H) corresponding to anatomically different vessel types, in accordance with some embodiments of the present disclosure. Figure 6B shows the LAD mask, Figure 6F shows the LCX mask, Figure 6D shows the mask containing the LAD subtree (LAD and its diagonal or branch), and Figure 6H shows the mask containing the LCX subtree (LCX and its peripheral or branch). In this example, Figures 6D and 6F each show large gaps in the detected masks compared to what can be seen in the x-ray angiogram image.

[0072] 6J, there is shown a superimposed two path-based ML-based vessel identifier output 601, 602 according to some embodiments of the present disclosure. Path 601 runs along the LCX and path 602 runs along the LAD. However, path 602 is for the most part misidentified and "jumps" to the wrong vessel.

[0073] Further, referring to FIG. 6K, there is shown a segmented result 603 generated based on the use of various correct and partially incorrect information available from the ML-based vascular identifier output shown in FIGS. 6B, 6D, 6F, 6H, and 6J in accordance with some embodiments of the present disclosure.

[0074] In some embodiments, the rules of combination resulting in the segmentation of FIG. 6K are as already described for the previous simpler case. Path-type segmentation parts that go out of the corresponding mask-type segmentation of the image and never return may be ignored in the final result. However, if the path-type output result expands the gap to join two mask-type "blobs" that are separated, the results can be merged into one segment. Although not shown in the figure, segments can also be distinguished from each other by knowing which anatomically defined vessel types are identified by the ML-based vessel identifiers of each mask or path type.

[0075] An example of ML-based vascular classifier repair using combinations and heuristics 7A-7D, which show an X-ray angiogram image (shown in FIG. 7A and FIG. 7C, respectively) along with different mask-type ML-based vessel identifier outputs (FIG. 7B, FIG. 7D) corresponding to different anatomical vessel types, according to some embodiments of the present disclosure. FIG. 7B shows an RCA mask, and FIG. 7D shows a mask consisting of the RCA subtree (RCA and its diagonal or branch). Further, with reference to FIG. 7E, a path-type ML-based vessel identifier output 701 is shown superimposed on the image of FIG. 7A-7C, according to some embodiments of the present disclosure. Further, with reference to FIG. 7F, a segmented result 703 corrected by further constraints is shown, which is generated based on the use of various correct and partially incorrect information obtained from the ML-based vessel identifier outputs shown in FIG. 7B, 7D, and 7E, according to some embodiments of the present disclosure.

[0076] In Figure 7B, the masked ML-based vessel identifier of the RCA generates an incorrect loop 702 because the branch of the RCA crosses in front of or behind itself. This error also occurs in the ML-based vessel identifier that generated the output of Figures 7D and 7E, and it does not have the necessary information to correct for each other. However, the basic information that there is an error can be derived from the pattern in Figure 7B. This is because the ML-based vessel identifier is supposed to identify only a single vessel, and a single vessel should not have such a loop 702.

[0077] Nevertheless, it is possible to resolve the error based on the mask morphology. For example, skeletonizing the mask in FIG. 7B allows identifying the locations of bifurcations (top of loop 702) and crossings (bottom of loop 702). Moving away from these crossings, the width of the mask can be identified perpendicular to the direction in which the skeletonized mask extends. Of the two bifurcations options, the one that matches better with the pre-bifurcation (or post-bifurcation) width can be considered as more likely to be an extension of the RCA. Additionally or alternatively, the intensity of the original image can be used to evaluate the similarity of the vessel morphology. Optionally, the locations of bifurcations and crossings are distinguished by additional darkening due to vessel crossings that is not usually present at the bifurcation points.

[0078] Additionally or alternatively, the directionality of the two loop branches may be determined to be toward the location of the branch / intersection. Again, this directionality can be determined from a skeletonized version of the mask. The loop branch whose directionality is most approximately parallel to the pre-loop and / or post-loop branches of the mask can be considered as the more likely extension of the RCA.

[0079] In this example, analysis as just described allows the short side of loop 702 to be assigned as part of the RCA itself, which then allows the identification of the remaining vessel portions to be assigned based on the ML-based vessel identifier output, producing, for example, the vessel segmentation 703 shown in FIG.

[0080] It is important to understand that the analysis described here is heuristic in nature. Heuristics, while often valuable, are prone to "fragility" - they focus on a particular metric that is assumed to mean something (e.g., mean something about the shape of a blood vessel), but in realistic situations, what is assumed is not necessarily guaranteed to happen.

[0081] Even with multiple heuristics, situations may arise where the heuristics are inconsistent or even consistent but produce erroneous results. Therefore, there is a potential benefit in reducing the number of ambiguities that need to be resolved heuristically by shifting the burden of vessel segmentation and identification as much as possible to a detector such as an ML-based vessel identifier. ML-based vessel identifiers are, on the one hand, relatively specialized (e.g., specialized to detect a specific target type) and, on the other hand, relatively robust in their ability to weight different image features to produce accurate vessel identification. The robustness stems, for example, from being generated from a much larger number of realistic examples than the algorithm designers could consider individually.

[0082] Note that, for example, the mask in Fig. 7D can also be used as a basis for streamlining the forks / crossings into a simple forked (loop-free) structure. However, the situation becomes more complicated because there are additional logical possibilities that must be ruled out. For example, in the example shown, a single RCA fork loops around itself twice. However, near the second crossing 704, the RCA has contracted somewhat, so that from the width alone it is unclear whether the RCA really continues in a straight line or whether it turns to the right and starts a meandering region. Although the directional heuristics can lead to different results, this result can lead to deadlock unless one of the two heuristics is preferred.

[0083] The directionality heuristic is correct in this case, but is not foolproof in general. Note, for example, that the angle of the turn required to turn right at the intersection 704 is similar to the angle of the next downstream turn 705 of the branched vessel. If both the RCA bifurcation and the RCA begin their tortuous regions near their closest regions, the directionality heuristic may produce erroneous results.

[0084] Thus, algorithm designers using a formula-based heuristic approach must determine how to prioritize or weight the heuristics. Determining the correct method of comparison precisely may be impractical, for example, due to uncertainty about the relative importance of the weaknesses of a given heuristic. Machine learning vascular identifiers have the advantage of being generated by a process that automatically adapts to actual results.

[0085] In essence, the more loops that must be solved using a heuristic, the more likely an error will occur. In the example shown in Figures 7A-7F, using an ML-based vessel classifier specialized for single vessel detection reduces the number of intersections that require heuristic solving compared to the results of the ML-based vessel classifier for the entire subtree.

[0086] Transition of vessel direction from path-type vessel identifier output to mask-type vessel identifier output Reference is now made to Figures 8A-8E, which illustrate generally the exchange of vessel direction information from a pass-type ML output to a mask-type ML-based vessel identifier output, according to some embodiments of the present disclosure.

[0087] The masked output itself detects vascular regions inside and outside of a particular type of vessel (such as the LAD or RCA), but is not inherently suited to determine which end is the arterial ostium (proximal upstream of the artery) and which end is the distal end of the artery.

[0088] FIG. 8A shows the output of a masked ML-based vessel identifier that detects the extent of the LAD. FIG. 8B is the corresponding path-based ML-based vessel identifier. The output of the path-based vessel identifier has an end, since it is derived from a path model with an end. The end itself may have been labeled as relatively upstream (proximal) or downstream (distal) based on information implicit in the path learning method and / or based on comparison with the results of yet another vessel identifier, e.g., based on a thresholded Franzi filter heatmap, and the Franzi filter may be adjusted to differentially (e.g., preferentially) find larger (and therefore more inlet) vessels compared to smaller vessels.

[0089] In Fig. 8B, the mask is skeletonized into a skeletonized mask 801, which can be done by an erosion filter. In Figs. 8C and 8D, the path-based ML-based vessel identifier result 802 is aligned with the skeletonized mask 801. Fig. 8C shows a zoomed-in portion of the path-based ML-based vessel identifier result 802 near the mouth edge. Arrow 803 indicates the distance between the most mouth edge of the path-based ML-based vessel identifier result 802 (where edge information is available) and the nearest edge of the skeletonized mask 801. This edge becomes the mouth edge of the skeletonized mask. The opposite edge is the distal edge.

[0090] The path of the skeletonized mask 801 is optionally used as a representation of the vessel centerline. Alternatively, the skeletonized mask is used to create a geodesic distance transform of the original masked output, and the centerline is defined as the shortest path along the geodesic transform between the locations of the inlet and distal ends of the skeletonized mask 801. The centerline is then transformed back to Cartesian coordinates, for example as shown in FIG. 8E.

[0091] Vascular "subtraction" in vascular masks Reference is now made to Figures 9A-9G, which illustrate generally a method for performing vessel subtraction between the outputs of masked vessel identifiers, in accordance with some embodiments of the present disclosure. In the illustrated example, the outputs of the vessel identifiers are the outputs of two different ML-based vessel identifiers, each distinguishing between the LAD (Figure 9A) and the LAD and its diagonal (Figure 9B).

[0092] Figure 9C shows the result of simple mask subtraction: light pixels are those that are only in the mask in Figure 9B, dark pixels are those that are only in the mask in Figure 9A, and grey pixels are in both masks or are absent from neither. The result of the subtraction is poor because it leaves a fringe of mixed black and white pixels where the two ML-based vessel classifiers reached slightly different results on where the edge of the LAD should be.

[0093] FIG. 9D shows the image on which the masks of FIGS. 9A-9B were derived. As a step to address the "fringe" problem, contrast enhancement has been applied to the mask region of FIG. 9B. The contour of this enhanced region is shown in FIG. 9E, where the contour is calculated only within the neighborhood of pixels that are part of the LAD (alone) mask. This can be achieved, for example, by expanding the mask of FIG. 9A until it slightly overfills the contour of the entire enhanced region, and keeping only the contours that are in contact with the expanded mask. FIG. 9F shows the resulting LAD mask, and FIG. 9G shows the difference mask that preserves the branches of the LAD subtree.

[0094] Machine Learning Technique Examples - Block Diagram / Flowchart 10A-13 show an example of determining an optimal image from an image sequence of angiographic images. Similar to the associated description of FIGS. 2-9G, the optimal image determination system 100 can implement the techniques described below.

[0095] 10A shows a block diagram of an example of an optimal image determination system 100 that filters angiographic images according to cardiac phase. The optimal image determination system 100 operates by filtering an image sequence 1000 to one that includes angiographic images associated with a particular cardiac phase. In some embodiments, the system 100 filters the sequence 1000 to include end-diastolic images.

[0096] As shown, the optimal image determination system 100 includes a cardiac phase determination engine 1002 that analyzes an image sequence 1000. In some embodiments, the cardiac phase determination engine 1002 may be a machine learning model 1008 that receives as input a subset 1004 of the image sequence 1000. For example, the subset 1004 may include one image, three images, five images, etc. Of the five images, a central image 1006 may be combined (e.g., concatenated) with the subsequent two images and the previous two images in the image sequence 1000.

[0097] The system 100 can compute a forward pass through the machine learning model 1008 to obtain a likelihood or value associated with the central image 1006 depicting the heart during a particular cardiac phase. In some embodiments, the machine learning model 1008 can represent a neural network trained to assign a value indicative of the likelihood (e.g., the final layer can use a sigmoid activation function). With respect to the example of five input images, as can be seen, the neural network can effectively learn to distinguish between signs of motion, such as cardiac expansion, associated with a particular cardiac phase. The neural network can also be trained to assign a classification associated with the cardiac phase. Neural networks include, for example, convolutional neural networks, fully connected neural networks, attention-based networks, recurrent neural networks, and the like.

[0098] With respect to a recurrent neural network, the model 1008 may be a long short-term memory (LSTM) network. The use of LSTM units may enable the model 1008 to exploit temporal information associated with the image sequence 1000. Additionally, the LSTM units may help distinguish between blood vessels and their locations using information about how the blood vessels change over time (e.g., whether they move or move relative to each other) or simply how accurately another ML-based blood vessel identifier identifies the blood vessels in the sequence of images.

[0099] As mentioned above, the end diastolic image is preferred because it captures when the blood vessels are at their largest. For example, this cardiac phase corresponds to when the ventricles of the heart are at their largest expansion. This image is relatively easy to identify because it occurs at the moment of transition between cardiac expansion and contraction. It is therefore useful for matching with other images taken later or earlier in the cardiac cycle, as there is less chance of differences in the shape of the heart due to contraction.

[0100] In the illustrated example, the cardiac phase determination engine 1002 outputs a filtered image 1004 associated with a particular cardiac phase (e.g., an end diastolic image). For implementations that use a likelihood or probability of a particular cardiac phase, the engine 1002 can filter (e.g., remove or discard) images whose likelihood or probability is below a threshold value (e.g., 60%, 75%, 90%).

[0101] 10B illustrates a block diagram of an example optimal image determination system 100 that performs additional filtering based on a mask score associated with a segmentation mask. As shown in FIG. 10A, the system 100 can analyze angiographic images in an image sequence and identify a subset 1004 that depicts the heart in a particular cardiac phase (e.g., end diastolic phase). As described below, the system 100 can further refine or filter the subset 1004 using a segmentation mask that indicates portions of the images that represent blood vessels.

[0102] The optimal image determination system 100 includes a segmentation engine 1020 that outputs a segmentation mask based on an input of an angiogram image. For example, in FIG. 10B, the segmentation engine 1020 receives an input angiogram image 1022. As shown, the input angiogram image 1022 shows one or more blood vessels (e.g., "Vessel A"). The resulting segmentation mask 1024 shows how one or more blood vessels, such as Vessel A, are segmented from the remainder of the image 1022. In some embodiments, the segmentation mask 1024 may include pixels that are assigned a binary value by the segmentation engine 1020. A first binary pixel value (e.g., white) indicates that the pixel forms part of a blood vessel. A second binary pixel value (e.g., black) indicates that the pixel does not form part of a blood vessel.

[0103] 10B illustrates a segmentation mask 1024, it can be appreciated that in some embodiments, the segmentation engine 1020 may output information identifying the location or orientation of blood vessels within the input image 1022. For example, the engine 1020 may output a data structure (e.g., a vector or other information) that identifies pixels that form blood vessels. As another example, the engine 1020 may output information indicative of a boundary or contour of a blood vessel.

[0104] The segmentation engine 1020 may represent a machine learning model trained to output a segmentation mask. For example, a convolutional neural network may be trained to output the segmentation 1024. In this example, the convolutional neural network may be trained based on the ground truth segmentation mask and the associated angiogram image. In some embodiments, the segmentation engine 1020 may represent the performance of classical computer vision techniques. As described herein, the neural networks may be specific to a side or artery of the heart. For example, a first neural network may output a segmentation mask of an image associated with the right side of the heart. As another example, one or more additional neural networks may output a segmentation mask of an image associated with a particular artery (e.g., left anterior descending artery, left marginal artery, left circumflex artery) of the left side of the heart.

[0105] These segmentation masks can be mask- or path-type, as shown in Figure 2. In overview, mask-type outputs consist of "blobs" of connected pixels that are classified as belonging to a particular vessel type, but are not necessarily structured, except insofar as their connectivity can be used for further analysis steps, such as skeletonization. Path-type outputs arrange the regions along the path in a certain order. The edges themselves are usually also identified, for example as closer to the opening or farther from it (e.g., relatively upstream / downstream, respectively, for arteries). Additionally, the segmentation masks can also be analyzed to address errors in the mask or to update the information, as explained in Figures 2-9G.

[0106] In some embodiments, at least one formula-based method (e.g., classical computer vision techniques) is combined with at least one machine learning-based method. For example, a Frangi filter can be used in combination with a segmentation network to identify smaller unidentified vessels that are missed by the segmentation network. More specifically, an edgeClearFrangi (ECF) Frangi filter variant can be used. The Frangi filter is a formula-based method used to generate a "heat map" indicating the likelihood that certain pixels are vessels, which is then converted, if necessary, by application of a threshold, into a mask of image pixels that are determined to be vessel regions. Frames with higher Frangi filter responsiveness (e.g., overall response strength and / or regions of the mask that meet certain threshold criteria) are accordingly deemed to have more and / or denser vascular filling with contrast agent. Additionally, the Frangi filter can be used to connect gaps that are missed by the segmentation network, reducing connectivity issues. Thus, the Frangi filter not only reduces connectivity issues that may exclude an image from consideration for optimal frames, but also notifies the user of the presence of vessels and their contours, allowing the user to later delineate those vessels.

[0107] A detailed description of segmentation masks and examples of graphical representations of segmentation masks are included herein with respect to Figures 2-9G. For example, the segmentation engine 1020 can output an ML-based vessel identifier as a different mask type output.

[0108] Based on the segmentation mask 1024, the segmentation engine 1020 generates a mask score 1026 associated with one or more vessels in the mask 1024. For example, the mask score may indicate a measurement or metric associated with a size of vessel A. In this example, the segmentation engine 1020 may determine a number of pixels (e.g., number of white pixels) that form vessel A. The segmentation engine 1020 may also determine a region (e.g., an image region) associated with vessel A. The segmentation engine 1020 may also determine a length of vessel A. For example, the length may indicate a length associated with a line (e.g., a centerline) located through the center of vessel A.

[0109] The segmentation mask may depict vessels incompletely. For example, the segmentation mask may have discontinuities or white portions separated by black portions. In some embodiments, the segmentation engine 1020 may correct the discontinuities. For example, the engine 1020 may assign a black portion (e.g., a portion that is not shown to form a vessel) as a white portion if the black portion is below a threshold size or length and is adjacent to a white portion on one or both sides. To correct the discontinuities, for example, the engine 1020 may apply a Frangi filter to the segmentation mask. Discontinuity correction is further described elsewhere in connection with FIGS. 3A-9G. In some embodiments, the segmentation engine 1020 may exclude certain angiogram images in the subset 1004 from further analysis based on whether they contain such discontinuities. Thus, the engine 1020 may prioritize angiogram images that result in a segmentation mask with reduced error.

[0110] Thus, the segmentation engine 1020 can determine a mask score 1030 for the subset 1004. As discussed above, the mask score may indicate a size or length associated with the vessel. In some embodiments, an image may have a mask score for each vessel depicted in an associated segmentation mask. In some embodiments, an image may have a single mask score based on the vessels depicted in the associated segmentation mask.

[0111] The segmentation engine 1020 can filter the subset 1004 using the mask score 1030. For example, the segmentation engine 1020 can discard angiogram images associated with a mask score less than a threshold percent of the maximum (e.g., higher) mask score. Thus, the segmentation engine 1020 can output filtered angiogram images 1028 along with the mask score 1030.

[0112] 10C illustrates a block diagram of an example of an optimal image determination system 100 that generates an optimal angiogram image 1042. As shown in FIG 10B, the system 100 can determine images 1028 that have associated segmentation masks that include long or large vessels, and thus, these images 1028 may provide a clearer view of the vessels.

[0113] The optimal image determination system 100 includes an optimal image selection engine 1040 that determines or outputs an optimal angiographic image 1042. In some embodiments, the optimal angiographic image 1042 represents one of the images 1028 with the best image quality. For example, the optimal angiographic image 1042 may have the best contrast.

[0114] As shown, the image 1028 is provided to a contrast score engine 1044. The contrast score engine 1044 may register the image 1028, e.g., align the images. As known to those skilled in the art, aligning the images allows them to be properly compared. For example, slight movement of the imaging tool (e.g., a C-arm) or the patient may result in slight differences between the viewpoints associated with the images 1028. Thus, the same blood vessel may be located in different positions in the images 1028.

[0115] After aligning the images, the contrast score engine 1044 can determine a contrast score 1026 for the images 1028. For example, the angiogram images may be grayscale. In this example, the engine 1044 may determine a contrast score based on the blood vessels depicted in one of the images 1028. The contrast score may indicate, for example, an average grayscale value. These contrast scores may be generated based on the same blood vessels depicted or included in the images 1028.

[0116] For example, once the images 1028 are aligned, the same segmentation mask may be used (e.g., applied) to the images. Thus, the segmentation mask may identify blood vessels in the image 1028 and the image information of each of the blood vessels may be used to generate the contrast score 1026. In some embodiments, the segmentation mask may represent a segmentation mask that is a combination, or alignment, of the segmentation masks determined for the image 1028 of FIG. 10B. In some embodiments, the segmentation masks may be aligned and then combined using a logical "Or" such that the mask extends beyond the blood vessels depicted in the image. As can be appreciated, having a shorter view of the blood vessels in the image may result in a lower contrast score for the image due to the use of additional darker non-vascular regions when calculating the contrast score, as described below.

[0117] The comparison engine 1044 can then identify an optimal angiogram image 1042 based on the contrast score 1026 and the mask score 1030. For example, the engine 1044 can select the image 1042 as one of the images 1028 with the highest contrast score 1026. This may indicate that the vessel size or length in the image 1042 is within an acceptable range (e.g., as described in FIG. 10B) and that the image contrast is the highest. As another example, the engine 1044 can use a combination of the contrast score 1026 and the mask score 1030. For example, the scores 1026, 1030 may be weighted and the highest combination may indicate the optimal angiogram image 1042. Optionally, the mask score 1030 may be used only for two or more images 1028 with contrast scores 1026 within a threshold.

[0118] Although FIGS. 10A-10C describe the use of machine learning models, it can be appreciated that a single machine learning model or multiple machine learning models may be used. For example, a single machine learning model may be trained end-to-end to identify optimal angiography images based on a received image sequence. As another example, multiple machine learning models may be used to perform different steps. For example, a first model may be used to identify end diastolic images and a second model may be used to output a segmentation mask.

[0119] Left artery / right artery As known to those skilled in the art, there are two major coronary arteries: the right coronary artery and the left main coronary artery. On the right side, there is a main artery called the right coronary artery, while on the left side, there are multiple arteries. For example, on the left side, it branches into the left anterior descending artery (LAD) and the left circumflex artery (LCX). The left circumflex artery has a branch called the left marginal artery (LMA).

[0120] In some embodiments, the system 100 can analyze the received image sequence 1000 and identify whether the image sequence 1000 depicts a vessel on the left side or the right side. For example, the system 100 can provide each image of the image sequence 1000 to a machine learning model (e.g., a convolutional neural network, optionally including one or more fully connected layers). The model can output, for each image, a probability that the depicted vessel is on the left side or the right side, respectively. The model also outputs a classification indicating whether the vessel is on the left side or the right side. The image sequence 1000 as a whole is determined to depict the right or left side of the heart according to what is determined to be the majority of the included angiographic images. For example, a voting technique may be used to select one side or the other based on whether the included images are determined to be the majority of the left side or the right side.

[0121] The system 100 can then use the left or right side assignment to refine its determination. For example, in some embodiments, the system 1000 can use separate machine learning models to determine the end diastolic image. In this example, the two machine learning models can be trained using training data specific to the left or right side. Similarly, the system 1000 can use separate segmentation networks or models for the left or right side.

[0122] With regard to separate segmentation masks, as described above, there may be more than one artery (e.g., LAD, and LCX or LMA) on the left side. Thus, for the left side, the system 1000 may generate more than one segmentation mask. In some embodiments, different segmentation networks may be used for the arteries. As shown in FIG. 10C, the system 1000 may determine a mask score based on the segmentation mask. For the left side, the system 1000 may determine more than one mask score for each image. These mask scores may optionally be combined or aggregated into a single mask score for the image. The mask scores may optionally be kept separate and used to filter the images as described above. For example, images with a mask score below a threshold percent of the maximum mask score may be discarded.

[0123] Flowchart example 11 illustrates a flow chart of an exemplary process 1100 for determining an optimal angiographic image. For convenience, the process 1100 is described as being performed by one or more computer or processor systems (e.g., optimal image determination system 100).

[0124] In block 1102, the system receives a vascular image sequence. As described, the system receives vascular images, such as angiogram images, of a portion of a patient's heart. The images may be taken by an imaging system or tool (such as a C-arm) at a particular rotation and / or translation relative to the patient's heart. Thus, the images may be associated with a particular viewpoint of the portion.

[0125] At block 1104, the system optionally determines whether a blood vessel in the image sequence corresponds to the left or right side of the patient's heart. In some embodiments, a machine learning model (e.g., a convolutional neural network) may be used to perform this determination. For example, as described above, the machine learning model may output a value indicating the respective likelihood that the blood vessel falls on the left or right side.

[0126] In block 1106, the system determines a subset of end-diastolic vascular images. As part of identifying the optimal images, as shown in FIG. 10A, the system determines vascular images associated with the lengthening and / or relaxation of cardiac blood vessels. This cardiac phase, such as the end-diastolic phase, may enhance downstream processing of the optimal images. For example, end-diastolic phase images may provide greater vascular detail and / or definition, allowing an end user to more quickly identify, mark, etc., blood vessels in the images. As another example, end-diastolic phase images may enable better (e.g., less error-prone) three-dimensional modeling.

[0127] In block 1108, the system generates a segmentation mask based on the subset. As shown in FIG. 10B, the system generates the segmentation mask to identify vessel images that depict larger and / or longer vessels. For example, during the capture of a particular vessel image, dye may not be properly loaded into the patient's vessels, causing the vessels to be less visible in the particular vessel image.

[0128] The system generates a mask score based on the segmentation mask. For example, the mask score may indicate a length or size associated with a vessel included in the segmentation mask. The system may filter, such as remove or discard, images that are below a threshold percentage of the maximum mask score.

[0129] As described above, the system may generate a single segmentation mask for the right vascular image and two or more segmentation masks for the left image. For example, for the left image, the system may generate a first segmentation mask that depicts vessels associated with the left anterior descending artery. The system may similarly generate a second segmentation mask that depicts vessels associated with the left marginal artery. These segmentation masks may be analyzed individually or combined into a single mask score as described herein.

[0130] In block 1110, the system determines an optimal vascular image based on the segmentation mask and a score associated with image quality. As shown in FIG. 10C, the image quality score, also referred to as the contrast score, indicates a measure of contrast quality in the vascular image. An example of a process for determining the contrast score is described in FIG. 12. As described above, the system can select the optimal image with the best image quality. The system can also select the optimal image with the best combination of image quality and mask score.

[0131] The system can output and present the optimal vascular image via an interactive user interface, as described in Figure 1. The end user can review the determined image and select a different vascular image from the sequence if desired. In some embodiments, such overrides may be used to update the machine learning models described herein.

[0132] 12 is a flow chart of an exemplary process 1200 for selecting an optimal angiographic image from an image sequence based on a contrast score. For convenience, the process 1200 is described as being performed by one or more computer or processor systems (e.g., optimal image determination system 100).

[0133] In block 1202, the system obtains a segmentation mask generated from the vascular image. In block 1204, the system determines a mask score associated with the vessels depicted in the segmentation mask. In block 1206, the system selects a subset of the vascular image based on the mask score.

[0134] In block 1208, the system registers the subsets of vascular images. For example, the system aligns the images (e.g., rotates the images or removes portions of the images) so that the subsets can have the same features or elements (e.g., blood vessels) in the same parts of the images. The system may generate a specific segmentation mask, such as a combination (e.g., sum of masks) of segmentation masks that is universally applied to the subsets. In some embodiments, the specific segmentation mask may be an aligned set of segmentation masks. In some embodiments, the specific segmentation mask may represent a segmentation mask generated from the aligned images.

[0135] In block 1210, the system determines a contrast score for the subset of vascular images. The system applies a particular segmentation mask to each image in the subset to identify the vessels depicted therein. The system then determines a contrast score for the image based on the average grayscale value of the pixels that form the vessels depicted in the image. In some embodiments, a lower average grayscale value may indicate a higher contrast score.

[0136] In block 1212, the system selects an optimal vascular image as described at least in FIGS.

[0137] System Example Reference is now made to Figure 13, which illustrates generally a system 1300 configured for use in segmenting a vascular tree from a cardiac image, in accordance with some embodiments of the present disclosure. In some embodiments, the system 1300 may be the system 100 described herein.

[0138] Computing device 1305 is the main subsystem of system 1300. Imaging device 1310 may comprise system 1300 or may be a separately provided imaging device to which system 1300 is interconnected to provide access to cardiovascular images generated by imaging device 1310. In some embodiments, imaging device 1310 is an X-ray angiography device.

[0139] In some embodiments, the computing device 1305 includes a processor 1301. The processor 1301 collectively comprises the processing capabilities of the computing device 1305 and is not meant to be limited to either a single processing unit, a single enclosure, or a single processing device type. For example, the processor 1301 may be a mix of CPU and GPU processing units. The processor 1301 is optionally distributed among processing units linked by network connections. The memory 1304A is comprised of one or both of the volatile storage 1302 and the data storage device 1304 (again, these blocks collectively comprise the available memory capabilities of the computing device). The processor instructions 1303 (stored in the memory 1304A) direct the processor to perform computational aspects of the system's functionality, e.g., computational aspects of one or more of the methods described herein. For example, the processor instructions may include programming instructions for one or more of the methods of Figures 11 and 12, and / or instructions for performing one or more of the methods or techniques described in connection with Figures 2-9G. Memory 1304A encodes a machine learning product, which in some embodiments may be an ML-based vascular identifier. In some embodiments, additionally or alternatively, memory 1304A encodes instructions for executing a formula-based algorithm. Optional user interface 1307, in some embodiments, is configured to support receipt of user-specified instructions to the system and provide display and instruction of device functionality, such as, for example, images of the cardiovascular system and / or segmentations thereof.

[0140] Other embodiments All processes described herein may be implemented and fully automated by software code modules executed by a computing system including one or more computers or processors. The code modules may be stored in any type of non-transitory computer readable medium or other computer storage device. Some or all of the methods may be implemented in dedicated computer hardware.

[0141] Many variations beyond those described herein will be apparent from the present disclosure. For example, depending on the embodiment, certain operations, events, or functions of any of the algorithms described herein may be performed in a different order, or may be added, combined, or omitted entirely (e.g., not all operations or events described are necessary for the execution of an algorithm). Furthermore, in certain embodiments, operations or events may be performed simultaneously rather than sequentially, for example, via multi-threading, interrupt processing, or multiple processors or processor cores, or other parallel architectures. Furthermore, different tasks or processes may be performed by different machines and / or computing systems that may function in conjunction.

[0142] The various example logic blocks and modules described in connection with the embodiments disclosed herein may be implemented or executed by a machine such as a processing unit or processor, a digital signal processor (DSP), an application specific integrated circuit (ASIC), a field programmable gate array (FPGA) or other programmable logic device, discrete gate or transistor logic, discrete hardware components, or any combination thereof designed to perform the functions described herein. The processor may be a microprocessor, but alternatively the processor may be a controller, a microcontroller, or a state machine, combinations thereof, and the like. The processor may include electrical circuitry configured to process computer-executable instructions. In another embodiment, the processor includes an FPGA or other programmable device that performs logical operations without processing computer-executable instructions. The processor may also be implemented as a combination of computing devices, for example, a combination of a DSP and a microprocessor, multiple microprocessors, one or more microprocessors in combination with a DSP core, or other similar configurations. Although described primarily in terms of digital technology, a processor may also include primarily analog components. For example, some or all of the signal processing algorithms described herein may be implemented in analog circuitry or mixed analog and digital circuitry. Computing environments include any type of computer system, including, but not limited to, computer systems based on microprocessors, mainframe computers, digital signal processors, portable computing devices, device controllers, computational engines within appliances, and the like.

[0143] Conditional language such as "can," "could," "might," or "may," unless expressly stated otherwise, is understood as being used generally to convey that certain embodiments include certain features, elements, and / or steps and that other embodiments do not, within the context. Thus, such conditional language does not generally imply that the features, elements, and / or steps are in any way required by one or more embodiments, nor does it imply that one or more embodiments necessarily include logic for determining whether those features, elements, and / or steps are included or performed in a particular embodiment, with or without user input or prompting.

[0144] Disjunctive language, such as the phrase "at least one of X, Y, Z," is understood to be generally used to indicate that an item, term, etc. may be either X, Y, Z, or any combination thereof (e.g., X, Y, and / or Z), unless otherwise indicated. Thus, such disjunctive language is generally not intended to, and should not, imply that a particular embodiment requires that at least one of X, at least one of Y, or at least one of Z, respectively, be present.

[0145] The process illustrations, elements, or blocks in the flow diagrams described herein and / or depicted in the accompanying figures should be understood as potentially representing modules, segments, or portions of code that include one or more executable instructions for implementing a particular logical function or element in the process. Alternative implementations are included within the scope of the embodiments described herein, where elements or functions may be omitted, performed out of the order shown, or discussed, including substantially simultaneously or in reverse order, depending on the functionality involved, as will be understood by those skilled in the art.

[0146] Unless expressly stated otherwise, articles such as "a" and "an" are generally construed to include one or more described items. Thus, phrases such as "a device configured" are intended to include one or more described devices. Such one or more described devices may also be collectively configured to execute the stated description. For example, "a processor configured to execute descriptions A, B, and C" may include a first processor configured to execute description A working in conjunction with a second processor configured to execute descriptions B and C.

[0147] It should be emphasized that many variations and modifications are possible in the above-described embodiment, and that the elements are to be understood as examples of other acceptable alternatives, and all such modifications and variations are intended to be included within the scope of this disclosure.

[0148] (Appendix 1) 1. A method for identifying a vascular portion of a vascular image, comprising: accessing the vascular image; applying a first ML-based vascular classifier to the vascular image to generate a first data structure that identifies at least some vascular portions within the vascular image; applying a second vascular identifier to the vascular image to generate a second data structure, the second data structure identifying a portion of the vascular image that is not included in the first data structure; combining the first data structure and the second data structure to generate a combined identification of vascular portions within the vascular image; Equipped with method.

[0149] (Appendix 2) the first ML-based vascular identifier is configured to recognize a first anatomically-defined vessel type based on a training input used to generate the ML-based vascular identifier; the second vascular identifier is also an ML-based vascular identifier and is configured to recognize a second anatomically-defined vessel type based on a training input used to generate the ML-based vascular identifier. The method described in Appendix 1.

[0150] (Appendix 3) the first anatomically-defined vessel type and the second anatomically-defined vessel type are selected from the group consisting of the LAD, other vessels in the LAD subtree, the LCX, other vessels in the LCX subtree, the RCA, and other vessels in the RCA subtree; The method described in Appendix 2.

[0151] (Appendix 4) the first ML-based vessel identifier and the second ML-based vessel identifier are configured to recognize the first anatomically-defined vessel type and the second anatomically-defined vessel type, respectively, using a training input including images and image location representations of the first anatomically-defined vessel type and the second anatomically-defined vessel type, respectively. The method described in Appendix 2.

[0152] (Appendix 5) the first data structure includes a definition of a path, the path extending along a vessel segment; the second data structure includes a pixel mask that identifies pixels corresponding to vascular segments; The method described in Appendix 1.

[0153] (Appendix 6) The first ML-based vascular classifier comprises a network whose output layer is a recurrent layer. The method described in Appendix 5.

[0154] (Appendix 7) The second vessel classifier is an ML-based vessel classifier, and the output layer is a classification layer. The method described in Appendix 5.

[0155] (Appendix 8) The combining step comprises: identifying a first region and a second region identified by both the first ML-based vessel identifier and the second vessel identifier; selecting for inclusion in the combined identification a third region identified by one of the first ML-based vessel identifier and the second vessel identifier based on a third region extending between the first region and the second region; Including, The method described in Appendix 5.

[0156] (Appendix 9) At least one of the first data structure and the second data structure includes incorrectly identified vessel segments; and The method comprises: determining whether a difference in identification of vascular segments between the first data structure and the second data structure corresponds to a known error propensity of one of the first ML-based vascular identifier and the second vascular identifier; and the combining step includes selectively including identified vessel portions in the combined identification according to identified known error propensities. The method described in Appendix 1.

[0157] (Appendix 10) 1. A method for identifying a vascular portion of a vascular image, comprising: accessing a plurality of vascular images including the vascular image; applying an ML-based vascular classifier to the plurality of vascular images to generate a first data structure that identifies at least some vascular portions; comparing the first data structure with the vascular image to identify vascular segments; Equipped with method.

[0158] (Appendix 11) The plurality of vascular images are each a part of the same time series image. 11. The method according to claim 10.

[0159] (Appendix 12) each image in the time series images the same anatomical region, and each adjacent image pair in the time series images the anatomical region one second or less apart. 12. The method according to claim 11.

[0160] (Appendix 13) The ML-based vessel identifier is an ML-based vessel identifier trained on multiple time series images. 11. The method according to claim 10.

[0161] (Appendix 14) the second data structure omits some vessel portions identified in the first data structure; The method described in Appendix 1.

[0162] (Appendix 15) 1. A system for identifying a vascular portion of a vascular image, comprising: A processor and a memory, The memory includes: The processor, accessing the vascular image; applying a first ML-based vascular classifier to the vascular image to generate and store in the memory a first data structure that identifies at least some vascular portions within the vascular image; applying a second vascular identifier to the vascular image to generate a second data structure and storing it in the memory, the second data structure identifying a portion of the vascular image that is not included in the first data structure; combining the first data structure and the second data structure to generate a combined identification of vascular portions within the vascular image and storing it in the memory; The command to carry out Encode, system.

[0163] (Appendix 16) the first ML-based vascular identifier is configured to recognize a first anatomically-defined vessel type based on a training input used to generate the ML-based vascular identifier; the second vascular identifier is also an ML-based vascular identifier and is configured to recognize a second anatomically-defined vessel type based on a training input used to generate the ML-based vascular identifier. 16. The system of claim 15.

[0164] (Appendix 17) the first anatomically-defined vessel type and the second anatomically-defined vessel type are selected from the group consisting of the LAD, other vessels in the LAD subtree, the LCX, other vessels in the LCX subtree, the RCA, and other vessels in the RCA subtree; 17. The system of claim 16.

[0165] (Appendix 18) the first ML-based vessel identifier and the second ML-based vessel identifier are configured to recognize the first anatomically-defined vessel type and the second anatomically-defined vessel type, respectively, using a training input including images and image location representations of the first anatomically-defined vessel type and the second anatomically-defined vessel type, respectively. 17. The system of claim 16.

[0166] (Appendix 19) the first data structure includes a definition of a path, the path extending along a vessel segment; the second data structure includes a pixel mask that identifies pixels corresponding to vascular segments; 16. The system of claim 15.

[0167] (Appendix 20) The first ML-based vascular classifier includes a network whose output layer is a regression layer. 20. The system of claim 19.

[0168] (Appendix 21) The second vessel classifier is an ML-based vessel classifier, and the output layer is a classification layer. 20. The system of claim 19.

[0169] (Appendix 22) The instructions cause the processor to: identifying a first region and a second region identified by both the first ML-based vessel identifier and the second vessel identifier; and selecting for inclusion in the combined identification the third region identified by one of the first ML-based vascular identifier and the second vascular identifier based on a third region extending between the first region and the second region. 20. The system of claim 19.

[0170] (Appendix 23) The instructions cause the processor to: determining a condition in which at least one of the first data structure and the second data structure contains an incorrectly identified vessel portion; determining whether a difference in identification of vascular segments between the first data structure and the second data structure corresponds to a known error propensity of one of the first ML-based vascular identifier and the second vascular identifier; Selectively combining the first data structure and the second data structure according to identified known error trends; Hereby further orders: 16. The system of claim 15.

[0171] (Appendix 24) the second data structure omits some vessel portions identified in the first data structure; 16. The system of claim 15.

[0172] (Appendix 25) 1. A system for identifying a vascular portion of a vascular image, comprising: A processor and a memory, The memory includes: The processor, accessing a plurality of vascular images including the vascular image; applying an ML-based vascular classifier to the plurality of vascular images to generate a first data structure that identifies at least some vascular portions; comparing the first data structure with the vascular image to identify vascular segments; The command to carry out Encode the system.

[0173] (Appendix 26) The plurality of vascular images are each part of a same time series of images. 26. The system of claim 25.

[0174] (Appendix 27) each image in the time series images the same anatomical region, and each adjacent image pair in the time series images an anatomical region within one second or less of each other. 27. The system of claim 26.

[0175] (Appendix 28) The ML-based vascular identifier is an ML-based vascular identifier trained on a plurality of time series images. 26. The system of claim 25.

[0176] (Appendix 29) 1. A method implemented by one or more computer systems, comprising: accessing an image sequence including a plurality of vascular images, the vascular images depicting a portion of the heart from a particular viewpoint, the vascular images associated with different times within a time range; determining a subset of the vascular images associated with a particular cardiac phase, the determining comprising computing a forward pass through a machine learning model trained to classify vascular images based on cardiac phase; generating a segmentation mask associated with the subset, the segmentation mask segmenting blood vessels included in each of the blood vessel images forming the subset, a mask score indicative of a size or length associated with the blood vessels included in the segmentation mask being determined for the segmentation mask, and filtering the subset to remove one or more blood vessel images based on the mask score; determining particular vascular images to be included in the filtered subset, said determining being based on an analysis of one or more image quality measures determined for each vascular image included in the filtered subset; Equipped with the particular vascular image is configured for inclusion in an interactive user interface. method.

[0177] (Appendix 30) The blood vessel image is an angiographic image. 29. The method according to claim 29.

[0178] (Appendix 31) The vascular image is a computed tomography image. 29. The method according to claim 29.

[0179] (Appendix 32) the vascular images included in the image sequence are acquired at a specific frequency and are separated in time according to the specific frequency; 29. The method according to claim 29.

[0180] (Appendix 33) the image sequence is acquired via a C-arm positioned to obtain the image sequence at a particular viewpoint; 29. The method according to claim 29.

[0181] (Appendix 34) Determining the subset of cardiac phase images includes: determining whether the image sequence is associated with a left or right side of the heart; The machine learning model is selected from a plurality of machine learning models based on the determination. 29. The method according to claim 29.

[0182] (Appendix 35) determining whether the image sequence is associated with the right side or the left side is based on the use of different machine learning models, the different machine learning models assigning to each vascular image a value indicative of the likelihood that the vascular image is associated with the left side and the right side; 35. The method according to claim 34.

[0183] (Appendix 36) The machine learning model is a convolutional neural network, and determining the subset includes: obtaining a threshold number of blood vessel images from the image sequence, the images forming the threshold number being consecutive in the image sequence; providing the threshold number of the vascular images as an input to the machine learning model, the machine learning outputting a classification associated with a particular image within the threshold number; Including, 29. The method according to claim 29.

[0184] (Appendix 37) the particular image is a median image of the threshold number of the vascular images; 37. The method according to claim 36.

[0185] (Appendix 38) The particular cardiac phase is end diastole. 29. The method according to claim 29.

[0186] (Appendix 39) The machine learning model outputs a value indicating the likelihood that the vascular image is in end diastole. 29. The method according to claim 29.

[0187] (Appendix 40) The machine learning model uses one or more long short-term memory units. 29. The method according to claim 29.

[0188] (Appendix 41) the segmentation masks are generated based on different machine learning models, the different machine learning models being trained to segment blood vessels depicted in a blood vessel image; 29. The method according to claim 29.

[0189] (Appendix 42) The different machine learning model is a convolutional neural network. 42. The method according to claim 41.

[0190] (Appendix 43) The segmentation mask is generated based on the use of a Frangi filter. 29. The method according to claim 29.

[0191] (Appendix 44) The segmentation mask includes pixels that are assigned a binary color, a first color indicating that the assigned pixel forms part of a blood vessel and a second color indicating that the assigned pixel does not form part of a blood vessel, 29. The method according to claim 29.

[0192] (Appendix 45) the image sequence is determined to be associated with a left or right side of the heart, and for the right side, a single segmentation mask is generated for each vascular image included in the subset; 29. The method according to claim 29.

[0193] (Appendix 46) the single segmentation mask segments blood vessels associated with the right coronary artery; 46. ​​The method according to claim 45.

[0194] (Appendix 47) the image sequence is determined to be associated with a left or right side of the heart, and for the left side, two or more segmentation masks are generated for each vascular image included in the subset; 29. The method according to claim 29.

[0195] (Appendix 48) the two or more segmentation masks segment vessels associated with two or more of a left anterior descending (LAD) artery, a left circumflex (LCX) artery, and a left marginal artery; 48. The method according to claim 47.

[0196] (Appendix 49) The segmentation mask is of blob or path type. 29. The method according to claim 29.

[0197] (Appendix 50) The output of the machine learning model is an ML-based vascular identifier. 29. The method according to claim 29.

[0198] (Appendix 51) one or more vessel images included in the subset are discarded based on the segmentation mask, discarding being based on vessels included in the segmentation mask having discontinuities; 29. The method according to claim 29.

[0199] (Appendix 52) The mask score indicates a length associated with a blood vessel or an area associated with the blood vessel. 29. The method according to claim 29.

[0200] (Appendix 53) Determining the particular angiographic image includes: registering the vascular images included in the filtered subset, where registering includes aligning features depicted in the vascular images; determining an image quality measure comprising a contrast score of the vascular images included in the filtered subset, the contrast score being indicative of an average grayscale value associated with blood vessels in the vascular images; determining the particular angiographic image based on the contrast score; Including, 29. The method according to claim 29.

[0201] (Appendix 54) a combined segmentation mask is generated for the vascular images in the filtered subset, and the contrast score compares the contrast of the same blood vessels in the vascular images; 54. The method according to claim 53.

[0202] (Appendix 55) the distance score is a filling score, and further, determining the filling score comprises determining a volume of pixels forming the blood vessel or determining an estimate of the volume of pixels using the segmentation mask. 54. The method according to claim 53.

[0203] (Appendix 56) the interactive user interface is configured to allow selection of a vascular image different from the particular vascular image, and the system is configured to update the machine learning model based on the selection. 29. The method according to claim 29.

[0204] (Appendix 57) the interactive user interface is responsive to user input indicating a contour or location of a blood vessel within a particular angiographic image; 29. The method according to claim 29.

[0205] (Appendix 58) A system comprising one or more computers and a non-transitory computer storage medium storing instructions that, when executed by the one or more computers, cause the one or more computers to perform a method according to any one of appendices 29 to 57.

[0206] (Appendix 59) A non-transitory computer storage medium storing instructions that, when executed by one or more computer systems, cause the one or more computers to perform the method of any one of appended claims 26 to 57.

[0207] (Appendix 60) 1. A method implemented by one or more computer systems, comprising: accessing an image sequence including a plurality of vascular images, the vascular images depicting a portion of the heart from a particular viewpoint, the vascular images associated with different times within a time range; applying a first machine learning model configured to generate a first set of vascular images, wherein one or more vascular images of the plurality of vascular images are removed to generate the first set, the first set including vascular images associated with end diastole of the heart; applying a second machine learning model configured to generate a second set of vascular images, wherein one or more vascular images of the first set are removed to generate the second set, the second set including vascular images associated with vessels having lengths greater than a threshold percentile; determining a particular angiographic image from the second set based on a contrast score associated with the second set, the contrast score indicating a measure of contrast quality associated with one or more blood vessels depicted in the angiographic image; Equipped with method.

[0208] (Appendix 61) the first and second machine learning models are selected based on whether the image sequence depicts a left side or a right side of the heart. 61. The method according to claim 60.

[0209] (Appendix 62) One or more computers; When executed by the one or more computers, the one or more computers applying a first machine learning model to an image sequence including a plurality of vascular images, the second machine learning model being configured to form a first set of vascular images, wherein one or more vascular images of the plurality of vascular images are removed to form the first set, the first set including vascular images associated with end diastole of the heart; applying a second machine learning model configured to form a second set of vascular images, wherein one or more vascular images of the first set are removed to form the second set, the second set including vascular images associated with vessels having a length greater than a threshold percentile; determining a particular angiographic image from the second set based on a contrast score associated with the second set, the contrast score indicating a measure of contrast quality associated with one or more blood vessels depicted in the angiographic image; The command to carry out a non-transitory computer storage medium for storing the Equipped with system.

[0210] (Appendix 63) the first and second machine learning models are selected based on whether the image sequence depicts a left side or a right side of the heart. 63. The system of claim 62.

[0211] (Appendix 64) When executed by a system of one or more computers, the system causes the one or more computers to: applying a first machine learning model to an image sequence including a plurality of vascular images, the first machine learning model being configured to form a first set of vascular images, wherein one or more vascular images of the plurality of vascular images are removed to form the first set, the first set including vascular images associated with end diastole of the heart; applying a second machine learning model configured to form a second set of vascular images, wherein one or more vascular images of the first set are removed to form a second set, the second set including vascular images associated with vessels having lengths greater than a threshold percentile; determining a particular angiographic image from the second set based on a contrast score associated with the second set, the contrast score indicating a measure of contrast quality associated with one or more blood vessels depicted in the angiographic image; A non-transitory computer storage medium that stores instructions for causing a program to execute a program.

[0212] (Appendix 65) the first and second machine learning models are selected based on whether the image sequence depicts a left side or a right side of the heart. 65. The computer storage medium of claim 64.

[0213] (Appendix 66) 1. A method implemented by one or more computer systems, comprising: accessing a plurality of image sequences each including a plurality of vascular images, the vascular images depicting a portion of the heart from a particular viewpoint, the vascular images associated with different times within a time range; causing presentation of an interactive user interface via a user device, the interactive user interface is responsive to a selection of a particular image sequence, the selection triggering a determination of a particular vascular image included in the particular image sequence; Determining the particular vascular image includes: determining a subset of the vascular images in the particular image sequence associated with a particular cardiac phase, the determining including computing a forward pass through a machine learning model trained to classify vascular images based on cardiac phase; generating a segmentation mask associated with the subset, the segmentation mask segmenting blood vessels included in each of the blood vessel images forming the subset, a mask score indicative of a size or length associated with the blood vessels included in the segmentation mask being determined for the segmentation mask, and the subset being filtered to remove one or more blood vessel images based on the mask score; determining the particular vascular images included in the filtered subset, the determining being based on analyzing one or more image quality measures determined for each vascular image included in the filtered subset; Including, Steps and updating the interactive user interface to include the particular vascular image; Equipped with method.

[0214] (Appendix 67) the interactive user interface presents an animation or video of the vascular images forming the particular image sequence. 67. The method according to claim 66.

[0215] (Appendix 68) the particular vascular image is identified in the animation or video presentation; 68. The method according to claim 67.

[0216] (Appendix 69) 1. A method implemented by one or more computer systems, comprising: accessing a plurality of image sequences each including a plurality of vascular images, the vascular images depicting a portion of the heart from a particular viewpoint, the vascular images associated with different times within a time range; causing presentation of an interactive user interface via a user device, the interactive user interface is responsive to a selection of a particular image sequence, the selection triggering a determination of a particular vascular image included in the particular image sequence; Determining the particular vascular image includes: applying a first machine learning model to the particular image sequence, the first machine learning model configured to generate a first set of vascular images, wherein one or more vascular images in the particular image sequence are removed to generate the first set, the first set including vascular images associated with end diastole of the heart; applying a second machine learning model configured to form a second set of vascular images, wherein one or more vascular images in the first set are removed to form the second set, the second set including vascular images associated with vessels having lengths greater than a threshold percentile; and determining a particular angiographic image from the second set based on a contrast score associated with the second set, the contrast score indicating a measure of contrast quality associated with one or more blood vessels depicted in the angiographic image; Including, Steps and updating the interactive user interface to include the particular vascular image; Equipped with method.

[0217] (Appendix 70) the interactive user interface presents an animation or video of the vascular images forming the particular image sequence. 69. The method according to claim 69.

[0218] (Appendix 71) the particular vascular image is identified in the animation or video presentation; 71. The method of claim 70.

Claims

1. 1. A method for identifying a vascular portion of a vascular image, comprising: accessing the vascular image; applying a first ML-based vascular identifier to the vascular image to generate a first data structure that identifies at least some vascular portions within the vascular image; applying a second vascular identifier to the vascular image to generate a second data structure, the second data structure identifying some vascular portions in the vascular image that are not included in the first data structure; combining the first data structure and the second data structure to generate a combined identification of vascular portions within the vascular image; Equipped with method.

2. the first ML-based vessel identifier is configured to recognize a first anatomically-defined vessel type based on a training input used to generate the ML-based vessel identifier; the second vascular identifier is also an ML-based vascular identifier and is configured to recognize a second anatomically-defined vessel type based on a training input used to generate the ML-based vascular identifier. The method of claim 1.

3. the first anatomically-defined vessel type and the second anatomically-defined vessel type are selected from the group consisting of the LAD, other vessels in the LAD subtree, the LCX, other vessels in the LCX subtree, the RCA, and other vessels in the RCA subtree; The method of claim 2.

4. the first ML-based vessel identifier and the second ML-based vessel identifier are configured to recognize the first anatomically-defined vessel type and the second anatomically-defined vessel type, respectively, using a training input including images and image location representations of the first anatomically-defined vessel type and the second anatomically-defined vessel type, respectively. The method of claim 2.

5. the first data structure includes a definition of a path, the path extending along a vessel segment; the second data structure includes a pixel mask that identifies pixels corresponding to vascular segments; The method of claim 1.

6. The first ML-based vascular classifier comprises a network whose output layer is a recurrent layer. The method according to claim 5.

7. the second vessel identifier is an ML-based vessel identifier, and the output layer is a classification layer. The method according to claim 5.

8. The combining step comprises: identifying a first region and a second region identified by both the first ML-based vessel identifier and the second vessel identifier; selecting for inclusion in the combined identification a third region identified by one of the first ML-based vessel identifier and the second vessel identifier based on a third region extending between the first region and the second region; Including, The method according to claim 5.

9. At least one of the first data structure and the second data structure includes incorrectly identified vessel segments; and The method comprises: determining whether a difference in identification of vascular segments between the first data structure and the second data structure corresponds to a known error propensity of one of the first ML-based vascular identifier and the second vascular identifier; and the combining step includes selectively including identified vessel portions in the combined identification according to identified known error propensities. The method of claim 1.

10. 1. A method for identifying a vascular portion of a vascular image, comprising: accessing a plurality of vascular images including the vascular image; applying an ML-based vascular identifier to the plurality of vascular images to generate a first data structure identifying at least some vascular portions; comparing the first data structure with the vascular image to identify vascular segments; Equipped with method.

11. The plurality of vascular images are each a part of the same time series image. The method of claim 10.

12. each image in the time series images the same anatomical region, and each adjacent image pair in the time series images the anatomical region one second or less apart; The method of claim 11.

13. The ML-based vessel identifier is an ML-based vessel identifier trained on multiple time series images. The method of claim 10.

14. the second data structure omits some vessel portions identified in the first data structure; The method of claim 1.

15. 1. A system for identifying a vascular portion of a vascular image, comprising: A processor and a memory, The memory includes: The processor, accessing the vascular image; applying a first ML-based vascular identifier to the vascular image to generate and store in the memory a first data structure identifying at least some vascular portions within the vascular image; applying a second vascular identifier to the vascular image to generate a second data structure stored in the memory, the second data structure identifying a portion of the vascular image that is not included in the first data structure; combining the first data structure and the second data structure to generate a combined identification of vascular portions within the vascular image and storing it in the memory; The command to carry out Encode, system.

16. the first ML-based vessel identifier is configured to recognize a first anatomically-defined vessel type based on a training input used to generate the ML-based vessel identifier; the second vascular identifier is also an ML-based vascular identifier and is configured to recognize a second anatomically-defined vessel type based on a training input used to generate the ML-based vascular identifier. The system of claim 15.

17. the first anatomically-defined vessel type and the second anatomically-defined vessel type are selected from the group consisting of the LAD, other vessels in the LAD subtree, the LCX, other vessels in the LCX subtree, the RCA, and other vessels in the RCA subtree; 17. The system of claim 16.

18. the first ML-based vessel identifier and the second ML-based vessel identifier are configured to recognize the first anatomically-defined vessel type and the second anatomically-defined vessel type, respectively, using a training input including images and image location representations of the first anatomically-defined vessel type and the second anatomically-defined vessel type, respectively.

17. The system of claim 16.

19. the first data structure includes a definition of a path, the path extending along a vessel segment; the second data structure includes a pixel mask that identifies pixels corresponding to vascular segments; The system of claim 15.

20. The first ML-based vascular classifier includes a network whose output layer is a recurrent layer.

20. The system of claim 19.

21. the second vessel identifier is an ML-based vessel identifier, and the output layer is a classification layer.

20. The system of claim 19.

22. The instructions cause the processor to: identifying a first region and a second region identified by both the first ML-based vessel identifier and the second vessel identifier; and selecting, based on a third region extending between the first region and the second region, the third region identified by one of the first ML-based vessel identifier and the second vessel identifier for inclusion in the combined identification.

20. The system of claim 19.

23. The instructions cause the processor to: determining a condition in which at least one of the first data structure and the second data structure includes an incorrectly identified vessel portion; determining whether a difference in identification of vessel portions between the first data structure and the second data structure corresponds to a known error propensity of one of the first ML-based vessel identifier and the second vessel identifier; selectively combining the first data structure and the second data structure according to identified known error trends; Hereby further orders: The system of claim 15.

24. the second data structure omits some vessel portions identified in the first data structure; The system of claim 15.

25. 1. A system for identifying a vascular portion of a vascular image, comprising: A processor and a memory, The memory includes: The processor, accessing a plurality of vascular images including the vascular image; applying an ML-based vascular identifier to the plurality of vascular images to generate a first data structure identifying at least some vascular portions; comparing the first data structure with the vascular image to identify vascular segments; The command to carry out Encode the system.

26. The plurality of vascular images are each part of a same time series of images.

26. The system of claim 25.

27. each image in the time series images the same anatomical region, and each adjacent image pair in the time series images an anatomical region within one second or less of each other.

27. The system of claim 26.

28. the ML-based vessel identifier is an ML-based vessel identifier trained on a plurality of time series images; 26. The system of claim 25.

29. 1. A method implemented by one or more computer systems, comprising: accessing an image sequence including a plurality of vascular images, the vascular images depicting a portion of the heart from a particular viewpoint, the vascular images associated with different times within a time range; determining a subset of the vascular images associated with a particular cardiac phase, where determining includes computing a forward pass through a machine learning model trained to classify vascular images based on cardiac phase; generating a segmentation mask associated with the subset, the segmentation mask segmenting blood vessels included in each of the blood vessel images forming the subset, a mask score indicative of a size or length associated with the blood vessels included in the segmentation mask being determined for the segmentation mask, and filtering the subset to remove one or more blood vessel images based on the mask score; determining particular vascular images to be included in the filtered subset, said determining being based on an analysis of one or more image quality measures determined for each vascular image included in the filtered subset; Equipped with the particular vascular image is configured for inclusion in an interactive user interface. method.

30. The blood vessel image is an angiographic image.

30. The method of claim 29.

31. The vascular image is a computed tomography image.

30. The method of claim 29.

32. the vascular images included in the image sequence are acquired at a specific frequency and are separated in time according to the specific frequency; 30. The method of claim 29.

33. the image sequence is acquired via a C-arm positioned to obtain the image sequence at a particular viewpoint; 30. The method of claim 29.

34. Determining the subset of cardiac phase images includes: determining whether the image sequence is associated with a left or right side of the heart; The machine learning model is selected from a plurality of machine learning models based on the determination.

30. The method of claim 29.

35. determining whether the image sequence is associated with the right side or the left side is based on the use of different machine learning models, the different machine learning models assigning to each vascular image a value indicative of the likelihood that the vascular image is associated with the left side and the right side; 35. The method of claim 34.

36. The machine learning model is a convolutional neural network, and determining the subset includes: obtaining a threshold number of blood vessel images from the image sequence, the images forming the threshold number being consecutive in the image sequence; providing the threshold number of the vascular images as an input to the machine learning model, the machine learning outputting a classification associated with a particular image within the threshold number; Including, 30. The method of claim 29.

37. the particular image is a median image of the threshold number of the vascular images; 37. The method of claim 36.

38. The particular cardiac phase is end diastole.

30. The method of claim 29.

39. The machine learning model outputs a value indicating the likelihood that the vascular image is in end diastole.

30. The method of claim 29.

40. The machine learning model uses one or more long short-term memory units.

30. The method of claim 29.

41. the segmentation masks are generated based on different machine learning models, the different machine learning models being trained to segment blood vessels depicted in a blood vessel image.

30. The method of claim 29.

42. The different machine learning model is a convolutional neural network.

42. The method of claim 41.

43. The segmentation mask is generated based on the use of a Frangi filter.

30. The method of claim 29.

44. The segmentation mask comprises pixels that are assigned a binary color, a first color indicating that the assigned pixel forms part of a blood vessel and a second color indicating that the assigned pixel does not form part of a blood vessel.

30. The method of claim 29.

45. the image sequence is determined to be associated with a left or right side of the heart, and for the right side, a single segmentation mask is generated for each vascular image included in the subset; 30. The method of claim 29.

46. the single segmentation mask segments blood vessels associated with the right coronary artery; 46. ​​The method of claim 45.

47. the image sequence is determined to be associated with a left or right side of the heart, and for the left side, two or more segmentation masks are generated for each vascular image included in the subset; 30. The method of claim 29.

48. the two or more segmentation masks segment vessels associated with two or more of a left anterior descending (LAD) artery, a left circumflex (LCX) artery, and a left marginal artery; 48. The method of claim 47.

49. The segmentation mask is of blob or path type.

30. The method of claim 29.

50. The output of the machine learning model is an ML-based vascular identifier.

30. The method of claim 29.

51. one or more vessel images included in the subset are discarded based on the segmentation mask, discarding being based on vessels included in the segmentation mask having discontinuities; 30. The method of claim 29.

52. The mask score indicates a length associated with a blood vessel or an area associated with the blood vessel.

30. The method of claim 29.

53. Determining the particular angiographic image includes: registering the vascular images included in the filtered subset, where registering includes aligning features depicted in the vascular images; determining an image quality measure comprising a contrast score of the vascular images included in the filtered subset, the contrast score being indicative of an average grayscale value associated with blood vessels in the vascular images; determining the particular angiographic image based on the contrast score; Including, 30. The method of claim 29.

54. a combined segmentation mask is generated for the vascular images in the filtered subset, and the contrast score compares the contrast of the same blood vessels in the vascular images; 54. The method of claim 53.

55. the distance score is a filling score, and further, determining the filling score comprises determining a volume of pixels forming the blood vessel or determining an estimate of the volume of pixels using the segmentation mask.

54. The method of claim 53.

56. the interactive user interface is configured to allow selection of a vascular image different from the particular vascular image, and the system is configured to update the machine learning model based on the selection.

30. The method of claim 29.

57. the interactive user interface is responsive to user input indicating a contour or location of a blood vessel within a particular angiographic image; 30. The method of claim 29.

58. 58. A system comprising one or more computers and a non-transitory computer storage medium storing instructions that, when executed by the one or more computers, cause the one or more computers to perform the method of any one of claims 29 to 57.

59. A non-transitory computer storage medium storing instructions which, when executed by one or more computer systems, cause the one or more computers to perform the method of any one of claims 26 to 57.

60. 1. A method implemented by one or more computer systems, comprising: accessing an image sequence including a plurality of vascular images, the vascular images depicting a portion of the heart from a particular viewpoint, the vascular images associated with different times within a time range; applying a first machine learning model configured to generate a first set of vascular images, wherein one or more vascular images of the plurality of vascular images are removed to generate the first set, the first set including vascular images associated with end diastole of the heart; applying a second machine learning model configured to generate a second set of vascular images, wherein one or more vascular images of the first set are removed to generate the second set, the second set including vascular images associated with vessels having lengths greater than a threshold percentile; determining a particular angiographic image from the second set based on a contrast score associated with the second set, the contrast score indicating a measure of contrast quality associated with one or more blood vessels depicted in the angiographic image; Equipped with method.

61. the first and second machine learning models are selected based on whether the image sequence depicts a left side or a right side of the heart.

61. The method of claim 60.

62. one or more computers; When executed by the one or more computers, the one or more computers applying a first machine learning model to an image sequence including a plurality of vascular images, the second machine learning model being configured to form a first set of vascular images, wherein one or more vascular images of the plurality of vascular images are removed to form the first set, the first set including vascular images associated with end diastole of the heart; applying a second machine learning model configured to form a second set of vascular images, wherein one or more vascular images of the first set are removed to form the second set, the second set including vascular images associated with vessels having lengths greater than a threshold percentile; determining a particular angiographic image from the second set based on a contrast score associated with the second set, the contrast score indicating a measure of contrast quality associated with one or more blood vessels depicted in the angiographic image; The command to carry out a non-transitory computer storage medium for storing the Equipped with system.

63. the first and second machine learning models are selected based on whether the image sequence depicts a left side or a right side of the heart.

63. The system of claim 62.

64. When executed by a system of one or more computers, the system causes the one or more computers to: applying a first machine learning model to an image sequence including a plurality of vascular images, the first machine learning model being configured to form a first set of vascular images, wherein one or more vascular images of the plurality of vascular images are removed to form the first set, the first set including vascular images associated with end diastole of the heart; applying a second machine learning model configured to form a second set of vascular images, wherein one or more vascular images of the first set are removed to form a second set, the second set including vascular images associated with vessels having lengths greater than a threshold percentile; determining a particular angiographic image from the second set based on a contrast score associated with the second set, the contrast score indicating a measure of contrast quality associated with one or more blood vessels depicted in the angiographic image; A non-transitory computer storage medium that stores instructions for causing a program to execute a program.

65. the first and second machine learning models are selected based on whether the image sequence depicts a left side or a right side of the heart.

65. The computer storage medium of claim 64.

66. 1. A method implemented by one or more computer systems, comprising: accessing a plurality of image sequences each including a plurality of vascular images, the vascular images depicting a portion of the heart from a particular viewpoint, the vascular images associated with different times within a time range; causing presentation of an interactive user interface via a user device, the interactive user interface is responsive to a selection of a particular image sequence, the selection triggering a determination of particular vascular images included in the particular image sequence; Determining the particular vascular image includes: determining a subset of the vascular images in the particular image sequence associated with a particular cardiac phase, the determining including computing a forward pass through a machine learning model trained to classify vascular images based on cardiac phase; generating a segmentation mask associated with the subset, the segmentation mask segmenting blood vessels included in each of the blood vessel images forming the subset, a mask score indicative of a size or length associated with the blood vessels included in the segmentation mask being determined for the segmentation mask, and the subset being filtered to remove one or more blood vessel images based on the mask score; determining the particular vascular images included in the filtered subset, the determining being based on analyzing one or more image quality measures determined for each vascular image included in the filtered subset; Including, Steps and updating the interactive user interface to include the particular vascular image; Equipped with method.

67. the interactive user interface presents an animation or video of the vascular images forming the particular image sequence.

67. The method of claim 66.

68. the particular vascular image is identified in the animation or video presentation; 68. The method of claim 67.

69. 1. A method implemented by one or more computer systems, comprising: accessing a plurality of image sequences each including a plurality of vascular images, the vascular images depicting a portion of the heart from a particular viewpoint, the vascular images associated with different times within a time range; causing presentation of an interactive user interface via a user device, the interactive user interface is responsive to a selection of a particular image sequence, the selection triggering a determination of particular vascular images included in the particular image sequence; Determining the particular vascular image includes: applying a first machine learning model to the particular image sequence, the first machine learning model configured to generate a first set of vascular images, wherein one or more vascular images in the particular image sequence are removed to generate the first set, the first set including vascular images associated with end diastole of the heart; applying a second machine learning model configured to form a second set of vascular images, wherein one or more vascular images in the first set are removed to form the second set, the second set including vascular images associated with vessels having lengths greater than a threshold percentile; determining a particular angiographic image from the second set based on a contrast score associated with the second set, the contrast score indicating a measure of contrast quality associated with one or more blood vessels depicted in the angiographic image; Including, Steps and updating the interactive user interface to include the particular vascular image; Equipped with method.

70. the interactive user interface presents an animation or video of the vascular images forming the particular image sequence.

70. The method of claim 69.

71. the particular vascular image is identified in the animation or video presentation; 71. The method of claim 70.

Images