Neural network device for aneurysm identification, segmentation, and treatment outcome prediction

Abstract

The neural network device receives as input from a user device digital image information and clinical information of the aneurysm patient, and uses a neural network trained for aneurysm outcome prediction to generate an outcome prediction for at least one intrasaccular implant device from a group of treatment devices to be placed in the aneurysm sac identified in the digital image information and predicted to be most likely to completely occlude the aneurysm sac using the digital image information and the clinical information, The device is further configured to output and display on the device the identification of the at least one intrasaccular implant device and each outcome prediction for the at least one intrasaccular implant device.

Description

CROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This patent application claims priority to U.S. Provisional Application No. 63 / 307,467, assigned to the present assignee, entitled "NEW DEEP NEURAL SEGMENTATION NETWORK FOR CEREBRAL ANEURYSMS IN 2D DIGITAL SUBTRACTION ANGIOGRAPHY," which is expressly incorporated by reference herein. [Technical field]

[0002] The present disclosure relates generally to the field of endovascular treatment, and more specifically, but without limitation, certain aspects relate to tools to aid in the treatment of cerebral aneurysms. [Background technology]

[0003] Aneurysms are abnormal localized dilations of arteries caused by weakening of the blood vessel walls, which can rupture. Weak or narrow parts of the cerebral arteries fill with blood and bulge or swell, and when these aneurysms rupture, they can have serious consequences on nerves and brain tissue. Aneurysms tend to occur in areas with impaired blood flow, so cerebral aneurysms often occur at arterial branching points, especially wide-necked bifurcation aneurysms (WNBAs). Some reports suggest that asymptomatic intracranial aneurysms occur in 1-5% of the population. If an aneurysm ruptures, subarachnoid hemorrhage can cause serious health problems, including hemorrhagic stroke, brain damage, coma, and death. If an aneurysm is discovered incidentally, the chances of successful treatment are high.

[0004] Endovascular treatment of cerebral aneurysms has advanced significantly with the introduction of the Guglielmi Detachable Coil (GDC). GDC is a bare platinum coil that is placed within the aneurysmal sac to promote thrombus formation and aneurysm occlusion. The development of flow diversion devices such as the intracranial flow diverter stent (FDS) has further advanced endovascular aneurysm treatment. In 2011, the first product of this type was approved by the US Food and Drug Administration (FDA) (FDA premarket approval: P100018). Flow diverters include finely braided mesh stents that divert blood flow away from the aneurysmal sac. Such devices can also be designed to be placed at the bifurcation of the parent vessel lumen. A variety of innovative devices have changed the range of treatment options. One of these is the Woven EndoBridge (WEB) Embolization System (manufactured by MicroVention, a Terumo group company), which was approved by the FDA at the end of 2018 (FDA premarket approval: P170032). The WEB device is an intrasaccular braided implant that is placed within the aneurysm sac. The WEB device can be used in both ruptured and unruptured aneurysms.

[0005] Successful treatment with intrasaccular devices depends on complete occlusion of blood flow into the aneurysm sac. Therefore, selecting the correct intrasaccular device size for a given aneurysm is a critical step in treatment planning. Therefore, improved aneurysmal sac measurements and improved treatment device sizing are needed. Summary of the Invention

[0006] In view of the above-mentioned problems and unmet needs, the following presents a simplified summary of one or more aspects to facilitate a basic understanding of these aspects. The following summary is not an extensive overview of contemplated aspects. It is not intended to identify key or critical elements of all aspects, nor is it intended to delineate the scope of any or all aspects. Its sole purpose is to present some concepts of one or more aspects in a simplified form as a prelude to the more detailed description that is presented later.

[0007] According to one aspect of the present disclosure, a neural network apparatus, method, and computer readable medium are provided for providing an outcome prediction for an intrasaccular implant device based on an angiographic image and clinical information. The neural network apparatus receives digital image information and clinical information of an aneurysm patient as input from a user device, and generates an outcome prediction for at least one intrasaccular implant device to be placed in an aneurysmal sac identified in the digital image information using a trained neural network pre-trained to perform object classification for aneurysm outcome prediction. The outcome prediction is classified by the neural network based on the likelihood of completely occluding the aneurysmal sac based on the received digital image information and the clinical information. The neural network apparatus is configured to output the identification of the at least one intrasaccular implant device and the respective outcome prediction of the at least one intrasaccular implant device for display on the user device in response to an input from the user device.

[0008] According to another aspect, a system, apparatus, method, and computer readable medium are provided that provide an outcome prediction of an intrasaccular implant device based on image information and clinical information. To illustrate the concept, some angiographic images are illustrated. However, the concept presented herein can also be adopted for other types of digital imaging such as non-angiographic techniques, including but not limited to optical coherence tomography imaging, positron emission tomography imaging, near infrared (IR) spectroscopy, and / or ultrasound-based techniques and image information. The system is configured to receive at least one of image information or clinical information related to an aneurysm patient as input from a user device, and generate an outcome prediction of an aneurysm treatment for the aneurysm patient based on at least one of the image information or the clinical information received as input. The system is configured to transmit the outcome prediction of the aneurysm treatment to the user device for display on the user device.

[0009] According to another aspect, a system, device, method, and computer readable medium are provided for providing semi-automatic or automatic segmentation of an aneurysm sac. The device is configured to receive digital information of a patient (e.g., angiogram information, CT images, etc., among others) as an input from a user device, and segment the aneurysm sac in the digital information using a trained neural network model having an encoder with a convolutional neural network pre-trained for object classification. The device is configured to transmit the aneurysm sac segmentation information to the user device.

[0010] The features and advantages described herein are not intended to be exhaustive. These features are indicative of only a few of the various ways in which the principles of the various aspects may be employed. Additional advantages and novel features of these aspects will be described in part in the specification that follows, and in part will become more apparent to those skilled in the art upon review of the drawings, specification, and claims, and upon practice of the invention. Additionally, it should be noted that the language used herein has been selected primarily for ease of reading and description, and may not have been selected to define or encompass the subject matter disclosed. [Brief description of the drawings]

[0011] The foregoing summary and the following detailed description will be better understood with reference to the accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate several exemplary aspects and implementations, and together with the description, further serve to explain the principles involved and enable those skilled in the relevant art to make and use the disclosed technology.

[0012] [Figure 1A]FIG. 1A illustrates two-dimensional digital subtraction angiography (DSA) images from both anterior-posterior (AP) and lateral perspectives that may be used to measure aneurysm sac height, dome width and neck size in accordance with an embodiment of the present disclosure.

[0013] [Figure 1B] FIG. 1B illustrates an example of an aneurysm treatment device that may be considered with respect to embodiments of the present disclosure.

[0014] [Figure 1C] FIG. 1C illustrates an example of an aneurysm sac with an implanted treatment device.

[0015] [Diagram 2] FIG. 2 is a diagram illustrating an exemplary system including an aneurysm treatment support service according to an embodiment of the present disclosure.

[0016] [Diagram 3] FIG. 3 is a diagram illustrating an exemplary aneurysm treatment support service according to an embodiment of the present disclosure.

[0017] [Figure 4A] FIG. 4A illustrates an exemplary user interface for accepting user input and providing output to a user in accordance with aspects of the present disclosure. [Figure 4B] FIG. 4B illustrates an exemplary user interface for accepting user input and providing output to a user according to aspects of the present disclosure.

[0018] [Figure 4C] FIG. 4C illustrates an exemplary predictive model building workflow for an embodiment of the present disclosure.

[0019] [Diagram 5] FIG. 5 is a diagram illustrating the architecture of a segmentation network proposed in accordance with an embodiment of the present disclosure.

[0020] [Figure 6A] FIG. 6A illustrates an example segmentation performance of a detected aneurysmal sac for an embodiment of the present disclosure. [Figure 6B] FIG. 6B illustrates an exemplary segmentation performance of a detected aneurysmal sac for aspects of the disclosure.

[0021] [Figure 7A] FIG. 7A illustrates an exemplary 3D mesh model of an aneurysmal sac reconstructed from segmented contours according to an embodiment of the present disclosure.

[0022] [Figure 7B] FIG. 7B illustrates an exemplary segmentation performance of a detected aneurysmal sac for aspects of the disclosure.

[0023] [Figure 8A] FIG. 8A is a flow chart illustrating a method of operation of an aneurysm treatment support service according to an embodiment of the present disclosure. [Figure 8B] FIG. 8B is a flow chart illustrating a method of operation of an aneurysm treatment support service according to an aspect of the present disclosure.

[0024] [Figure 9A] FIG. 9A is a flow chart illustrating a method of operation of an aneurysm treatment module according to an embodiment of the present disclosure.

[0025] [Figure 9B] FIG. 9B is a flow chart illustrating a method of requesting and receiving aneurysm treatment assistance at a user device, according to an aspect of the present disclosure.

[0026] [Figure 10] FIG. 10 is a diagram showing a computer system capable of implementing each aspect of the aneurysm treatment support service according to an embodiment of the present disclosure.

[0027] The figures and the following description are intended only to illustrate aspects of the present disclosure. Those skilled in the art will readily recognize from the following description that alternatives to the structures and methods illustrated herein may be used without departing from the principles described herein. Reference will now be made in detail to several embodiments that are illustrated in the accompanying drawings. It is noted that, wherever possible, similar or identical reference numbers may be used in the drawings to indicate similar features. DETAILED DESCRIPTION

[0028] The detailed description set forth below in conjunction with the accompanying drawings is intended to describe various configurations and is not intended to represent the only configurations in which the concepts described herein may be practiced. The detailed description includes specific content for a thorough understanding of various concepts. However, it will be apparent to those skilled in the art that these concepts may be practiced without these specific details. In some instances, well-known structures and components are shown in block diagram form to avoid obscuring such concepts.

[0029] Accurate measurement of aneurysmal sac dimensions is a critical step in the treatment planning of cerebral aneurysms. In some embodiments, imaging information is acquired in the preoperative state. Marking and delineating the contours of the aneurysmal sac may be performed by a neurointerventional radiologist, but this can be time-consuming and expensive, especially in the time-critical context of implant selection. The embodiments presented herein provide a tool that adds additional information to aid in the measurement and implant selection.

[0030] The aspects presented herein provide semi-automated and fully automated methods for segmenting an aneurysm sac based on imaging and / or clinical information input into a neural network or machine learning model. The aspects further allow for automatic identification of a therapeutic device or potential therapeutic devices to be implanted in the aneurysm sac. The aspects also provide automatic predictive results for the identified therapeutic device or devices. The present disclosure provides a decision support software system as a tool to aid in the rapid identification of a promising treatment. The automated detection and / or segmentation, identification of potential therapeutic devices (e.g., including size or model), and / or information regarding predicted outcomes of treatment provide additional tools for the user to consider when developing a treatment plan for the aneurysm. The automated aspects can reduce the time it takes to identify, segment, and identify a therapeutic device, and increase the pace at which treatment can be delivered to patients. The aspects include a neural network or neural network component that allows additional consideration of other historical data, including past aneurysm data, in performing the detection, identification, and segmentation. The feedback aspects allow the model to continually improve the output provided to the user. Combining image data with clinical information allows for more accurate and efficient estimation and prediction, providing users with a more efficient and accurate tool for developing treatment plans.

[0031] Successful treatment with an intrasaccular device, such as an intrasaccular embolization device, depends on complete occlusion of blood flow into the aneurysmal sac. The completeness of embolization is influenced by the size selection of the treatment device. A clinician, such as a neurointerventional radiologist (INR), may perform an angiogram of the vascular complex to measure the major dimensions of the aneurysmal sac. By way of example, the clinician may perform a preoperative digital subtraction angiogram (DSA), an intraoperative DSA, or other imaging systems, such as, but not limited to, optical coherence tomography, positron emission tomography, near infrared spectroscopy, and ultrasound. The clinician may measure the aneurysmal sac height, aneurysmal sac dome width, and aneurysmal sac neck size in both lateral and anterior-posterior (AP) views on the DSA image. FIG. 1A shows an image of an exemplary aneurysmal sac 102, and FIG. 1B shows an exemplary treatment device 104 that may be implanted in the aneurysmal sac. The image of FIG. 1A is a two-dimensional digital subtraction angiography (DSA) image 100 that can be used to measure the height 106, dome width 108, and neck size 110 of the aneurysm sac 102. As an example, a clinician may perform the measurements manually or in multiple perspectives, such as both anterior-posterior and lateral perspectives. The treatment device 104 of FIG. 1B is a braided embolic type device, one of a variety of treatment devices that may be considered for the present disclosure. There are also various models of certain types of treatment devices, each with a maximum height and maximum diameter in the fully open state of the implant model. FIG. 1B shows the height and diameter dimensions of the treatment device 104. The size of the treatment device can be selected based on these measurements to ensure proper fit and operation of the device. The size of the treatment device 104 shown in FIG. 1B is not proportional to the DSA image of FIG. 1A.

[0032] 1C is an exemplary diagram 120 of a treatment device 104 implanted within an aneurysm sac 102 to impede flow into the aneurysm sac. In some embodiments, the treatment devices come in a variety of height and diameter combinations. Table 1 shows example diameter and height measurements for various models. Table 1 only shows a few example dimensions to illustrate the concept, and various diameter and height combinations can be further explored to select a treatment device for a particular aneurysm treatment. [Table 1]

[0033] In some embodiments, the largest two dimensions of the aneurysm sac 102 may be used to determine the diameter / height combination of the treatment device 104 to be implanted into the aneurysm sac 102. DSA may be used to measure the dimensions and characteristics of the aneurysm and as a step to plan the intervention. Mis-sizing the aneurysm sac may result in an incomplete treatment due to the use of an intrasaccular implant device that is too small or too large to effectively treat the aneurysm. Intrasaccular flow occluders, such as the treatment devices shown in Figures 1B and 1C, treat aneurysms by diverting blood flow from the aneurysm sac. Following intervention with implantation of the treatment device 104, residual blood flow to the aneurysm sac may be reduced by 100%.

[0034] To avoid undersizing the implanted treatment device 104, the dimensions of the aneurysm sac can be measured on a working projection DSA image showing the maximum dimensions of the sac before the procedure. The embodiments presented herein provide an automated or semi-automated, reliable and fast algorithm to segment the aneurysm sac and / or identify possible treatment devices with a prediction of the treatment outcome, which can reduce the measurement time and improve the accuracy of implant selection. For example, the embodiments presented herein can be useful to improve the selection of a treatment device 104 that minimizes the residual blood flow to the aneurysm sac after the treatment device 104 is implanted. The embodiments presented herein further provide a transfer learning based tool to train a deep neural network for high accuracy segmentation to contour the sac of a wide neck bifurcation aneurysm in a more efficient and reliable manner, identify devices for implantation, and provide outcome predictions for the identified devices based on image data and / or clinical information.

[0035] FIG. 2 illustrates an exemplary system 200 that provides tools to aid in planning aneurysm treatment, including treatment device selection using automated or semi-automated, reliable, and efficient algorithms for segmenting the aneurysm sac and / or identifying potential treatment devices along with prediction of treatment outcome. FIG. 2 illustrates an aneurysm treatment module 220 (also referred to as an aneurysm treatment planning tool) configured to receive patient imaging and / or clinical information from one or more user devices 204 and perform automated measurements thereon and / or provide predictive information thereon. Receipt of imaging and / or clinical information may be received by the module 220 via a user data interface component 222, which may be a communications interface enabling data transfer between one or more user devices 204 and the aneurysm treatment module 220. FIG. 10 illustrates a computer system 1000 that may implement aspects of the aneurysm treatment module, illustrating various examples of communications interfaces that may be used to receive patient imaging and / or clinical information.

[0036] 2 further illustrates a communication system that can be used in the present invention. The communication system includes one or more users (also referred to herein interchangeably as one or more "users") who access the communication system via one or more user devices (also referred to as terminals) 204. One embodiment of the data used in the embodiments presented herein may be entered and / or accessed by a user via a user device 204, such as a personal computer (PC), a minicomputer, a mainframe computer, a microcomputer, a server, a telephone, a tablet, or a wireless device such as a personal digital assistant ("PDA"), a handheld wireless device, or an image and measurement acquisition device. The user device 204 may be coupled to a server 212, such as a PC, a minicomputer, a mainframe computer, a microcomputer, or other device with a processor and repository of data and / or a connection to the repository of data, via a network 218, such as the Internet or an intranet, and a coupling 245. The coupling 245 may include, for example, a wired, wireless, or fiber optic link. In some aspects, the user device 204 may have a communication interface 246, connection, or communication link with the aneurysm treatment module 220, for example, via the application 217. In some aspects, the methods and systems presented herein operate in a standalone environment, such as a single terminal. For example, in some aspects, the aneurysm treatment module 220 may be used as a component within the user device 204 or as part of an image acquisition or measurement device as the exemplary user device 204 may be used by a clinician to obtain image or measurement information for a patient.

[0037] The module 220 may include a query component 224 configured to receive a user query for measurement and / or predicted results. In some aspects, once the patient's imaging and / or clinical information is received, the module 220 may provide automatic measurement or predicted information to the user device, e.g., without further query. In other aspects, the module 220 may be configured to return measurement or predicted information to the user device in response to a particular user query. FIG. 4A illustrates an exemplary user interface 400 that may be presented on a user device, which allows the user device to submit or input the patient's imaging or clinical information, e.g., via a submission component 402, and request measurement or predicted information from the module 220. As an example, the user device 204 may send a request or query for an automatic or semi-automatic measurement based on the imaging information, e.g., based on the user selecting a segmentation request option 404 presented to the user in the user interface 400. The request may be received via the query component 224. In response to the request, the module 220 may perform an automated measurement and transmit the measurement information to the user device 204. As another example, the user device 204 may transmit a request to the module 220 for an identification of a treatment device and / or a predicted outcome of treatment with the treatment device, e.g., based on a user selecting a device identification request option 406 or an outcome prediction request option 408 presented to the user in the user interface 400. In response to the request, the module 220 may perform an automated measurement and transmit to the user device 204 an identification of one or more possible treatment devices (e.g., type and / or size), with or without predicted treatment outcome information.FIG. 4B illustrates an exemplary user interface showing exemplary measurement and prediction information that may be provided, for example for display, in a user interface 425 of the user device 204 based on information generated and transmitted by module 220.

[0038] The module 220 may reside remotely on one or more servers connected to the network 218. For example, the module 220 may reside on the server 212 and / or other servers on the network 218, and the user device 204 may send information and / or requests to and receive output from the module 220 via the network 218. In another aspect, the module 220 may be configured within or directly coupled to the user device 204.

[0039] Module 220 may include a neural network component 226, also referred to as a neural network component, artificial intelligence component, etc. Neural network component 226 uses predictive models to provide measurement information, treatment device identification, and / or predicted treatment outcomes for a patient based on, for example, the patient's clinical background and imaging information, which may be received via user data interface component 222. Table 2 provides various examples of clinical information that may be received as user input. [Table 2]

[0040] The pre-existing conditions may include information regarding the patient's existing clinical conditions, which may be grouped by different body systems and may be grouped or entered in different ways, as shown in the example of Table 2. Additionally, clinical information related to the pre-existing conditions may be further grouped or entered based on grouping. As an example, under the cardiovascular and circulatory grouping, the health conditions may include hypertension, coronary artery disease, valvular disease / dysfunction, hypotension, arrhythmia, myocardial infarction, angina, or heart failure.

[0041] The measurement information may include various measurements from one or more perspectives. As an example, the measurement information input may include height, width, and / or dome measurements in anterior-posterior and lateral perspectives from a DSA image. The image data may include one or more perspectives from various types of image sources. The image data may be input based on, among other examples, magnetic resonance imaging (MRI), magnetic resonance angiography (MRA), or computed tomography (CT) scans, etc. As presented herein, cerebral arteriography information may be used to confirm the presence of an aneurysm and evaluate which treatment method is most suitable for a particular aneurysm based on the size, shape, and location of the aneurysm. In some embodiments, the image data may include two-dimensional (2-D) DSA images. The images may be pre-operative image information. The images may be acquired in an operating room, for example, prior to implanting an intra-saccular device for the treatment of the aneurysm. In some embodiments, the image data may include three-dimensional (3-D) data, such as an axial slice stack reconstructed from a series of three-dimensional DSA images generated by a rotational C-arm angiogram. Other imaging examples include slice stacks (axial, sagittal, coronal), magnetic resonance angiography (MRA) with contrast, and computed tomography angiography (CTA) (such as DynaCTA and VasoCTA). The models presented herein may be trained to use any type of image information, such as, for example, two-dimensional images, axial slice stacks reconstructed from a series of three-dimensional DSA images, etc. In some embodiments, the image data information may be raw image information that has not been marked, cropped, trimmed, or annotated by the user to guide the segmentation, for example. In another embodiment, the user may provide some additional input to the image information. For example, the user may select a rectangular region to be analyzed. The user may mark, crop, annotate, or otherwise adjust the image data before it is received by the system.

[0042] As an example of annotations of image data input for a particular patient or as part of the training data for training a neural network model, annotations may be included in the two-dimensional or three-dimensional images to indicate the dimensions and / or contours of the aneurysm sac. For example, annotations may be added to image data as shown in FIG. 1A to indicate the height, width, and / or dome length of the aneurysm region in the image information. As another example, annotations may be shown in multiple orthogonal DSA projections, such as anterior-posterior and lateral projections, to indicate multiple dimensions for analysis (e.g., six dimensions for two projections). An additional measurement may be annotated to indicate the area of ​​the aneurysm sac. The area may be calculated, for example, by a clinician marking the contour of the aneurysm. As with the input for a particular patient, these measurements may be included in the dataset of image information as training data for training a neural network component for automatic segmentation and measurement and / or automatic identification and detection of the aneurysm sac.

[0043] In some embodiments, the annotation information may include annotations indicating the shape, size, and / or branching angle of the vessel. As an example, the annotations may indicate the diameter of the daughter branches and the parent vessel. The annotations may indicate the angle between each daughter vessel (e.g., dominant and non-dominant vessels) and the parent vessel. The annotations may include or be based on markings inserted by the clinician into the image. FIG. 1C illustrates exemplary markings 126 and 128 and angle information 122 and 124 that may be annotated by the clinician into the image. FIG. 1A illustrates exemplary measurement information 132, 134, and 136 that may be annotated by the clinician into the image. As an example, the angle measurements may include, for example, the angle between the parent vessel and the left daughter vessel at 122 and the angle between the parent vessel and the right daughter vessel at 124, as well as the normalized variation of the angle. The normalized variation of the angle corresponds to the normalized left angle of the vessel=(left angle of the vessel) / 180. The diameters of the parent vessel, left daughter branch, and right daughter branch, indicated by reference numeral 126, may be marked or measured. The larger daughter branch may be indicated and the ratio of such measurements may be included as annotations.

[0044] In some embodiments, as described above, the image data may include three-dimensional image data, such as an axial slice stack reconstructed from an imaging study. The system may receive clinician annotations indicating contours around the aneurysm sac in images in the stack in which the aneurysm sac is visible. The annotations may be used to reconstruct a volumetric mesh model of the aneurysm sac and estimate the volume and surface area of ​​the sac.

[0045] In some embodiments, one or more lateral perspective measurements may be omitted for image data, whether as input for a particular patient or as training data. In such embodiments, aggregate measurements may be calculated from anterior-posterior perspectives and subsequently displayed. As an example, as three-dimensional features, the surface area and volume of the aneurysmal sac may be calculated from the three-dimensional mesh model. The non-sphericity index (NSI) and / or intra-weekly ratio (IPR) may be calculated from the three-dimensional mesh model.

[0046] Manually annotating images to calculate image features may not be possible or may require additional time to perform. According to aspects presented herein, tools are provided that allow and improve the automation of the calculation of measurements, e.g., 2D and / or 3D measurements of area and volume, based on image data entered into the service. Automated measurements can be performed efficiently and accurately, shortening time to treatment. In some aspects, the step of delineating the aneurysm boundaries may be performed by, for example, a neurointerventional radiologist. Delineation steps, which are performed in the time-critical situation of implant selection, are time-consuming and expensive. The tools presented herein save the physician time and provide additional auxiliary information useful for measurements and implant selection.

[0047] For example, in some embodiments, the neural network component 226 may include a segmentation component 230 configured to output semi-automated measurement information of the aneurysm sac based on input imaging study information for the patient. In some embodiments, the neural network component 226 may include an identification component 236 configured to output automatic or semi-automated identification information of the aneurysm sac based on input imaging study information for the patient. The neural network component 226 may include an identification component 236 configured to perform aneurysm detection and segmentation, and the segmentation component 230 may perform image segmentation by user input, such as cropping, marking, annotating, or otherwise specifying image portions for aneurysm segmentation. The detection or identification of the aneurysm, together with the measurement or segmentation, may be referred to as an automatic measurement. As presented herein, a neural network or machine learning algorithm may be trained and configured to automatically segment aneurysm regions from image data and use the measurements as features in a predictive modeling function. In some embodiments, deep learning based algorithms may be used in the neural network component 226 as part of the segmentation component 230 and / or the identification component 236. In some embodiments, different deep learning based algorithms may be provided for two-dimensional image information and three-dimensional image information. segmentation

[0048] The process of marking the aneurysmal sac on a DSA image involves first locating the sac in the image and then carefully marking the contour of the sac. Clinicians can intuitively and quickly complete the first step because they have prior knowledge of anatomy and vessels. Meanwhile, for automatic segmentation of lesions, the "detection" step can be a big challenge because the vessels and aneurysmal sac may be in the same intensity range. Two different models may be used in this task: (1) a segmentation model, or (2) a detection + segmentation model.

[0049] The segmentation model may receive input from image information of a limited region, such as a DSA image containing an aneurysmal sac. In semi-automatic segmentation, a pre-processing step may create a rectangular region contained in the received image information. For example, a clinician, such as a radiologist, may provide minimal input to the image information, such as heuristic padding around the aneurysmal sac. For example, image information marked, annotated, or adjusted by the clinician may be received as input to the segmentation network. Detection + Segmentation

[0050] In the detection+segmentation model, images may be received as input without any manipulation or annotation by the clinician, and the neural network model may provide a fully automated method of aneurysmal sac detection and segmentation. In the detection+segmentation model setting, the algorithm may implicitly learn or identify the location of aneurysms in the images and then perform segmentation on the identified aneurysms. Device Identification / Sizing

[0051] The neural network component 226 may include a treatment device identification component 232 configured to output an identification of one or more treatment devices having better predicted outcomes than other treatment devices. The treatment device may include, for example, one of a variety of intrasaccular embolic devices. As an example, the service may return to the user device a treatment device size, or a subset or list of treatment device sizes, that is predicted to be more likely to fully occlude the aneurysm sac based on the neural network model. The neural network component 226 may include an outcome prediction component 234 that outputs predicted treatment outcomes for one or more treatment device sizes. Additional aspects that may be incorporated into the neural network component are described with reference to FIGS. 3, 4C, and 5.

[0052] The embodiments presented herein may identify a size of the flow interrupting implant device, such as a size of a WEB implant. The embodiments presented herein may provide a prediction for one or more flow interrupting implant devices that have a higher predicted outcome of total occlusion compared to other analyzed implant devices based on the received information, which may be said to have the highest predicted probability of total occlusion compared to other possible implant devices. In other embodiments, the embodiments presented herein may include analyzing and identifying the type and size of the aneurysm treatment device from multiple types of aneurysm treatment devices using neural networks or machine learning based on imaging and / or clinical information for a particular patient. Examples of different aneurysm treatment devices include coil embolization treatment devices, coil assisted stents, other types of stents, flow interruption treatment devices, flow diversion treatment devices, and other treatment device types. There are device models of different sizes for each type of device, and each can be analyzed according to a neural network model to predict the treatment outcome when these different sizes of treatment devices are placed in the aneurysm. Prediction of results

[0053] A machine learning or neural network model may be constructed using clinical and / or imaging features to predict treatment outcomes. To illustrate the concept, as an example, the system may predict the outcome of treatment with an intrasaccular embolic device for a wide-necked bifurcation aneurysm. As shown in FIG. 4B, the outcome prediction component 234 may provide predicted outcomes for various types and / or sizes of devices for display to the user. The outcome prediction component 234 may suggest to the user the size of the intrasaccular device that has the highest chance of success, e.g., the highest chance of total occlusion. The outcome prediction component 234 may suggest a size and / or type and may include the chance of total occlusion associated with the proposed size.

[0054] The Raymond-Dauroy occlusion classification is a classification system that includes Class I, which corresponds to complete occlusion of the aneurysm sac, Class II, which corresponds to a residual neck of the aneurysm sac, and Class III, which corresponds to a residual aneurysm. The outcome of the aneurysm occlusion may be observed after the treatment device is placed in the aneurysm, for example, one year after placement. In some embodiments, the predicted outcome may include a predicted percentage of complete occlusion. In some embodiments, the predicted outcome may include a predicted classification (e.g., a predicted classification of complete occlusion, partial occlusion, and residual neck). In some embodiments, the predicted outcome may include a predicted percentage of one or more classifications (e.g., a predicted percentage of complete occlusion, a predicted percentage of partial occlusion, and / or a predicted percentage of residual neck). The outcome of an aneurysm treatment may be influenced by various factors. As described with respect to FIG. 1C, the size of the treatment device (e.g., dimensions such as diameter and height) may affect the outcome of the treatment. Additionally, the relative position of the parent and daughter vessels in a bifurcated aneurysm may affect the hemodynamics to the sac region and therefore the rupture of the aneurysm. Other clinical information, such as history of neurological disease, gender, smoking history, etc., may be associated with increased incidence of aneurysms. In some embodiments, retrospective studies evaluating clinical outcomes of aneurysm treatment device implants may be used to train machine learning models for automated prediction of outcomes for various treatment devices and treatment device sizes.

[0055] The module 220 may include a training component 228 configured to train and / or update the neural network model applied by the neural network component 226. The training component 228 may be configured to perform, for example, any of the aspects described in connection with the model training component 328 of FIG.

[0056] Various feature selection techniques may be employed as part of the neural network component 226 and / or the training component 228. As an example, statistical co-occurrence and / or information gain may be used for feature selection. Co-occurrence based techniques may be used to identify relevant health conditions from a variety of possible health conditions that may be included as image data and / or clinical information regarding the outcome. The selected health conditions may be chosen as determinants of the outcome based on their different prevalence in the complete occlusion or partial occlusion classes. The ratio of the co-occurrence rate to the total number of cases of a particular health condition class. As an example, the hypertension / complete occlusion ratio corresponds to 28 / 49=0.57, and the hypertension / partial occlusion ratio corresponds to 2819 / 32=0.59. The migraine / complete occlusion ratio corresponds to 28 / 49=0.057, and the migraine / partial occlusion ratio corresponds to 14 / 32=0.44. Health conditions with ratios below a threshold may be filtered. For example, health conditions with ratios below 0.30 are filtered from consideration. The threshold may be based on input from a clinician. The health conditions remaining after filtering correspond to candidate conditions for the classification task. To select discriminatory conditions between two classes from the candidate conditions, the absolute difference between the prevalence ratios from the two classes can be calculated. Then, the conditions with the difference greater than or equal to 0.10 can be kept. As an example, migraine can be selected with an absolute difference of 0.13 between the two ratios (|0.57-0.11|), and hypertension can be filtered with an absolute difference of 0.02 between the two ratios (|0.57-0.059|). Information gain can be used as a metric to rank features. Information gain metrics can rank features independently for their class separability. Features can be differently ranked when used in a set for classification. To discriminate such features while keeping the top features with information gain greater than 0.15, various combinations of other clinical and imaging features can be added in a greedy manner based on weighted F1 scores, for example in a 10-fold cross-validation.

[0057] FIG. 4C illustrates an exemplary model building workflow 450, which includes receiving image data at 452, which may include two-dimensional and / or three-dimensional image data. The image data may include different types of image data, for example, as shown at 451, 452, 455. The data may be annotated or unannotated. At 454, extraction of measurements occurs. In some embodiments, the measurements may be extracted from annotations or calculated based on annotations. In other embodiments, the measurements may be based on the image data itself without annotations. The measurement information and clinical background and medical history information 456 are input to a classification model 458. Various different types of classification algorithms or combinations of clinical and image features may be employed to obtain an output of a likelihood of total occlusion at 460. The results may be based on various outcome classifications, such as classification of partial occlusion versus total occlusion. Various types of classification algorithms may be used in the neural network component 226 or the training component 228. As an example, any combination of random forest algorithms, multi-layer perceptron (MLP) neural networks, logistic regression, naive Bayes, or support vector machines (SVM) may be implemented. One or more algorithms may be selected and incorporated into the neural network component 226. As an example, the selection of algorithms is based on 10-fold cross-validation and performance measures including, for example, F1 score, sensitivity, specificity, and ROC.

[0058] The embodiments presented herein allow for multiple models and different combinations of clinical and imaging features. As another example, feature set A may include all clinical features and preoperative imaging features and may be used to develop a baseline model. Feature set B may not include imaging features, but instead may include a portion of clinical features selected to maximize class separability, for example as described above. Feature set C may include selected clinical features from feature set B and may add selected imaging features, such as from 2D DSA images, calculated directly from clinical annotations. Feature set D may include similar features as feature set C and may additionally include aneurysmal sac volume with respect to measurements extracted from annotations on 3D images. Feature set E may replace contour measurements from 2D and volume, surface, and IPR from 3D features with equivalent values ​​calculated from automatically drawn contours obtained using neural networks. Feature set E may allow for evaluation of the automatic measurements. The clinical features are the same for feature sets B, C, D, and E. Feature set F allows for the investigation of the effect of excluding clinical features and may include image features from feature set D excluding clinical features. Feature set G may similarly include image features from feature set E excluding clinical features and may use three-dimensional images automatically calculated using a deep neural network segmentation model. Different classification algorithms may be used with different feature sets to compare and evaluate performance to determine the algorithm and feature set to employ in the neural network component 226 and / or training component 228. Comparisons may be made based on one or more of accuracy, specificity, sensitivity, F1 score, weighted F1 score, F1 score for complete occlusion, F1 score for partial occlusion, and / or ROC. Statistical comparison of prediction results for different feature sets may indicate the effectiveness of various feature set and algorithm combinations. As another example, a random forest classifier may be used in conjunction with any combination of feature sets. As further described in conjunction with FIG. 3, the model is trained and updated as additional training data and feedback are provided.The algorithms and / or feature set combinations may be modified based on updated data and feedback.

[0059] FIG. 3 illustrates a schematic diagram 300 of an aneurysm treatment module 302 including a neural network 306, machine learning component, or artificial intelligence component configured to provide automated or semi-automated segmentation or treatment outcome prediction for one or more users. In the embodiments described herein, machine learning can be accomplished using, for example, a neural network. The aneurysm treatment module 302 may have similar or identical functionality to, for example, the aneurysm treatment module 220 of FIG. 2. The aneurysm treatment module 302 may receive information 314 from a user device 304 that provides one or more of an automated or semi-automated measurement of the aneurysm sac, an identification (e.g., type and / or size) of a treatment device from multiple possible treatment devices, and / or a predicted outcome for one or more aneurysm treatment devices for a particular aneurysm patient based on imaging and / or clinical information. The output may be generated based on the machine learning model, neural network, or artificial intelligence.

[0060] FIG. 3 also illustrates that the aneurysm treatment module 302 may receive information 314 from multiple user devices of various types. The information 314 may be based on various different patients. The information from the user devices is input to the model estimation component, which may include the neural network 306. The information may include imaging data, which may include one or more types of images of the patient's aneurysms, and may include measurement information obtained from the patient's images. In some embodiments, the imaging information may include images that have been marked or annotated by a clinician. In some embodiments, the imaging information may include images without annotation, also referred to as raw images. The information 314 may include clinical information of the patient, such as demographic information (e.g., including age, sex, height, weight, race, among other demographic information), aneurysm information (e.g., including location, side, type, detection method, type, or score, among other demographic information), patient allergies, medications taken by the patient, and the patient's medical history (e.g., smoking history, substance abuse, affected body systems, among other).

[0061] Table 2 illustrates various, non-limiting examples of clinical information that may be provided for a particular patient at 314. In some embodiments, the information 314 may include a query or request for information from the user device 304. An automated or semi-automated segmentation or measurement of the aneurysm may be requested. The request may identify an optimal subset of treatment devices, treatment device models, or treatment device sizes. The request may be for a prediction of treatment outcome for one or more treatment devices. The aneurysm treatment module 302 may include any of the components described with respect to FIG. 2, including, for example, the user data interface component 222, the query component 224, the neural network component 226, the training component 228, the identification component 236, the segmentation component 230, the treatment device identification component 232, and / or the outcome prediction component 234.

[0062] The neural network 306 may receive user input information 314 via an input component 318 and may output measurement information for the aneurysm, an identification of one or more aneurysm treatment devices from a group of treatment devices, and / or a predicted outcome for the identified one or more aneurysm treatment devices. The output 316 may be provided to a corresponding user device via an output component 320, presented to the user as a displayed response to a search query in a user interface, sent to the user in a message, or provided to the user in some other visual or audio indicator that identifies or flags information about the content at the user device 304.

[0063] 3, the aneurysm treatment module 302 may receive additional information and data 317 from a variety of different sources 305. For example, the model training component 328 may receive training data that may include a set of data associated with angiographic images of various classifications (e.g., total occlusion, residual aneurysm, residual neck, etc.). The data may be of patients with various clinical information, including, for example, different demographic information. Aneurysm information, clinical information, and / or outcome information associated with the image data may be input as training data. The training data may be obtained from other sources.

[0064] The model estimation component or neural network 306 may include an image classification neural network, which may include any of a convolutional neural network (CNN), a recurrent neural network (RNN), a transfer learning model, and / or a multiplayer perceptron neural network, among others.

[0065] As an example, the EfficientNet may be pre-trained on millions of different object images to provide thousands of object classifications. Although the various objects are different from aneurysms, the classification data may be received as input to aid in the model's identification, segmentation, size prediction, and / or outcome prediction for aneurysms. The model training component 328 may include a data collection component that receives, acquires, and / or prepares data for model training. Data preparation may include data pre-processing, cleaning, formatting, and conversion. In some aspects, the data received via the model training component may include inferred data that is provided as input to the neural network 306. The obtained data may be used as training data to train the neural network 306 to provide an output to the user device 304. The model training component 328 may perform machine learning, or neural network training, validation, and testing. Model performance metrics may be generated as part of the model testing procedure. The model training component 328 may deploy or update the trained, validated, and tested models to the neural network 306 and may receive feedback regarding the performance of the neural network 306.

[0066] In some aspects, one or more user devices 304 may provide feedback 319 to the aneurysm treatment module 302. For example, the user device 304 may provide feedback regarding the treatment device selected and / or placed within the aneurysm sac. The user device 304 may provide feedback regarding treatment results, user evaluation feedback, and the like.

[0067] FIG. 3 illustrates that an exemplary neural network 306 may include a network of interconnected nodes. The output of one node is connected as an input to another node. The connections between nodes may also be referred to as edges, and weights may be applied to the connections / edges to adjust the output from one node to be applied as an input to another node. The nodes may apply thresholds to determine if or when to provide an output to a connected node. The output of each node may be calculated as a nonlinear function of the sum of the inputs to the node. The neural network 306 may include any number of nodes and any type of connection between the nodes. The neural network 306 may include one or more hidden nodes. The nodes may be aggregated into layers, and different layers of the neural network may perform different types of transformations on the inputs. A signal may travel from an input in a first layer through multiple layers of the neural network to an output of a final layer of the neural network, traversing the layers multiple times. As an example, the system may input information to the neural network 306 from 318 and receive or obtain an output (e.g., from 320). The output is provided to a user device 304 .

[0068] In some aspects, the neural network 306 may receive content and use machine learning algorithms, deep learning algorithms, neural networks, reinforcement learning, regression, boosting, and / or advanced signal processing methods to identify content that is of interest to a particular user.

[0069] In some embodiments, unsupervised learning may be used to train the neural network 306. For example, an unsupervised neural network may learn a representation of the input that corresponds to important features of the input distribution. The neural network may use algorithms, such as clustering or distribution algorithms, to analyze the data and discover patterns or clustering of the data without user intervention (e.g., when analyzing training data that does not have additional labels or categories attached). For example, a deep learning algorithm may identify conclusions or patterns from a set of unlabeled data by implicitly learning the distribution function of the observed data.

[0070] Reinforcement learning is a type of machine learning that aims to maximize outcomes by choosing optimal actions in an environment. Reinforcement learning is a machine learning framework. Other frameworks include supervised learning and unsupervised learning. Basic reinforcement can be modeled as a Markov decision process (MDP) that includes a set of environment states, agent states, and a set of agent actions. A decision can be made about the likelihood of a state transition based on the actions and outcomes after the state transition. The action selection by the agent can be modeled as a policy. Reinforcement learning can enable the agent to learn an optimal or near-optimal policy that maximizes outcomes. Supervised learning can involve learning a function that maps inputs to outputs based on example input-output pairs (sometimes called training examples) that can be inferred from a set of training data. Supervised learning algorithms provide algorithms that analyze the training data and map it to new examples.

[0071] Regression analysis may involve statistical analysis to estimate the relationship between a dependent variable (e.g., an outcome variable) and one or more independent variables. Linear regression is an example of regression analysis. Non-linear regression models may also be used. Regression analysis may involve estimating or determining causal relationships between variables in a data set.

[0072] Boosting includes one or more algorithms for reducing variance or bias in supervised learning. Boosting may include iterative learning based on a weak classifier (e.g., somewhat correlated with the true classification) with a distribution added to the strong classifier (e.g., more closely correlated with the true classification) to transform the weak classifier into a stronger classifier. Data weighting may be re-adjusted throughout the process, for example, with respect to accuracy.

[0073] In particular, examples of machine learning models or neural networks that may be included in the aneurysm treatment module 302 may include, for example, artificial neural networks (ANNs), decision tree learning, convolutional neural networks (CNNs), deep learning architectures such as where the output of a first layer of neurons is the input to a second layer of neurons, support vector machines (SVMs) that include a separating hyperplane (e.g., decision boundary) to classify data, regression analysis, Bayesian networks, genetic algorithms, deep convolutional networks (DCNs) with additional pooling and normalization layers, and deep belief networks (DBNs).

[0074] FIG. 3 illustrates an example of a machine learning model, such as an artificial neural network (ANN), that includes a set of artificial neurons (e.g., neuron models) interconnected as nodes. The connections of the neuron models can be modeled as weights in some embodiments. Machine learning models, such as the example of FIG. 3, can provide predictive models, adaptive control, and other applications by training through a set of data. Machine learning models can be adapted, for example, based on external or internal information processed by the machine learning model. In some embodiments, machine learning models can include nonlinear statistical data models and / or decision-making models. Machine learning can model complex relationships between input data and output information.

[0075] A machine learning model may include multiple layers and / or operations, which may be formed by concatenating one or more of the referenced operations. Examples of operations that may be included include extraction of various features of the data, convolution operations, fully connected operations that may be activated or deactivated, compression, decompression, quantization, flattening, etc. The term layer may also refer to operations on input data. The weights, biases, coefficients, and operations may be adjusted to achieve an output that is close to a target output. Weights and biases are examples of parameters of a trained machine learning model. Different layers of a machine learning model may be trained separately.

[0076] Machine learning models may include various connection patterns, for example, feedforward networks, hierarchical layers, recurrent architectures, feedback connections, etc. The connections of the layers may be fully connected or locally connected. In a fully connected network, the neurons of the first layer may transmit outputs to each neuron of the second layer. Each neuron of the second layer may receive inputs from each neuron of the first layer. In a locally connected network, the neurons of the first layer may connect to a subset of the neurons of the second layer, but not to each neuron of the second layer. Convolutional networks may be locally connected and may consist of shared connection strengths for the inputs of each neuron of the second layer. In a locally connected layer of a network, each neuron of the layer may have the same or similar connection patterns, but with different connection strengths.

[0077] A machine learning model, artificial intelligence component, or neural network can be trained, such as training based on supervised learning. During training, the machine learning model can be provided with inputs that the model uses in its calculations to generate an output. The actual outputs are compared to target outputs, and the differences can be used to adjust the parameters (e.g., weights, biases, coefficients, etc.) of the machine learning model to provide an output that is closer to the target output. Before training, the output may be incorrect or less accurate. The difference between the output and the target output can be used to adjust the weights of the machine learning model to make the output closer to the target.

[0078] The learning algorithm can calculate a gradient vector to adjust the weights. The gradient can indicate how much the output will increase or decrease from the target output if the weights are adjusted. The weights, biases, or coefficients of the model can be adjusted until the achievable error rate no longer decreases or until the error rate reaches a target level.

[0079] FIG. 5 illustrates an exemplary architecture 500 of a segmentation network according to the systems and methods described herein. This architecture can be incorporated into the neural network component 226 of FIG. 2 and FIG. 10, as well as the aneurysm treatment module 302 of FIG. 3. The architecture may include an encoder-decoder structure. As an example, the encoder function may change a representation or image (e.g., an image of an aneurysm) into a code in a latent space, and the decoder may construct an output based on the code. Fully convolutional deep neural networks, such as UNet and MNet, can be used for segmentation of anatomical structures in medical images. UNet may include an encoder that acts as a feature generator, followed by a decoder that creates a region mask of equal size to the input image through a deconvolution layer. The embodiments presented herein further incorporate transfer learning from pre-trained networks, such as ImageNet, in models for segmentation.

[0080] In one example shown in FIG. 5, an encoder 502 may be configured to process images 501 or imaging exam information from one or more types of imaging devices, and a decoder 504 may determine a segmentation (e.g., measurement information) representative of an aneurysm sac based on the code provided by the encoder, as shown at 545. A U-net with encoder and decoder functions may be configured to perform segmentation on the images. The encoder arm of the U-net structure may include a convolutional neural network trained for classification of a wide variety of objects. FIG. 5 illustrates the incorporation of a pre-trained (e.g., sometimes referred to as pre-trained) convolutional neural network in the encoder 502. The convolutional neural network incorporated in the encoder function may be pre-trained (e.g., classification or object classification), which may improve the segmentation results provided by the decoder 504. The pre-trained convolutional neural network may be trained with a number of images, for example, different from the images used in training the model to identify various types of objects in different categories. In particular, a family of convolutional neural networks includes EfficientNet. As an example, EfficientNet-b0 is a convolutional neural network trained on millions of images from an image database with categories including many different types of objects, e.g., common household objects, inanimate objects, animals, etc., to provide object classification. For example, the weights may be trained on the JFT-300M data set, which contains 300 million images labeled with 18291 categories. Incorporating a pre-trained convolutional neural network for classification can improve aneurysmal sac segmentation and measurement. In some embodiments, the neural network may be run without pre-trained weights.

[0081] FIG. 5 illustrates an encoder block (e.g., 502) of a UNet architecture that includes a pre-trained EfficientNet-B0 layer, and a decoder block (e.g., 504) that may include a deconvolution block trained from scratch. The figure shows a skip connection 506 between the encoder block (e.g., 502) and the decoder block (e.g., 504). In the example illustrated in FIG. 5, the encoder block (e.g., 502) includes a 3x3 convolution block 508, a 3x3MB convolution block 510, a 2x3x3MB convolution block 512, a 2x5x5MB convolution block 514, a 3x3x3MB convolution block 516, a 3x5x5MB convolution block 518, a 4x5x5MB convolution block 520, and a 3x3MB convolution block 522. A 3x3 decoder block 524 may couple the encoder 502 and the decoder 504. The decoder block (e.g., 504) includes a concatenation block 526, a 3x3 decoder block 528, a concatenation block 530, a 3x3 decoder block 532, a concatenation block 534, a 3x3 decoder block 536, a concatenation block 538, a 3x3 decoder block 540, a 3x3 deconvolution block 542, and a 3x3 activation block 544. In the illustrated example, the 3x3 decoder block 524 forms a bottleneck layer between the encoder block (e.g., 502) and the decoder block (e.g., 504).

[0082] In one implementation, the deep learning model is based on the PyTorch Python package, which implements the network architecture. To illustrate the concept, as an example, each fold can be trained for 20 epochs using 16 image batches with an input image size of 1024x1024x3. The initial learning rate is 1e -3 At , the plateau scheduler reduces the learning rate and sets a patience rate of 2 epochs. In some embodiments, the variety of training images may be increased by augmenting them with flipping, rotation, and Gaussian blurring.

[0083] As shown in Table 3, incorporating a convolutional neural network in the encoder, e.g., by further using a pre-trained convolutional neural network, an improved Dice coefficient (2 × intersections / junctions) is obtained compared to clinician-marked aneurysms. When the two contours completely overlap, the Dice coefficient is maximum, 1. [Table 3]

[0084] The tool can be used to detect, segment, and measure aneurysms in a variety of locations, including basilar, middle cerebral artery, anterior communicating artery, and internal carotid artery (terminal carotid artery) aneurysms.

[0085] As shown in Table 3, when the presented network works in a semi-automatic setting for a segmentation-only problem, the average Dice coefficient for 15 folds is 0.904 with a standard deviation of 0.062. UNet segmentation based on the EfficientNet encoder can be performed from scratch, without pre-training, and achieves a score of 0.864 ± 0.091 (second row of Table 3). The results in Table 3 show that the EfficientNet architecture and pre-trained weights improve the results. In addition to showing higher accuracy than the UNet architecture, which was not shown by the VGG architecture or UNet as a pixel-by-pixel classifier pre-trained on ImageNet, the presented embodiment differs from the detection and segmentation by the UNet structure and LSTM architecture to model the changes in the image when a contrast agent is introduced. Table 3 also shows the results of 3D segmentation using the neural network described herein.

[0086] FIG. 6A illustrates a contour 602 based on the segmentation of the aneurysmal sac detected using the model structure described in connection with FIG. 5. The second line 604 shows the segmentation by the UNet without the EfficientNet encoder arm, which may be referred to as the comparative ground truth contour. FIG. 6A illustrates the improved segmentation on the aneurysmal image provided by the model structure of FIG. 5. FIG. 6B illustrates the detection and segmentation of the aneurysmal sac by the presented network. The first line 612 shows the aneurysm detected and segmented by the architecture of FIG. 5, and the second line 614 shows the comparative ground truth contour. The proposed segmentation approach provides reliable automated measurements similar to those in clinical settings using 2D images. The robustness of the method is evident from the fact that the neural network produced accurate contours even on raw angiogram images, as shown in FIG. 6B.

[0087] The detection+segmentation network provides a semi-automatic alternative approach to marking the aneurysmal sac and can provide the entire image as input to the network. The automatic identification and segmentation is provided to the user, such as the INR, and the user can accept or reject the contours provided by the neural network. In some embodiments, if the user rejects the detection+segmentation result, the user can provide additional input, such as marking the region of the sac, and the image is segmented by a segmentation-only network or feature rather than the identification+segmentation feature. In some embodiments, if the detection and / or segmentation is rejected by the user, the rejection can be provided as feedback to the model training component, such as shown at 319 in FIG. 3.

[0088] In addition to the 2D segmentation algorithm, a 3D axial stack segmentation algorithm may be employed. In such an embodiment, an extended version of a 3D UNet (e.g., AH-Net) may be used to segment the 3D data. As an example, the architecture may include a pre-trained ResNet 50 architecture to generate the encoder features. The feature decoder may use anisotropic 3D convolutions to learn contextual information across the 2D slice-by-slice features generated in the encoder. This allows the network to leverage a well-trained 2D feature generator even for complex 3D tasks. A pre-processing step for semi-automatic extraction of heuristically determined cubes around the aneurysm may be based on input from a user, such as a radiologist. As an example to illustrate the concept, this step may include a PyTorch implementation of the AHNet architecture. Some extensions with random affine and / or random anisotropic may be used. K-fold cross-validation (k=15, training: 38.27±0.44, validation: 2.73±0.44) can be performed with an image size of 60x60x60, 12 batches for 30 epochs, 1e -3 The network may be run with a learning rate of . The Dice coefficient loss may be used in conjunction with deep learning techniques to optimize the network. Table 3 shows an example of a 3D segmentation result performed according to the aspects presented herein. FIG. 7A shows an example image 700 of a 3D mesh model 706 of an aneurysm sac reconstructed from the segmented contours and can be input to the model of the present disclosure. FIG. 7B shows an example result 725 of the presented 3D neural network for segmentation of a detected aneurysm. Line 702 shows the contour output by the automated system, and line 704 shows the ground truth or baseline contour, e.g., corresponding to the 3D baseline of Table 3.

[0089] Accurately measuring the dimensions of the aneurysm sac is a critical step in treatment planning for cerebral aneurysms. The embodiments presented herein provide semi-automated and fully automated methods for segmenting the sac based on imaging and / or clinical information that is input into a machine learning model. The embodiments further allow for automatic identification of a treatment device or potential treatment devices to be implanted in the aneurysm sac. The embodiments also provide a prediction of outcomes associated with the identified treatment device or devices. The decision support software system provides a tool to aid in the rapid identification of promising treatments. The automated detection and / or segmentation, identification of potential treatment devices (e.g., including size or model), and / or information regarding the predicted outcome of treatment provide additional tools for the user to consider when developing a treatment plan. The automated embodiments can reduce the time it takes to identify, segment, and identify treatment devices, and increase the pace at which treatment is delivered to patients. The use of neural networks and machine learning components also allows for additional consideration of other historical data, including past aneurysm data, in the detection, identification, and segmentation. The feedback aspect allows the machine learning model to continually improve the output provided to the user. The combination of image data and clinical information allows for more accurate and efficient estimations and predictions. The increased efficiency and accuracy provide a valuable tool for users in developing treatment plans.

[0090] 8A shows a flow chart 800 for a method or algorithm for providing an outcome prediction of an intrasaccular implant device based on images (e.g., angiographic images) and clinical information from a neural network device. The neural network device may correspond to aneurysm treatment module 220 of FIG. 2 or aneurysm treatment module 302 of FIG. 3, and may include aneurysm auxiliary component 1075 of FIG. 10.

[0091] At 802, the neural network device receives digital image information and clinical information of the aneurysm patient as input from a user device. As an example, the input can be received via one or more of the user data interface component 222 or the query component 224 of the aneurysm treatment module 220. The input can be received at a user interface, for example, as described with respect to FIG. 4A. In some embodiments, the input can be received directly from an image measurement device or image capture device, as shown in FIG. 2. In some embodiments, the image capture device or image measurement device may comprise a neural network. In some embodiments, image data, anatomical data, and / or clinical data may be uploaded to the module 220 / 302 as an encoder input. In some embodiments, the input may be received as a user query for outcome prediction information.

[0092] At 804, the neural network device generates an outcome prediction for at least one intrasaccular implant device for implantation into the aneurysmal sac identified in the digital image information, e.g., an implant device predicted to have the highest probability of complete occlusion of the aneurysmal sac from a group of possible treatment devices. The outcome prediction is generated using a neural network trained for aneurysm outcome prediction, the digital image information, and the clinical information. The neural network can include an encoder made of an image classification neural network (e.g., the image classification neural network may include a convolutional neural network or other types of NNs such as RNNs, transfer learning models, and / or multi-layer perceptron neural networks), and the classification neural network may be pre-trained for object classification such as, but not limited to, a vessel wall or an aneurysm. The outcome prediction is generated by the neural network and may include a size or model of the intrasaccular device predicted to have the highest probability of complete occlusion of the aneurysmal sac based on the received digital image information and / or clinical information. The intrasaccular device predicted to have the highest probability of complete occlusion may be the device with the highest percentage of achieving complete occlusion compared to the percentage of other devices considered by the model. Alternatively, the outcome prediction generated by the neural network may include a list of intracapsular device sizes and the corresponding probability of total occlusion for each size.

[0093] The neural network may be comprised of a classification algorithm based on at least one of a random forest algorithm, an MLP neural network algorithm, a logistic regression algorithm, a naive Bayes machine learning algorithm, or an SVM algorithm. The outcome prediction may be generated, for example, by the neural network component 226, the outcome prediction component 234, or the neural network 306, as examples. The generation may further include any of the aspects described with respect to Figures 1A-7B.

[0094] At 806, the neural network apparatus outputs the identification of the at least one intracapsular implant device and each outcome prediction for the at least one intracapsular implant device for display on a device in response to input from the user device. This device may be the display of the user device that sent the input or may be an additional or separate device. As an example, the output may be provided to the user device via the user data interface component 222. The output may be presented to a user of the user device on a display. An example of a user interface for displaying the output on a user device is shown in FIG. 4B.

[0095] In some aspects, the neural network device may be further configured to obtain one or more measurements of the aneurysm sac by performing semi-automatic segmentation of the digital image information based on the annotations of the digital image information, passing the digital image information to an encoder to obtain a code, and outputting one or more measurements of the aneurysm sac based on the code via a decoder.

[0096] In some aspects, the neural network device may be further configured to obtain one or more measurements of the aneurysm sac by performing an automatic segmentation of the raw image information to identify the aneurysm sac, passing the digital image information to an encoder to obtain a code, and outputting one or more measurements of the aneurysm sac based on the code via a decoder.

[0097] Each outcome prediction for the at least one intrasaccular implant device may be based on one or more measurements of the aneurysm sac, dimensions of the at least one intrasaccular implant device, and clinical information of the aneurysm patient.

[0098] 8B shows a flow chart 810 of a method or algorithm for providing aneurysm sac segmentation information obtained from a neural network device. The neural network device may correspond to aneurysm treatment module 220 of FIG. 2 or aneurysm treatment module 302 of FIG. 3, and may include aneurysm auxiliary component 1075 of FIG. 10.

[0099] At 812, the neural network device receives digital image information (e.g., CT images, ultrasound images, etc., among others) of the patient. In some aspects, the digital information may be angiography data. The image classification neural network (e.g., CNN, RNN, among others) may be pre-trained to classify objects other than aneurysms. The input may be received at a user interface, for example, as described with respect to FIG. 4A. As an example, the input may be received via one or more of the user data interface component 222 or the query component 224 of the aneurysm treatment module 220.

[0100] An example of a neural network architecture is described in relation to FIG. 5. The trained neural network model may be trained from scratch to segment the aneurysmal sac in the image and may further include a decoder configured to output measurement information of the aneurysmal sac after the digital information is processed by the encoder. The digital image information may include one or more two-dimensional DSA images of the wide-necked bifurcation aneurysm prior to placement of the intrasaccular device. The digital information may include one or more of lateral and anterior-posterior perspectives on the two-dimensional DSA images or a three-dimensional axial slice stack reconstructed from a series of DSA images.

[0101] At 814, the neural network device segments the aneurysm sac in the digital image information using a trained neural network model having an encoder with an object classification neural network (e.g., CNN, RNN, among others) pre-trained for object classification. The digital information may include annotations or adjustments that identify the aneurysm sac, and the neural network device may semi-automatically segment the aneurysm sac using the trained neural network model. The digital information may include raw angiographic images, and the neural network device may identify the presence of the aneurysm sac in the raw angiographic images using the trained neural network model prior to the automatic segmentation of the aneurysm sac. The segmentation may be performed, for example, by the neural network component 226, the outcome prediction component 234, or the neural network 306. The neural network device may further perform any of the aspects described with respect to FIGS. 1A-7B.

[0102] At 816, the neural network device outputs the aneurysm sac segmentation information. As an example, the output may be provided to a user device via the user data interface component 222. The output may be presented to the user on a display of the user device, or on a different additional display.

[0103] 8A shows a flow chart 900 for a method or algorithm for providing an outcome prediction for an intrasaccular implant device based on digital image information and clinical information. The method may be performed by a system or service corresponding to aneurysm treatment module 220 of FIG. 2 or aneurysm treatment module 302 of FIG. 3, and may include aneurysm auxiliary component 1075 of FIG. 10.

[0104] At 902, the system receives at least one of image information or clinical information associated with an aneurysm patient as input from a user device. As an example, the input may be received via one or more of the user data interface component 222 or the query component 224 of the aneurysm treatment module 220. The input may be received at a user interface, for example, as described with respect to FIG. 4A. In some aspects, the input may be received directly from an image measurement device or image capture device, as shown in FIG. 2. In some aspects, the image capture device or image measurement device may comprise a neural network. In some aspects, the input may be received as a user query for outcome prediction information.

[0105] The input may include both imaging information and clinical information associated with the aneurysm patient, and the system may generate an outcome prediction for the aneurysm treatment based on both the imaging information and the clinical information. Clinical information may include at least one of demographic information of the aneurysm patient, aneurysm information associated with the imaging information, dimensional information of the aneurysm imaged in the imaging information, allergies of the aneurysm patient, medication information of the aneurysm patient, and pre-existing medical information of the aneurysm patient. Various examples of clinical information are described, for example, in connection with Table 2. The imaging information may include one or more of MRI information, MRA information, CT scan information, 2-dimensional DSA information, or a 3-dimensional reconstruction from a series of 2-dimensional images.

[0106] In some aspects, the input may include image information alone, and the system may generate an outcome prediction for the aneurysm treatment based solely on the image information.

[0107] At 904, the system generates an outcome prediction for the aneurysm treatment for the aneurysm patient based on at least one of the image information or clinical information received as input. The system may include a neural network trained for aneurysm outcome prediction as described with respect to any of FIGs. 2, 3, 4C, or 5. The system may process the image information and the clinical information using the neural network to generate the outcome prediction for the aneurysm treatment. The neural network may include an encoder and a decoder, and the encoder may comprise a convolutional neural network pre-trained for object classification. The neural network may be comprised of a classification algorithm based on at least one of a random forest algorithm, an MLP neural network algorithm, a logistic regression algorithm, a naive Bayes machine learning algorithm, or an SVM algorithm.

[0108] At 906, the system transmits the aneurysm treatment outcome prediction to the user device for display on the device. This device may be the display of the user device that transmitted the input, or may be an additional or separate device. The outcome prediction may be associated with the treatment device, which may include an intrasaccular implant device. There may be a group of possible intrasaccular implant devices of different sizes and / or types. The system may identify a subset of one or more devices that have a high probability of a complete occlusion outcome after placement in the aneurysm sac. The outcome prediction may indicate a predicted likelihood of complete occlusion of the aneurysm sac imaged in the image information. The system may output, for example, a best predicted size that has the highest success rate of complete occlusion of the aneurysm sac. As an example, the output may be provided to the user device via the user data interface component 222. The output may be presented on a display to a user of the user device. An example of a user interface for displaying the output on a user device is shown in FIG. 4B.

[0109] As shown at 912, the system may be further configured to transmit one or more measurements of the aneurysm imaged in the image information to the device (e.g., for display on the device). This device may be a display of the user device that transmitted the input, or may be an additional or separate device. For example, the system may be configured to perform segmentation as described with respect to any of FIGS. 2, 3, 5, or 8B. The segmentation may consist of semi-automatic segmentation and / or automatic segmentation and may include defining a contour surrounding the image of the aneurysm sac. For example, the image information may include at least one annotation identifying the region of the aneurysm, and the segmentation may include semi-automatic segmentation upon receipt of the input image information. As another example, the image information may include raw image information, and the system may be configured to transmit to the device an identification of the presence of an aneurysm in the image information in addition to the one or more measurements, for example, as shown at 910. The identification may include a contour showing the outline of the aneurysm sac imaged in the image information. Following the identification, the system may transmit one or more measurements, which may be referred to as automatic segmentation, for example, to the device at 912. This device may be a display of the user device that sent the input, or may be an additional or separate device. The outcome prediction at 906 may include one or more measurements of the aneurysm imaged in the image information and / or a predicted best size for the intrasaccular device.

[0110] As shown at 908, the system may transmit to the user device an identification of at least one treatment device for placement in the aneurysm patient's aneurysm sac based on at least one of the imaging information or clinical information of the aneurysm patient, the at least one treatment device being identified based on a higher predicted likelihood of complete occlusion of the aneurysm sac imaged in the imaging information of the aneurysm patient than other treatment devices in the group of possible treatment devices. This device may be a display of the user device that transmitted the input, or may be an additional or separate device. In some aspects, the identification may include a list of the multiple treatment devices and a respective outcome prediction associated with each treatment device in the list of the multiple treatment devices. FIG. 4B illustrates an exemplary user interface displaying the identified treatment devices and the associated outcome prediction. In some aspects, the multiple treatment devices may include different sizes of the same type of intrasaccular implant device having the most favorable outcome prediction from the group of possible treatment devices. The multiple treatment devices may include different types of aneurysm treatment devices.

[0111] The algorithm may further include any of the aspects described with respect to FIG. 4C, FIG. 8A or FIG. 8B.

[0112] FIG. 9B illustrates a flow chart 950 for a method or algorithm for obtaining an outcome prediction of an intrasaccular implant device based on image information and clinical information of an aneurysm patient. The method may be performed, for example, by the user device 204 of FIG. 2 or the user device 304 of FIG. 3. At 952, the user device provides a request to a neural network module (e.g., such as module 220 or 302) related to at least one of digital image information or clinical information of the aneurysm patient. In some aspects, the request may be based on sending the digital imager or clinical information to the neural network module. In another aspect, the request may be sent in response to a user selection in a user interface of the user device, for example, as described with respect to FIG. 4A.

[0113] At 954, the user device receives in response from the neural network module at least one of an outcome prediction of the aneurysm treatment, a first identification of the aneurysm in the image information, one or more measurements of the aneurysm, or a second identification of at least one treatment device for placement in the aneurysm sac. The response may include, for example, any of the information described with respect to the information sent to the device at 316, 806, 816, 906, 908, 910, and / or 912.

[0114] At 956, the user device may further display the outcome prediction, the first identification, the second identification, or one or more measurements in a user interface. FIG. 4B illustrates examples of aspects that may be presented in a user interface. In some aspects, the user may accept or reject one or more aspects provided as a result. In some aspects, the user interface may include functionality that allows the user to input an acceptance or rejection of information, such as an aneurysm sac identification or measurement. The user input may be provided as feedback to the neural network model.

[0115] FIG. 10 is a block diagram illustrating an exemplary computer system 1020 illustrating aspects of systems and methods for providing aneurysm image segmentation, aneurysm identification, and outcome prediction using neural networks, for example, as disclosed herein. The computer system 1020 may correspond to a physical server on which the application 1017 executes in some aspects. The computer system 1020 may be part of or comprise an imaging device that acquires imaging information of the aneurysm. The computer system may be connected, for example, directly or via a local or internet network to a device that acquires image data, such as those described above. In some aspects, the computer system may be connected, for example, directly or via a local or internet network to a user device. The computer system 1020 may include an auto-aneurysm auxiliary component 1075 configured to perform aspects described with respect to FIGS. 1A-9A. As an example, the neural network component 226 may be configured to perform any of the aspects described with respect to the algorithms of Figures 4C, 5, 8A, 8B, and / or 9A, as well as any of the aspects described with respect to the aneurysm treatment module 220 of Figure 2 or the aneurysm treatment module 302 of Figure 3, and / or receive inputs described with respect to Figure 4A, or provide any of the outputs described with respect to Figures 4B, 6A, 6B, 7A, or 7B. In some aspects, one or more aspects of the computer system 1000 may be configured to perform the aspects described with respect to Figure 9B and may be executed by the user device 204 and / or the user device 304.

[0116] As shown, computer system 1020 (which may be a personal computer or a server) includes a central processing unit 1021, a system memory 1022, and a system bus 1023 that connects various system components including the memory associated with the central processing unit 1021. As can be appreciated by those skilled in the art, system bus 1023 may be comprised of a bus memory or bus memory controller, a peripheral bus, and a local bus that can interface with any other bus architecture. The system memory may include a permanent memory (ROM) 1024 and a random access memory (RAM) 1025. A basic input / output system (BIOS) 1026 may store basic instructions for transferring information between elements of computer system 1020, such as when loading an operating system via ROM 1024.

[0117] The computer system 1020 may also include a hard disk 1027 for reading and writing data, a magnetic disk drive 1028 for reading and writing data from a removable magnetic disk 1029, and an optical drive 1030 for reading and writing data from a removable optical disk 1031, such as a CD-ROM, DVD-ROM, or other optical media. The hard disk 1027, the magnetic disk drive 1028, and the optical drive 1030 are connected to the system bus 1023 via a hard disk interface 1032, a magnetic disk interface 1033, and an optical drive interface 1034, respectively. The drives and corresponding computer information media are non-power independent modules for storing computer instructions, data structures, program modules, and other data for the computer system 1020.

[0118] An exemplary embodiment includes a system using a hard disk 1027, a removable magnetic disk 1029, and a removable optical disk 1031 connected to a system bus 1023 via a controller 1055. Those skilled in the art will appreciate that any type of medium 1056 capable of storing data in a computer readable form may be utilized, such as a solid state drive, a flash memory card, a digital disk, random access memory (RAM), etc.

[0119] The computer system 1020 has a file system 1036 on which an operating system 1035 may be stored, as well as additional program applications 1037, other program modules 1038, and program data 1039. A user of the computer system 1020 may enter commands and information using a keyboard 1040, a mouse 1042, or any other input device known to those skilled in the art, such as, but not limited to, a microphone, joystick, game controller, scanner, etc. Such input devices are typically connected to the computer system 1020 through a serial port 1046, which is connected to the system bus, although those skilled in the art will appreciate that the input devices may also be connected in other manners, such as, but not limited to, a parallel port, a game port, or a universal serial bus (USB). A monitor 1047 or other type of display device may also be connected to the system bus 1023 via an interface, such as a video adapter 1048. In addition to the monitor 1047, personal computers may also include other peripheral output devices (not shown), such as speakers, printers, etc.

[0120] Computer system 1020 may operate in a networked environment using network connections to one or more remote computers 1049. The remote computer(s) 1049 may be a local computer workstation or server that includes most or all of the elements previously described in describing the nature of computer system 1020, and may include, for example, applications 1037 and 1037'. Other devices, such as, but not limited to, routers, network stations, peer devices, or other network nodes, may also be present in the computer network.

[0121] The network connections can form local area computer networks (LAN) 1050 and wide area computer networks (WAN). Such networks are used in corporate computer networks, intranets within a company, and are generally accessible to the Internet. In a LAN or WAN network, the computer system 1020 is connected to the local area network 1050 via a network adapter or network interface 1051. When using a network, the computer system 1020 may use a modem 1054 or other modules known to those skilled in the art that allow communication with a wide area computer network, such as the Internet. The modem 1054 (which may be an internal or external device) may be connected to the system bus 1023 by a serial port 1046. Those skilled in the art will appreciate that the above network connections are non-limiting examples of the many well-understood ways of establishing a connection from one computer to another using a communication module.

[0122] In various aspects, the systems and methods described herein may be implemented in hardware, software, firmware, or any combination thereof. When implemented in software, the methods may be stored as one or more instructions or codes on a non-transitory computer-readable medium. Computer-readable media include data storage. As a non-limiting example, such computer-readable media may include RAM, ROM, EEPROM, CD-ROM, flash memory, or other types of electrical, magnetic, or optical storage media, or any other medium that can be used to carry or store desired program code in the form of instructions or data structures and that can be accessed by a processor of a general-purpose computer.

[0123] In various aspects, the systems and methods described in this disclosure can be viewed in terms of modules. The term "module" as used herein refers to a real-world device, component, or arrangement of components implemented using hardware, such as by an application specific integrated circuit (ASIC) or field programmable gate array (FPGA), or as a combination of hardware and software, such as by a microprocessor system and a set of instructions for implementing the functionality of the module that (when executed) transforms the microprocessor system into a special-purpose device. A module may also be implemented as a combination of the two, with certain functions realized only by hardware and other functions realized by a combination of hardware and software. In certain implementations, at least some, and possibly all, of the modules may be executed on the processor of a general-purpose computer. Thus, each module may be implemented in a variety of suitable configurations and should not be limited to any particular implementation illustrated herein.

[0124] In one configuration, aneurysm auxiliary component 1075 and / or computer system 1020, particularly file system 1036 and / or processor (e.g., 1021), or one or more interfaces provide means for performing any of the aspects described with respect to the flowcharts of Figures 4C, 5, 8A, 8B, and / or 9A, as well as any of the aspects described with respect to aneurysm treatment module 220 of Figure 2 or aneurysm treatment module 302 of Figure 3, and / or receiving inputs described in Figure 4A, or providing any of the outputs described with respect to Figures 4B, 6A, 6B, 7A, or 7B. In some aspects, one or more aspects of system 1000 may be configured to perform aspects described with respect to Figure 9B, may be executed by user devices 204 and / or 304, and may correspond to means for performing aspects described with respect to Figure 9B and user devices 204 and / or 304.

[0125] One or more elements, aspects, or steps, or any portions thereof, from any one or more of the systems and methods described herein may be combined with one or more elements, aspects, or steps, or any portions thereof, from any one or more of the other systems and methods described herein to form one or more additional implementations and / or claims of the present disclosure.

[0126] One or more of the illustrated components, steps, features, and / or functions may be rearranged and / or combined into a single component, block, feature, or function, or embodied in several components, steps, or functions. Additional elements, components, steps, and / or functions may be added without departing from the disclosure. The illustrated apparatus, devices, and / or components may be configured to perform one or more of the methods, features, or steps described in the figures. The algorithms described herein may be efficiently implemented in software and / or embedded in hardware.

[0127] The term "one embodiment" or "embodiment" as used herein means that a particular feature, structure, or characteristic described in connection with the embodiment is included in at least one embodiment of the invention. The appearances of the phrase "in one embodiment" in various places in the specification are not necessarily all referring to the same embodiment.

[0128] Some portions of the detailed descriptions are presented in terms of algorithms and symbolic representations of operations on data bits within a computer memory. These algorithmic descriptions and representations are the means used by those skilled in the data processing arts to most effectively convey the substance of their work to others skilled in the art. An algorithm is here, and generally, conceived to be a self-consistent sequence of steps leading to a desired result. The steps are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven useful at times, primarily for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0129] It should be noted, however, that all of these and similar terms are associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless otherwise specified, as will be apparent from the disclosure that follows, disclosure terms such as "processing," "computing," "calculating," "determining," "displaying," and the like, will be understood to refer to operations and processes of a computer system, or similar electronic computing device, to manipulate and transform data represented as physical (electronic) quantities in the registers and memory of the computer system into other data, also represented as physical quantities in the memory or registers of the computer system, or to store, transmit, or display other such information.

[0130] Finally, the algorithms and displays presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove useful to construct more specialized apparatus to perform the required method steps. The required structure for a variety of these systems will appear from the description that follows. It will be understood that a variety of programming languages ​​may be used to implement the teachings of the invention as described herein.

[0131] The figures and the following description describe specific embodiments by way of example only. Those skilled in the art will readily recognize from the following description that alternative embodiments of the structures and methods illustrated herein are possible without departing from the principles described herein. Reference will now be made in detail to several embodiments that are illustrated in the accompanying drawings. It is noted that, wherever possible, similar or identical reference numbers may be used in the drawings to indicate similar features.

[0132] The foregoing description of the embodiments of the present invention has been presented for purposes of illustration and description. It is not intended to be exhaustive or to limit the invention to the precise form disclosed. Many modifications and variations are possible in light of the above teachings. The scope of the present invention is limited not by this detailed description, but by the claims of this application. As will be understood by those skilled in the art, the present invention may be embodied in other specific forms without departing from its spirit or essential characteristics. Similarly, the specific naming and division of modules, routines, features, attributes, methodologies and other aspects is not required or important, and mechanisms for implementing the present invention or its features may have different names, divisions and / or formats.

[0133] Moreover, as will be apparent to one of ordinary skill in the relevant art, the modules, routines, features, attributes, methodologies and other aspects of the invention may be implemented as software, hardware, firmware, or any combination of the three. Also, when a component, such as a module, of the invention is implemented as software, the component may be implemented as a stand-alone program, as part of a larger program, as multiple separate programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and / or in any other manner now or in the future known to those of ordinary skill in the art of computer programming.

[0134] Additionally, the present invention is not limited to implementation in any specific programming language, or for any specific operating system or environment. Accordingly, the present disclosure is intended to be illustrative, but not limiting, of the scope of the invention, which is set forth in the following claims.

[0135] It will be understood that the particular order or hierarchy of the blocks in the disclosed processes / flowcharts is illustrative of example approaches. Based on design preferences, it will be understood that the particular order or hierarchy of the blocks within the processes / flowcharts may be rearranged. Additionally, some blocks may be combined or omitted. The accompanying methods present elements of the various blocks in a sample order and are not limited to the particular order or hierarchy presented.

[0136] The foregoing description is provided to enable those skilled in the art to practice the various aspects described herein. Various modifications to these aspects will be readily apparent to those skilled in the art, and the general principles defined herein may be applied to other aspects. Thus, the claims are not intended to be limited to the aspects set forth herein but are to be accorded full scope consistent with the scope of the language claims, and references to elements in the singular are not intended to mean "only one" unless otherwise specified, but rather "one or more." As used herein, the word "exemplary" is used to mean "serving as an example, instance, or illustration." Any aspect described herein as "exemplary" should not necessarily be construed as preferred or advantageous over other aspects. Unless otherwise specified, the term "some" refers to one or more. Combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" include any combination of A, B, and / or C, and may include multiples of A, multiples of B, or multiples of C. Specifically, combinations such as "at least one of A, B, or C," "one or more of A, B, or C," "at least one of A, B, and C," "one or more of A, B, and C," and "A, B, C, or any combination thereof" may include only A, only B, only C, A and B, A and C, B and C, or only A and C, and such combinations may include one or more members of A, B, or C. All structural and functional equivalents of the elements of the various embodiments described throughout this disclosure that are known or later become known to those of ordinary skill in the art are expressly incorporated herein by reference and are intended to be encompassed by the claims. Furthermore, nothing disclosed herein is intended to be made publicly available, regardless of whether such disclosure is expressly set forth in the claims.Terms such as "module," "mechanism," "element," "device," and the like are not substitutes for the term "means." Thus, no claim element should be construed as a means plus function unless the element is expressly recited using the phrase "means for."

[0137] The following aspects are illustrative only and may be combined, without limitation, with other aspects or teachings described herein.

[0138] Aspect 1 is a method for providing an outcome prediction for an intrasaccular implant device in a neural network device based on digital images and clinical information, the method comprising: receiving digital image information and clinical information of an aneurysm patient as input from a user device; using a neural network trained for aneurysm outcome prediction, generating an outcome prediction for at least one intrasaccular implant device from a group of treatment devices using the digital image information and the clinical information that is predicted to be most likely to be placed in an aneurysm sac identified in the digital image information and completely occlude the aneurysm sac; identifying the at least one intrasaccular implant device and outputting and displaying each outcome prediction for the at least one intrasaccular implant device on the device.

[0139] In an aspect 2, the method of aspect 1, wherein the neural network is configured with a classification algorithm based on at least one of a random forest algorithm, a multi-layer perceptron (MLP) neural network algorithm, a logistic regression algorithm, a naive Bayes machine learning algorithm, or a support vector machine (SVM) algorithm.

[0140] Aspect 3 is the method of aspect 1 or 2, further comprising at least one of: obtaining the one or more measurements of the aneurysm sac by performing a semi-automatic segmentation of the digital image information, passing the digital image information to an encoder to obtain a code, and outputting one or more measurements of the aneurysm sac based on the code via a decoder; and obtaining the one or more measurements of the aneurysm sac by performing an automatic segmentation of raw image information, identifying the aneurysm sac, passing the digital image information to the encoder to obtain the code, and outputting the one or more measurements of the aneurysm sac based on the code via the decoder; and each of the outcome predictions of the at least one intrasaccular implant device is based on the one or more measurements of the aneurysm sac, dimensions of the at least one intrasaccular implant device, and the clinical information of the aneurysm patient.

[0141] Aspect 4 is a neural network apparatus for providing an outcome prediction for an intracapsular implant device based on digital image information and clinical information, the apparatus comprising a memory and at least one processor coupled to the memory, the processor configured to perform any of the methods of aspects 1-3 based at least in part on the information stored in the memory.

[0142] Aspect 5 is a neural network apparatus for providing outcome prediction of an intracapsular implant device based on digital image information and clinical information, the apparatus including means for performing any of the methods of aspects 1-3.

[0143] Aspect 6 is a non-transitory computer readable storage medium having stored thereon computer executable code for providing an outcome prediction of an intracapsular implant device in a neural network device based on digital images and clinical information, the code, when executed by a processor, causing the processor to perform any of the methods of aspects 1-3.

[0144] Aspect 7 is a method for providing an outcome prediction for an intrasaccular implant device in a system based on image information and clinical information, the method comprising receiving at least one of image information or clinical information related to an aneurysm patient as input from a user device, generating an outcome prediction for an aneurysm treatment for the aneurysm patient based on the at least one of the image information or the clinical information received as input, and transmitting the outcome prediction for the aneurysm treatment to a display for display on the user device.

[0145] In aspect 8, the system includes a neural network trained for aneurysm outcome prediction, and the method of aspect 7 comprises using the neural network to process the image information and the clinical information to generate the outcome prediction of the aneurysm treatment.

[0146] Example 9 is the method of example 7 or 8, wherein the neural network includes an encoder and a decoder, and the encoder comprises an image classification neural network pre-trained for object classification.

[0147] Aspect 10 is the method of any of aspects 7 to 9, wherein the neural network is configured with a classification algorithm based on at least one of a random forest algorithm, a multi-layer perceptron (MLP) neural network algorithm, a logistic regression algorithm, a naive Bayes machine learning algorithm, or a support vector machine (SVM) algorithm.

[0148] Example 11 is a method of any of Examples 7-10, wherein the input includes both the imaging information and the clinical information related to the aneurysm patient, and the method comprises generating an outcome prediction for the aneurysm treatment based on both the imaging information and the clinical information.

[0149] Aspect 12 is any of the methods of aspects 7 to 11, wherein the clinical information includes at least one of demographic information of the aneurysm patient, aneurysm information related to the image information, dimensional information of the aneurysm imaged in the image information, allergies of the aneurysm patient, medication information of the aneurysm patient, and medical history information of the aneurysm patient.

[0150] Example 13 is any of the methods of Examples 7-11, wherein the outcome prediction includes at least one of one or more measurements of the aneurysm imaged in the imaging information, or a predicted best size of an intrasaccular device for the aneurysm imaged in the imaging information.

[0151] Example 14 is the method of example 13, wherein the image information includes at least one annotation identifying a region of the aneurysm.

[0152] Example 15 is the method of example 13, wherein the image information includes raw image information, and the method further comprises transmitting to the user device, in addition to the one or more measurements, an identification of the presence of the aneurysm in the image information.

[0153] Example 16 is the method of example 15, wherein the identification includes a contour indicative of an outline of an aneurysmal sac imaged in the image information.

[0154] Aspect 17 is any of the methods of aspects 7-9, further comprising transmitting to a user device an identification of at least one treatment device to be placed in the aneurysm sac of the aneurysm patient based on at least one of the image information of the aneurysm patient or the clinical information, wherein the at least one treatment device is identified based on having the highest predicted likelihood of complete occlusion of the aneurysm sac imaged in the image information of the aneurysm patient among a group of possible treatment devices.

[0155] Example 18 is the method of Example 17, wherein the identifying includes a list of a plurality of therapeutic devices and a respective outcome prediction associated with each therapeutic device in the list of a plurality of therapeutic devices.

[0156] Example 19 is the method of Example 18, wherein the plurality of treatment devices includes different sizes of intracapsular implant devices of the same type having a most favorable outcome prediction from the group of possible treatment devices.

[0157] Example 20 relates to the method of Example 18, wherein the plurality of treatment devices includes different types of aneurysm treatment devices.

[0158] Example 21 is the method of any of Examples 7-20, wherein the outcome prediction comprises the size of the intracapsular device that has the highest likelihood of total occlusion.

[0159] Example 22 is any of the methods of Examples 7 to 21, wherein the image information includes any of magnetic resonance imaging (MRI) information, magnetic resonance angiography (MRA) information, computed tomography (CT) scan information, two-dimensional (2D) digital subtraction angiography information, and three-dimensional (3D) construction from a series of two-dimensional images.

[0160] Aspect 23 is the method of any of aspects 7-22, wherein the outcome prediction indicates a predicted likelihood of complete occlusion of an aneurysmal sac imaged in the image information.

[0161] Example 24 is an apparatus for providing an outcome prediction for an intracapsular implant device based on imaging and clinical information, the apparatus comprising a memory and at least one processor connected to the memory, the processor configured to perform any of the methods of Examples 7-23 based at least in part on information stored in the memory.

[0162] Example 25 is an apparatus for providing an outcome prediction for an intracapsular implant device based on image information and clinical information, the apparatus including a means for performing any of the methods of Examples 7-23.

[0163] Example 26 is a non-transitory computer-readable storage medium storing computer-executable code for providing an outcome prediction for an intracapsular implant device based on images and clinical information, the code, when executed by a processor, causing the processor to perform any of the methods of Examples 7-23.

[0164] Aspect 27 is a method for providing segmentation information obtained for an aneurysm sac in a neural network device, the method comprising: receiving digital image information of a patient; segmenting the aneurysm sac in the digital image information using a trained neural network model having an encoder with an image classification neural network pre-trained for object classification; and outputting segmentation information of the aneurysm sac.

[0165] Example 28 is the method of example 27, wherein the image classification neural network is pre-trained to classify objects other than aneurysms.

[0166] Example 29 is the method of example 27 or 28, wherein the trained neural network model is further trained to segment an aneurysm sac in an image, and includes a decoder configured to output measurement information of the aneurysm sac after the digital image information is processed by the encoder.

[0167] Example 30 is any of the methods of Examples 27-29, wherein the digital image information includes one or more two-dimensional (2D) digital subtraction angiography (DSA) images of the wide-necked bifurcation aneurysm prior to placement of an intrasaccular device.

[0168] Example 31 is any of the methods of Examples 27-30, wherein the digital image information includes one or more of either a lateral or an anterior-posterior (AP) perspective on a two-dimensional (2D) digital subtraction angiography (DSA) image, or a three-dimensional (3D) axial slice stack reconstructed from a series of DSA images.

[0169] Example 32 is any of the methods of Examples 27 to 31, wherein the digital image information includes annotations or adjustments identifying the aneurysm sac, and the at least one processor is configured to semi-automatically segment the aneurysm sac using the trained neural network model.

[0170] Example 33 is a method of any of Examples 27 to 32, wherein the digital image information includes a raw angiographic image, and the method comprises using the trained neural network model to identify the presence of the aneurysmal sac in the raw angiographic image prior to automatic segmentation of the aneurysmal sac.

[0171] A thirty-fourth aspect is the method of any one of aspects twenty-seven to thirty-three, further comprising outputting a recommended size of an intrasaccular device to be placed in the aneurysm sac.

[0172] Example 35 is the method of any of Examples 27-34, comprising indicating a likelihood of complete occlusion of the aneurysmal sac.

[0173] Example 36 is an apparatus for providing segmentation information obtained for an aneurysmal sac, the apparatus comprising a memory and at least one processor connected to the memory, the processor configured to perform any of the methods of Examples 27-35 based at least in part on information stored in the memory.

[0174] Example 37 is an apparatus for providing segmentation information obtained about an aneurysmal sac, the apparatus including means for performing the method of any of Examples 27-35.

[0175] Example 38 is a non-transitory computer-readable storage medium storing computer-executable code that provides segmentation information obtained about an aneurysmal sac, the code, when executed by a processor, causing the processor to perform any of the methods of Examples 27-35.

[0176] Example 39 is a method of providing an outcome prediction for an intrasaccular implant device based on digital image information and clinical information of an aneurysm patient, the method comprising: providing a request related to at least one of the digital image information or the clinical information of the aneurysm patient to a neural network module; and receiving in response from the neural network module at least one of an outcome prediction for the aneurysm treatment, a first identification of the aneurysm in the image information, one or more measurements of the aneurysm, or a second identification of at least one treatment device for placement in an aneurysm sac.

[0177] Example 40 is the method of example 39, comprising displaying the outcome prediction, the first identification, the second identification, or the one or more measurements in a user interface.

[0178] Example 41 is the method of example 39 or 40, wherein the neural network component is located remotely from the user device, the request is provided to the neural network module, and the response from the neural network module is received at the user device via a communication interface.

[0179] Example 42 is the method of example 39 or 40, wherein the neural network component is included in the user device.

[0180] Example 43 is an apparatus for providing segmentation information obtained for an aneurysmal sac, the apparatus comprising a memory and at least one processor connected to the memory, the processor configured to perform any of the methods of Examples 39 to 42 based at least in part on information stored in the memory.

[0181] Example 44 is an apparatus for providing segmentation information obtained about an aneurysmal sac, the apparatus including a memory and means for performing the method of any of Examples 39-42.

[0182] Example 45 is a non-transitory computer-readable storage medium storing computer-executable code that provides segmentation information obtained about an aneurysmal sac, the code, when executed by a processor, causing the processor to perform any of the methods of Examples 39 to 42.

Claims

1. 1. A neural network apparatus for providing segmentation information obtained about an aneurysmal sac, the apparatus comprising: Memory, at least one processor coupled to the memory, the processor performing, based at least in part on the information stored in the memory: receiving digital image information of a patient; Segmenting the aneurysm sac in the digital image information using a trained neural network model having an encoder with an image classification neural network pre-trained for object classification; and outputting segmentation information of the aneurysm sac.

2. 10. The neural network device of claim 1, wherein the image classification neural network is pre-trained to classify objects other than aneurysms.

3. 2. The neural network device of claim 1, wherein the trained neural network model is further trained to segment an aneurysm sac in an image, and further comprising a decoder configured to output measurement information of the aneurysm sac after the digital image information is processed by the encoder.

4. 2. The neural network apparatus of claim 1, wherein the digital image information includes one or more two-dimensional (2D) digital subtraction angiography (DSA) images of a wide-necked bifurcation aneurysm prior to placement of an intrasaccular device.

5. 2. The neural network device of claim 1, wherein the digital image information includes one or more of either a lateral or an anterior-posterior (AP) view on a two-dimensional (2D) digital subtraction angiography (DSA) image, or a three-dimensional (3D) axial slice stack reconstructed from a series of DSA images.

6. 2. The neural network device of claim 1, wherein the digital image information includes annotations or adjustments that identify the aneurysm sac, and the at least one processor is configured to semi-automatically segment the aneurysm sac using the trained neural network model.

7. 2. The neural network device of claim 1, wherein the digital image information includes raw angiographic images, and the at least one processor further comprises: a neural network device configured to identify the presence of the aneurysmal sac in the raw angiographic image using the trained neural network model prior to automatic segmentation of the aneurysmal sac.

8. 2. The neural network apparatus of claim 1, wherein the at least one processor further comprises: and outputting a recommended size of an intrasaccular device to be placed in the aneurysm sac.

9. 2. The neural network apparatus of claim 1, wherein the at least one processor further comprises: and indicating a likelihood of complete occlusion of the aneurysm sac.

10. 1. A neural network apparatus for providing an outcome prediction for an intracapsular implant device based on digital image information and clinical information, the apparatus comprising: Memory, at least one processor coupled to the memory, the processor performing, based at least in part on the information stored in the memory: receiving digital image information and clinical information of the aneurysm patient as input from a user device; using a neural network trained for aneurysm outcome prediction, generating an outcome prediction using the digital image information and the clinical information for at least one intrasaccular implant device from a group of treatment devices that is predicted to be most likely to be placed in the aneurysm sac identified in the digital image information and to completely occlude the aneurysm sac; and a neural network apparatus configured to output and display on a device an identification of the at least one intracapsular implant device and a respective outcome prediction for the at least one intracapsular implant device.

11. 11. The neural network device of claim 10, wherein the neural network is configured with a classification algorithm based on at least one of a random forest algorithm, a multi-layer perceptron (MLP) neural network algorithm, a logistic regression algorithm, a naive Bayes machine learning algorithm, or a support vector machine (SVM) algorithm.

12. 11. The neural network apparatus of claim 10, wherein the at least one processor performs, at least in part based on information stored in the memory: obtaining one or more measurements of the aneurysm sac by performing a semi-automated segmentation of the digital image information, passing the digital image information to an encoder to obtain a code, and passing the digital image information to a decoder to output one or more measurements of the aneurysm sac based on the code; performing an automated segmentation of raw image information to identify the aneurysm sac, passing the digital image information to the encoder to obtain the code, and outputting the one or more measurements of the aneurysm sac via the decoder based on the code, thereby obtaining the one or more measurements of the aneurysm sac; A neural network apparatus, wherein each of the outcome predictions for the at least one intrasaccular implant device is based on the one or more measurements of the aneurysm sac, dimensions of the at least one intrasaccular implant device, and the clinical information of the aneurysm patient.

13. 1. A system for providing an outcome prediction for an intracapsular implant device based on image and clinical information, the system comprising: Memory, at least one processor coupled to the memory, the processor performing, based at least in part on the information stored in the memory: receiving at least one of image information or clinical information related to the aneurysm patient as input from a user device; generating an outcome prediction for the aneurysm treatment for the aneurysm patient based on at least one of the imaging information or the clinical information received as input; The system is configured to transmit the outcome prediction of the aneurysm treatment to a display for display on a user device.

14. 14. The system of claim 13, wherein the system includes a neural network trained for aneurysm outcome prediction, and the at least one processor is configured to use the neural network to process the image information and the clinical information to generate the outcome prediction of the aneurysm treatment.

15. 15. The system of claim 14, wherein the neural network includes an encoder and a decoder, the encoder comprising an image classification neural network pre-trained for object classification.

16. 16. The system of claim 15, wherein the neural network comprises a classification algorithm based on at least one of a random forest algorithm, a multi-layer perceptron (MLP) neural network algorithm, a logistic regression algorithm, a naive Bayes machine learning algorithm, or a support vector machine (SVM) algorithm.

17. 14. The system of claim 13, wherein the input includes both the imaging information and the clinical information related to the aneurysm patient, and the at least one processor generates the outcome prediction of the aneurysm treatment based on both the imaging information and the clinical information.

18. 14. The system of claim 13, wherein the clinical information comprises: demographic information of the aneurysm patient; aneurysm information associated with the image information; Dimensional information of the aneurysm imaged in the image information; and allergies of the aneurysm patient; Medication information for aneurysm patients; and medical history information of the aneurysm patient.

19. 14. The system of claim 13, wherein the outcome prediction comprises: one or more measurements of the aneurysm imaged in the image information; or a predicted best size of an intrasaccular device for the aneurysm imaged in the image information.

20. 20. The system of claim 19, wherein the image information includes at least one annotation identifying a region of the aneurysm.

21. 21. The system of claim 20, wherein the image information comprises raw image information, and the at least one processor performs, based at least in part on information stored in the memory: The system transmits to the user device an identification of the presence of the aneurysm in the image information in addition to the one or more measurements.

22. 22. The system of claim 21, wherein the identification includes a contour indicative of an outline of an aneurysmal sac imaged in the image information.

23. 14. The system of claim 13, wherein the at least one processor performs, at least in part based on information stored in the memory: The system is configured to transmit to a user device an identification of at least one treatment device for placement in the aneurysm sac of the aneurysm patient based on at least one of the image information of the aneurysm patient or the clinical information, wherein the at least one treatment device is identified based on having the highest predicted likelihood of complete occlusion of the aneurysm sac imaged in the image information of the aneurysm patient among a group of possible treatment devices.

24. 24. The system of claim 23, wherein the identification includes a list of a plurality of therapeutic devices and a respective outcome prediction associated with each therapeutic device in the list of a plurality of therapeutic devices.

25. 25. The system of claim 24, wherein the plurality of treatment devices includes different sizes of intracapsular implant devices of the same type having a most favorable outcome prediction from the group of possible treatment devices.

26. 25. The system of claim 24, wherein the plurality of treatment devices includes different types of aneurysm treatment devices.

27. 14. The system of claim 13, wherein the outcome prediction comprises a size of intracapsular device having the highest likelihood of total occlusion.

28. 14. The system of claim 13, wherein the image information comprises: Magnetic resonance imaging (MRI) information, Magnetic resonance angiography (MRA) information, Computed tomography (CT) scan information, Two-dimensional (2D) digital subtraction angiography information; three-dimensional (3D) construction from a series of two-dimensional images.

29. 14. The system of claim 13, wherein the outcome prediction indicates a predicted likelihood of complete occlusion of an aneurysmal sac imaged in the image information.

Images